Skip to content

Instantly share code, notes, and snippets.

@rafeco

rafeco/homebrew_hadoop.markdown

Created July 24, 2012 19:43
Show Gist options
  • Save rafeco/3172183 to your computer and use it in GitHub Desktop.
Save rafeco/3172183 to your computer and use it in GitHub Desktop.
Download ZIP
Running Hadoop and Cascading under Homebrew
Raw
homebrew_hadoop.markdown

http://geekiriki.blogspot.com/2011/10/flume-and-hadoop-on-os-x.html

http://ragrawal.wordpress.com/2012/04/28/installing-hadoop-on-mac-osx-lion/

Most instructions for running Hadoop from Homebrew tell you to edit the configuration files in the Hadoop distribution. I'm not a fan of that. Instead, copy the config directory to your home directory, as follows:

cp -r /usr/local/Cellar/hadoop/1.0.3/libexec/conf ~/hadoop-conf

Then edit the files in that directory.

I added these variables to my .bashrc but you could also put them in a wrapper:

export HADOOP_OPTS="-Djava.security.krb5.realm=OX.AC.UK -Djava.security.krb5.kdc=kdc0.ox.ac.uk:kdc1.ox.ac.uk"
export HADOOP_CONF_DIR=~/hadoop-conf
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment