Create a gist now

Instantly share code, notes, and snippets.

What would you like to do?
Install Apache Spark

Install Apache Spark (OSX 10.6)

You need the package manager Homebrew (if not installed), see http://brew.sh/

ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

Get Spark 'Source Code':

http://spark.apache.org/downloads.html

Add to .bash_profile:

export SPARK_HOME=/path/to/spark/folder
source ~/.bash_profile

Install the Scala build tool and build Spark (took ~70 min):

cd $SPARK_HOME
brew install sbt
sbt assembly

If Error:

[error] (streaming-flume-sink/compile:compile) java.io.IOException: Cannot run program "/System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/bin/javac": error=2, No such file or directory

Then link the bin folder where javac is to the path searched by sbt:

which javac
/usr/bin/javac
ln -s /usr/bin /System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment