Create a gist now

Instantly share code, notes, and snippets.

Install Apache Spark

Install Apache Spark (OSX 10.6)

You need the package manager Homebrew (if not installed), see http://brew.sh/

ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

Get Spark 'Source Code':

http://spark.apache.org/downloads.html

Add to .bash_profile:

export SPARK_HOME=/path/to/spark/folder
source ~/.bash_profile

Install the Scala build tool and build Spark (took ~70 min):

cd $SPARK_HOME
brew install sbt
sbt assembly

If Error:

[error] (streaming-flume-sink/compile:compile) java.io.IOException: Cannot run program "/System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/bin/javac": error=2, No such file or directory

Then link the bin folder where javac is to the path searched by sbt:

which javac
/usr/bin/javac
ln -s /usr/bin /System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment