Skip to content

Instantly share code, notes, and snippets.

@fspaolo
Last active February 24, 2017 11:44
  • Star 1 You must be signed in to star a gist
  • Fork 1 You must be signed in to fork a gist
Star You must be signed in to star a gist
Save fspaolo/53daaf85318ae89629dd to your computer and use it in GitHub Desktop.
Install Apache Spark

Install Apache Spark (OSX 10.6)

You need the package manager Homebrew (if not installed), see http://brew.sh/

ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

Get Spark 'Source Code':

http://spark.apache.org/downloads.html

Add to .bash_profile:

export SPARK_HOME=/path/to/spark/folder
source ~/.bash_profile

Install the Scala build tool and build Spark (took ~70 min):

cd $SPARK_HOME
brew install sbt
sbt assembly

If Error:

[error] (streaming-flume-sink/compile:compile) java.io.IOException: Cannot run program "/System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/bin/javac": error=2, No such file or directory

Then link the bin folder where javac is to the path searched by sbt:

which javac
/usr/bin/javac
ln -s /usr/bin /System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment