Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
Install Spark on Mac OS X Yosemite
  1. Download spark from http://spark.apache.org/downloads.html

  2. Extract file contents to a directory. E.g., /usr/local/

  3. Modify bash profile:

export SPARK_PATH=/usr/local/spark-X-X-X-bin-hadoop2.7
alias pyspark="$SPARK_PATH/bin/pyspark --master local[4]"
alias snotebook="PYSPARK_DRIVER_PYTHON='jupyter' PYSPARK_DRIVER_PYTHON_OPTS='notebook' pyspark"
  1. Reset your terminal and type:
$ pyspark

Additional Resources:
https://medium.com/data-science-cafe/apache-spark-1-6-0-setup-on-mac-os-x-yosemite-d58076e8064e#.4i6pt2eb4
http://genomegeek.blogspot.com/2014/11/how-to-install-apache-spark-on-mac-os-x.html

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.