Skip to content

Instantly share code, notes, and snippets.

@crawles

crawles/install_spark_mac.md

Last active January 13, 2017 22:08
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save crawles/3461673f4c69657d91b524bcc8ebf008 to your computer and use it in GitHub Desktop.
Save crawles/3461673f4c69657d91b524bcc8ebf008 to your computer and use it in GitHub Desktop.
Download ZIP
Install Spark on Mac OS X Yosemite
Raw
install_spark_mac.md

  1. Download spark from http://spark.apache.org/downloads.html

  2. Extract file contents to a directory. E.g., /usr/local/

  3. Modify bash profile:

export SPARK_PATH=/usr/local/spark-X-X-X-bin-hadoop2.7
alias pyspark="$SPARK_PATH/bin/pyspark --master local[4]"
alias snotebook="PYSPARK_DRIVER_PYTHON='jupyter' PYSPARK_DRIVER_PYTHON_OPTS='notebook' pyspark"
  1. Reset your terminal and type:
$ pyspark

Additional Resources:
https://medium.com/data-science-cafe/apache-spark-1-6-0-setup-on-mac-os-x-yosemite-d58076e8064e#.4i6pt2eb4
http://genomegeek.blogspot.com/2014/11/how-to-install-apache-spark-on-mac-os-x.html

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment