Skip to content

Instantly share code, notes, and snippets.

@genomics-geek

genomics-geek/README.md

Last active December 11, 2016 03:15
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save genomics-geek/c36fcce0c2529662c52e2d00b54a73b4 to your computer and use it in GitHub Desktop.
Save genomics-geek/c36fcce0c2529662c52e2d00b54a73b4 to your computer and use it in GitHub Desktop.
Download ZIP
Setup Apache Spark on MacOSX
Raw
README.md

Guide for setting up Apache Spark on MacOSX

Just for me to remember the steps needed

Download Apache Spark

http://spark.apache.org/downloads.html

Add enviornment variables to bash_profile

export PYSPARK_PYTHON=python3

export SPARK_HOME=$HOME/Resources/spark/spark-2.0.2-bin-hadoop2.7/

export PYTHONPATH=$SPARK_HOME/python/:$PYTHONPATH

Add alias to bashrc

alias pyspark='$SPARK_HOME/bin/pyspark'

alias spark-submit='$SPARK_HOME/bin/spark-submit'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment