Skip to content

Instantly share code, notes, and snippets.

@alfredrichards

alfredrichards/pyspark-jupyter-notebook-integration

Created August 31, 2019 03:13
Show Gist options
  • Save alfredrichards/5c229aed041662a485cadd631d70668c to your computer and use it in GitHub Desktop.
Save alfredrichards/5c229aed041662a485cadd631d70668c to your computer and use it in GitHub Desktop.
Download ZIP
Raw
pyspark-jupyter-notebook-integration
export SPARK_HOME="/path/to/spark-home"
export PATH="$SPARK_HOME/bin:$PATH"
export PYSPARK_SUBMIT_ARGS="pyspark-shell"
export PYSPARK_DRIVER_PYTHON=ipython
export PYSPARK_DRIVER_PYTHON_OPTS='notebook' pyspark
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment