Skip to content

Instantly share code, notes, and snippets.

@brianspiering

brianspiering/launch_spark_shells.sh

Last active February 20, 2017 23:11
Show Gist options
  • Save brianspiering/a954339840434bdf8ce1 to your computer and use it in GitHub Desktop.
Save brianspiering/a954339840434bdf8ce1 to your computer and use it in GitHub Desktop.
Download ZIP
Launch local Spark REPLs for Spark 2.x
Raw
launch_spark_shells.sh
# PySpark in IPython
PYSPARK_DRIVER_PYTHON=ipython pyspark
# PySpark in Jupyter Notebook backed by Python kernel
PYSPARK_DRIVER_PYTHON=ipython Jupyter Notebook
# Spark with Scala REPL
spark-shell --master local[*]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment