Skip to content

Instantly share code, notes, and snippets.

@iguoli

iguoli/spark_learning.md

Last active April 8, 2018 02:28
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save iguoli/e1d836e7ad94fda2bac86f0eb3bd33d0 to your computer and use it in GitHub Desktop.
Save iguoli/e1d836e7ad94fda2bac86f0eb3bd33d0 to your computer and use it in GitHub Desktop.
Download ZIP
Spark学习，spark, rdd
Raw
spark_learning.md

启动spark交互式shell(python)

# 安装pyspark
$ pip install pyspark

# 设置环境变量
$ export SPARK_HOME=/home/cloudera/py3project/venv/lib/python3.5/site-packages/pyspark
$ export PYSPARK_DRIVER_PYTHON=/home/cloudera/py3project/venv/bin/ipython
$ export PYSPARK_DRIVER_PYTHON_OPTS='notebook'

$ pyspark
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment