Skip to content

Instantly share code, notes, and snippets.

@guerbai

guerbai/spark_setup.py

Created June 2, 2019 04:22
Show Gist options
  • Save guerbai/a8582ee36c42914124dd87a2dbdb88fe to your computer and use it in GitHub Desktop.
Save guerbai/a8582ee36c42914124dd87a2dbdb88fe to your computer and use it in GitHub Desktop.
Download ZIP
spark setup #Jupyter
Raw
spark_setup.py
import findspark
findspark.init()
import pyspark
import random
sc = pyspark.SparkContext(appName="")
sc.stop()
# get shape
print ((df.count(), len(df.columns)))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment