Skip to content

Instantly share code, notes, and snippets.

@l1x

l1x/test_spark_python.py

Created August 4, 2017 10:12
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save l1x/c45a93f15b47cf8d84367d61b93e5422 to your computer and use it in GitHub Desktop.
Save l1x/c45a93f15b47cf8d84367d61b93e5422 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
test_spark_python.py
import pandas as pd
import io
import requests
url="https://raw.githubusercontent.com/cs109/2014_data/master/countries.csv"
s=requests.get(url).content
p_df =pd.read_csv(io.StringIO(s.decode('utf-8')))
spark = SQLContext(sc)
s_df = spark.createDataFrame(p_df)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment