Skip to content

Instantly share code, notes, and snippets.

@kovid-r

kovid-r/pyspark_cheatsheet_create_df.py

Last active October 11, 2022 04:49
Show Gist options
  • Star 1 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save kovid-r/883f23ba35a179fe921869e71d924c3e to your computer and use it in GitHub Desktop.
Save kovid-r/883f23ba35a179fe921869e71d924c3e to your computer and use it in GitHub Desktop.
Download ZIP
Creating DataFrames using Row construct
Raw
pyspark_cheatsheet_create_df.py
from pyspark.sql import Row
# populate two rows with random values
f1 = Row(original_title='Eroica', budget='13393950', year=1992)
f2 = Row(original_title='Night World', budget='1255930', year=1998)
# store the two rows in an array and pass it to Spark
films = [f1, f2]
df = spark.createDataFrame(films)
df.show()
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment