Skip to content

Instantly share code, notes, and snippets.

@siakon89
Created May 15, 2020 18:31
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save siakon89/ff0f3c79f000cdb304e2bdf185b977fb to your computer and use it in GitHub Desktop.
Save siakon89/ff0f3c79f000cdb304e2bdf185b977fb to your computer and use it in GitHub Desktop.
# Reading a CSV
df = spark.read.csv("filename.csv")
# Reading a CSV with header
df = spark.read.csv("filename.csv", header=True)
# Reading a CSV using the load method
df = spark.read.format("csv").load("filename.csv")
# Reading a CSV using the load method with header
df = spark.read.format("csv").\
option("header", "true").\
load("filename.csv")
# The same goes for different formats
df = spark.read.format("<file format>"). \
load("filename.<format>")
# Or using the given method
df = sparl.read.<format method>.("filename.<format>)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment