Skip to content

Instantly share code, notes, and snippets.

@jbn

jbn/spark_json_reader.scala

Created December 10, 2015 18:19
Show Gist options
  • Star 1 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save jbn/b88004bbea6efde2050d to your computer and use it in GitHub Desktop.
Save jbn/b88004bbea6efde2050d to your computer and use it in GitHub Desktop.
Download ZIP
JSON + bz2 + Spark = WINNING
Raw
spark_json_reader.scala
// Load a DataFrame of users. Each line in the file is a JSON
// document, representing one row.
val sqlContext = new org.apache.spark.sql.SQLContext(sc)
val people = sqlContext.read.json("users.json.bz2")
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment