Skip to content

Instantly share code, notes, and snippets.

@robenalt

robenalt/read_avro_spark_1.3.0.py

Last active June 10, 2016 19:08
Show Gist options
  • Save robenalt/4c28ef9bc7182825ac698c18af405066 to your computer and use it in GitHub Desktop.
Save robenalt/4c28ef9bc7182825ac698c18af405066 to your computer and use it in GitHub Desktop.
Download ZIP
1.3.0 pyspark read avro
Raw
read_avro_spark_1.3.0.py
# pyspark --packages com.databricks:spark-avro_2.10:1.0.0
# read avro files from 1.3.0 spark
df = sqlCtx.load("/path/to/my_avro", "com.databricks.spark.avro")
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment