Skip to content

Instantly share code, notes, and snippets.

@Arnold1
Created February 19, 2018 04:42
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save Arnold1/769555c0c712a63ddb7bc2adfe307faa to your computer and use it in GitHub Desktop.
Save Arnold1/769555c0c712a63ddb7bc2adfe307faa to your computer and use it in GitHub Desktop.
spark infos
https://stackoverflow.com/questions/33878433/spark-write-avro-file
http://www.bigdatatidbits.cc/2015/01/how-to-load-some-avro-data-into-spark.html
https://stackoverflow.com/questions/33899417/avro-schema-to-spark-structtype/
https://stackoverflow.com/questions/36078420/spark-avro-to-parquet
https://github.com/tomwhite/hadoop-book/blob/master/ch19-spark/src/test/scala/RDDCreationTest.scala
https://gist.github.com/MLnick/5864741781b9340cb211
http://alvincjin.blogspot.com/2015/11/append-spark-dataframe-with-new-column.html
https://stackoverflow.com/questions/27033823/how-to-overwrite-the-output-directory-in-spark
https://gist.github.com/yzhong52/f81e929e5810271292bd08856e2f4512
https://stackoverflow.com/questions/41567859/extract-a-column-value-from-a-spark-dataframe-and-add-it-to-another-dataframe
https://github.com/databricks/spark-avro/issues/243
https://mapr.com/blog/using-apache-spark-dataframes-processing-tabular-data/
https://medium.com/@thejasbabu/spark-dataframes-10c349de04c
http://kvque.com/spark/value-todf-is-not-a-member-of-org-apache-spark-rdd-rdd/
https://stackoverflow.com/questions/36317002/spark-programatically-creating-dataframe-schema-in-scala
https://stackoverflow.com/questions/37011267/how-to-convert-an-rddrow-back-to-dataframe
https://stackoverflow.com/questions/36795680/copy-schema-from-one-dataframe-to-another-dataframe?rq=1
https://stackoverflow.com/questions/31477598/how-to-create-an-empty-dataframe-with-a-specified-schema
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment