Skip to content

Instantly share code, notes, and snippets.

@hadoopsters
Created September 1, 2017 19:28
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save hadoopsters/ca66048038a0128a395b09c7070bbabb to your computer and use it in GitHub Desktop.
Save hadoopsters/ca66048038a0128a395b09c7070bbabb to your computer and use it in GitHub Desktop.
val mydataframe = ... //put some data in your dataframe, friend
mydataframe
.write
.partitionBy("year", "month", "day", "hour")
.option("orc.compress", "snappy")
.mode(SaveMode.Append)
.orc("/this/is/another/hdfs/directory")
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment