Skip to content

Instantly share code, notes, and snippets.

@dalazx
dalazx / csv_to_parquet.scala
Last active February 19, 2018 19:15
CSV to Parquet using Spark
// sbt console
import org.apache.spark.SparkConf
import org.apache.spark.SparkContext
import org.apache.spark.sql.SQLContext
import org.apache.spark.sql.SparkSession
val sparkConf = new SparkConf().setAppName("test").setMaster("local[*]")
val sparkContext = new SparkContext(sparkConf)