SparkSQL with json file example

SparkSQL with json example

// download sample json
http://bit.ly/2gY39Ay

// start spark-shell in same directory as where customers.json
// is downloaded
val customers = sqlContext.jsonFile("customers.json")

// register a temp table
customers.registerTempTable("customers")

// create dataframe with sql
val firstCityState = sqlContext.sql("SELECT first_name, address.city, address.state FROM customers")

firstCityState.show
firstCityState.collect.foreach(println)

// etc