Skip to content

Instantly share code, notes, and snippets.

@krishnabhargav

krishnabhargav/spark-sql-people.scala

Created November 22, 2014 02:25
Show Gist options
  • Save krishnabhargav/17fb00e289741cbc9c7b to your computer and use it in GitHub Desktop.
Save krishnabhargav/17fb00e289741cbc9c7b to your computer and use it in GitHub Desktop.
Download ZIP
Spark SQL example ... beautiful...
Raw
spark-sql-people.scala
val sqlContext = new org.apache.spark.sql.SQLContext(sc); //sc is SparkContext
import sqlContext._
case class Person(name: String, age: Int)
val randomPeople = sc.parallelize(1 to 1000).map(i => Person("Yu_"+i,i))
randomPeople.registerAsTable("people")
val aged30 = sql("select * from people where age = 30")
aged30.foreach(println(_))
@krishnabhargav
Copy link
Author

IPYTHON_OPTS="notebook --pylab inline" ./pyspark to launch spark supported python notebook

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment