Skip to content

Instantly share code, notes, and snippets.

@eaorak
Last active February 26, 2018 20:42
Show Gist options
  • Star 1 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save eaorak/6c323df3ebe8304ab78a04548ac5ea46 to your computer and use it in GitHub Desktop.
Save eaorak/6c323df3ebe8304ab78a04548ac5ea46 to your computer and use it in GitHub Desktop.
Simple Scala method to print RDD content in Spark
// To be able to easily print RDD content, you can either create a function inside the shell or an implicit class
// Ref to my answer: http://stackoverflow.com/a/41317574/1095213
def p(rdd: org.apache.spark.rdd.RDD[_]) = rdd.foreach(println) // Option 1
implicit class Printer(rdd: org.apache.spark.rdd.RDD[_]) { // Option 2
def print = rdd.foreach(println)
}
// Example
val rdd = sc.parallelize(List(1,2,3,4)).map(_*2)
p(rdd) // 1
rdd.print // 2
// Output
// 2
// 4
// 6
// 8
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment