Skip to content

Instantly share code, notes, and snippets.

@milindjagre

milindjagre/Spark WordCount

Created October 13, 2016 00:11
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save milindjagre/037e5317f3f261729ae9704f8eb7cfe7 to your computer and use it in GitHub Desktop.
Save milindjagre/037e5317f3f261729ae9704f8eb7cfe7 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
Spark WordCount
val input_file=sc.textFile("input.txt")
val step1=input_file.flatMap(line => line.split(" "))
val step2=step1.map(word => (word,1))
val step3=step2.reduceByKey(_+_)
step3.saveAsTextFile("wordcount")
@milindjagre
Copy link
Author

This is the wordcount program which we run in spark shell

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment