Skip to content

Instantly share code, notes, and snippets.

@koushikmln

koushikmln/SparkExample.py

Created July 6, 2018 12:48
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save koushikmln/6d183f87ed3613336510b6e48aed712d to your computer and use it in GitHub Desktop.
Save koushikmln/6d183f87ed3613336510b6e48aed712d to your computer and use it in GitHub Desktop.
Download ZIP
Spark Example to Give Count of Orders By Status
Raw
SparkExample.py
rdd = sc.textFile("/public/retail_db/orders/part-00000")
status_count = rdd.map(lambda x: (x.split(",")[3], 1))
.reduceByKey(lambda a,b: a + b).collect()
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment