Skip to content

Instantly share code, notes, and snippets.

Show Gist options
  • Save dgadiraju/4f9c1c6860e9a123b2b1b24d4313c415 to your computer and use it in GitHub Desktop.
Save dgadiraju/4f9c1c6860e9a123b2b1b24d4313c415 to your computer and use it in GitHub Desktop.
val path = "/public/retail_db"
val products = sc.textFile(path + "/products")
val minPricedProductsByCategory = products.
filter(product => product.split(",")(4) != "").
map(product => {
val p = product.split(",")
(p(1).toInt, product)
}).
reduceByKey((agg, product) => {
if(agg.split(",")(4).toFloat < product.split(",")(4).toFloat)
agg
else
product
}).
map(rec => rec._2)
minPricedProductsByCategory.collect.foreach(println)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment