Skip to content

Instantly share code, notes, and snippets.

@Arnonrgo

Arnonrgo/gist:ea39f45ee35176e7656f

Created September 14, 2014 09:47
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save Arnonrgo/ea39f45ee35176e7656f to your computer and use it in GitHub Desktop.
Save Arnonrgo/ea39f45ee35176e7656f to your computer and use it in GitHub Desktop.
Download ZIP
Raw
gistfile1.scala
val data = sc.textFile("src/test/resources/*.csv")
val calls = data.map (Call(_)).cache()
val hourlyPairs = calls.map(c => (c.getHourly,c))
val weeklyPairs = calls.map(c => (c.getWeekly,c))
val groupedHourly = hourlyPairs.groupByKey()
val groupedWeekly = weeklyPairs.groupByKey()
val hourlyAggregates = groupedHourly.values.map(g => (null,CalcAggregations(g)))
val weeklyAggregates = groupedWeekly.values.map(g => (null,CalcAggregations(g)))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment