Skip to content

Instantly share code, notes, and snippets.

View kholodilov's full-sized avatar

Dmitry Kholodilov kholodilov

View GitHub Profile
val pairs = sc.parallelize(List(("aa", 1), ("bb", 2),
("aa", 10), ("bb", 20),
("aa", 100), ("bb", 200)))
/* aggregateByKey takes an initial accumulator (here an empty list),
a first lambda function to merge a value to an accumulator, and a
second lambda function to merge two accumulators */
pairs.aggregateByKey(List[Any]())(
(aggr, value) => aggr ::: (value :: Nil),
(aggr1, aggr2) => aggr1 ::: aggr2