Skip to content

Instantly share code, notes, and snippets.

@dongjinahn
Last active February 14, 2018 08:21
Show Gist options
  • Save dongjinahn/d4030a57924c984eff09ce5e9f32a31b to your computer and use it in GitHub Desktop.
Save dongjinahn/d4030a57924c984eff09ce5e9f32a31b to your computer and use it in GitHub Desktop.
df to map spark scala
ageGroup.collect.map(_.toSeq.toList)
res55: Array[List[Any]] = Array(List(31, 4), List(65, 10), List(53, 7), List(34, 6), List(28, 9), List(27, 7), List(26, 2), List(44, 16), List(47, 5), List(52, 3), List(40, 9), List(57, 11), List(54, 14), List(48, 13), List(64, 8), List(41, 15), List(43, 13), List(37, 13), List(61, 6), List(35, 5), List(55, 16), List(59, 9), List(39, 11), List(23, 1), List(49, 14), List(51, 8), List(63, 10), List(50, 4), List(45, 12), List(38, 7), List(25, 10), List(24, 5), List(62, 6), List(29, 11), List(32, 10), List(60, 11), List(56, 13), List(58, 5), List(33, 6), List(68, 5), List(42, 19), List(30, 4), List(66, 6), List(67, 8), List(46, 14), List(36, 14))
ageGroup.collect.map(_.toSeq.toList)
.toMap
:77: error: Cannot prove that List[Any] <:< (T, U).
.toMap
^
ageGroup.collect.map(_.toSeq.toList)
.map(x => (x(0).asInstanceOf[Int], x(1).asInstanceOf[Long]))
res59: Array[(Int, Long)] = Array((31,4), (65,10), (53,7), (34,6), (28,9), (27,7), (26,2), (44,16), (47,5), (52,3), (40,9), (57,11), (54,14), (48,13), (64,8), (41,15), (43,13), (37,13), (61,6), (35,5), (55,16), (59,9), (39,11), (23,1), (49,14), (51,8), (63,10), (50,4), (45,12), (38,7), (25,10), (24,5), (62,6), (29,11), (32,10), (60,11), (56,13), (58,5), (33,6), (68,5), (42,19), (30,4), (66,6), (67,8), (46,14), (36,14))
ageGroup.collect.map(_.toSeq.toList)
.map(x => (x(0).asInstanceOf[Int], x(1).asInstanceOf[Long]))
.toMap
res60: scala.collection.immutable.Map[Int,Long] = Map(56 -> 13, 42 -> 19, 24 -> 5, 37 -> 13, 25 -> 10, 52 -> 3, 46 -> 14, 57 -> 11, 29 -> 11, 61 -> 6, 60 -> 11, 28 -> 9, 38 -> 7, 33 -> 6, 65 -> 10, 53 -> 7, 41 -> 15, 32 -> 10, 34 -> 6, 45 -> 12, 64 -> 8, 44 -> 16, 59 -> 9, 27 -> 7, 54 -> 14, 49 -> 14, 39 -> 11, 66 -> 6, 35 -> 5, 48 -> 13, 63 -> 10, 50 -> 4, 67 -> 8, 31 -> 4, 43 -> 13, 40 -> 9, 26 -> 2, 55 -> 16, 23 -> 1, 58 -> 5, 36 -> 14, 30 -> 4, 51 -> 8, 47 -> 5, 68 -> 5, 62 -> 6)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment