AKB428/PageRanking_GraphX

## PageRanking_GraphX
Multiple main classes detected, select one to run:

 [1] Sample1
 [2] Sample2
 [3] Sample3
 [4] TwitterFollowerScoreRanking

Enter number: 4

[info] Running TwitterFollowerScoreRanking
Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
15/06/03 18:34:21 INFO SparkContext: Running Spark version 1.3.1
15/06/03 18:34:21 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
15/06/03 18:34:21 INFO SecurityManager: Changing view acls to: usr0101839
15/06/03 18:34:21 INFO SecurityManager: Changing modify acls to: usr0101839
15/06/03 18:34:21 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(usr0101839); users with modify permissions: Set(usr0101839)
15/06/03 18:34:21 INFO Slf4jLogger: Slf4jLogger started
15/06/03 18:34:22 INFO Remoting: Starting remoting
15/06/03 18:34:22 INFO Remoting: Remoting started; listening on addresses :[akka.tcp://sparkDriver@192.168.107.80:60834]
15/06/03 18:34:22 INFO Utils: Successfully started service 'sparkDriver' on port 60834.
15/06/03 18:34:22 INFO SparkEnv: Registering MapOutputTracker
15/06/03 18:34:22 INFO SparkEnv: Registering BlockManagerMaster
15/06/03 18:34:22 INFO DiskBlockManager: Created local directory at /var/folders/xq/cjwpdzb90_zdsrhcfd4ckgyc0000gq/T/spark-b92cc7eb-d427-4ae4-a2c1-3d5aac42be61/blockmgr-6edd9203-4cc1-4d8d-977c-c4ecb8df0af5
15/06/03 18:34:22 INFO MemoryStore: MemoryStore started with capacity 533.3 MB
15/06/03 18:34:22 INFO HttpFileServer: HTTP File server directory is /var/folders/xq/cjwpdzb90_zdsrhcfd4ckgyc0000gq/T/spark-85002751-3e77-453e-8a9f-2ee1086f8327/httpd-0b05aabe-1e1a-466c-84a9-a431c913ce7e
15/06/03 18:34:22 INFO HttpServer: Starting HTTP Server
15/06/03 18:34:22 INFO Server: jetty-8.y.z-SNAPSHOT
15/06/03 18:34:22 INFO AbstractConnector: Started SocketConnector@0.0.0.0:60835
15/06/03 18:34:22 INFO Utils: Successfully started service 'HTTP file server' on port 60835.
15/06/03 18:34:22 INFO SparkEnv: Registering OutputCommitCoordinator
15/06/03 18:34:22 INFO Server: jetty-8.y.z-SNAPSHOT
15/06/03 18:34:22 INFO AbstractConnector: Started SelectChannelConnector@0.0.0.0:4040
15/06/03 18:34:22 INFO Utils: Successfully started service 'SparkUI' on port 4040.
15/06/03 18:34:22 INFO SparkUI: Started SparkUI at http://192.168.107.80:4040
15/06/03 18:34:22 INFO Executor: Starting executor ID <driver> on host localhost
15/06/03 18:34:22 INFO AkkaUtils: Connecting to HeartbeatReceiver: akka.tcp://sparkDriver@192.168.107.80:60834/user/HeartbeatReceiver
15/06/03 18:34:22 INFO NettyBlockTransferService: Server created on 60836
15/06/03 18:34:22 INFO BlockManagerMaster: Trying to register BlockManager
15/06/03 18:34:22 INFO BlockManagerMasterActor: Registering block manager localhost:60836 with 533.3 MB RAM, BlockManagerId(<driver>, localhost, 60836)
15/06/03 18:34:22 INFO BlockManagerMaster: Registered BlockManager
15/06/03 18:34:23 INFO MemoryStore: ensureFreeSpace(138675) called with curMem=0, maxMem=559153152
15/06/03 18:34:23 INFO MemoryStore: Block broadcast_0 stored as values in memory (estimated size 135.4 KB, free 533.1 MB)
15/06/03 18:34:23 INFO MemoryStore: ensureFreeSpace(18512) called with curMem=138675, maxMem=559153152
15/06/03 18:34:23 INFO MemoryStore: Block broadcast_0_piece0 stored as bytes in memory (estimated size 18.1 KB, free 533.1 MB)
15/06/03 18:34:23 INFO BlockManagerInfo: Added broadcast_0_piece0 in memory on localhost:60836 (size: 18.1 KB, free: 533.2 MB)
15/06/03 18:34:23 INFO BlockManagerMaster: Updated info of block broadcast_0_piece0
15/06/03 18:34:23 INFO SparkContext: Created broadcast 0 from textFile at GraphLoader.scala:72
15/06/03 18:34:23 INFO FileInputFormat: Total input paths to process : 1
15/06/03 18:34:23 INFO SparkContext: Starting job: count at GraphLoader.scala:93
15/06/03 18:34:23 INFO DAGScheduler: Got job 0 (count at GraphLoader.scala:93) with 1 output partitions (allowLocal=false)
15/06/03 18:34:23 INFO DAGScheduler: Final stage: Stage 0(count at GraphLoader.scala:93)
15/06/03 18:34:23 INFO DAGScheduler: Parents of final stage: List()
15/06/03 18:34:23 INFO DAGScheduler: Missing parents: List()
15/06/03 18:34:23 INFO DAGScheduler: Submitting Stage 0 (GraphLoader.edgeListFile - edges (./data/pageRank/followers.txt) MapPartitionsRDD[2] at mapPartitionsWithIndex at GraphLoader.scala:74), which has no missing parents
15/06/03 18:34:23 INFO MemoryStore: ensureFreeSpace(2880) called with curMem=157187, maxMem=559153152
15/06/03 18:34:23 INFO MemoryStore: Block broadcast_1 stored as values in memory (estimated size 2.8 KB, free 533.1 MB)
15/06/03 18:34:23 INFO MemoryStore: ensureFreeSpace(2091) called with curMem=160067, maxMem=559153152
15/06/03 18:34:23 INFO MemoryStore: Block broadcast_1_piece0 stored as bytes in memory (estimated size 2.0 KB, free 533.1 MB)
15/06/03 18:34:23 INFO BlockManagerInfo: Added broadcast_1_piece0 in memory on localhost:60836 (size: 2.0 KB, free: 533.2 MB)
15/06/03 18:34:23 INFO BlockManagerMaster: Updated info of block broadcast_1_piece0
15/06/03 18:34:23 INFO SparkContext: Created broadcast 1 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:23 INFO DAGScheduler: Submitting 1 missing tasks from Stage 0 (GraphLoader.edgeListFile - edges (./data/pageRank/followers.txt) MapPartitionsRDD[2] at mapPartitionsWithIndex at GraphLoader.scala:74)
15/06/03 18:34:23 INFO TaskSchedulerImpl: Adding task set 0.0 with 1 tasks
15/06/03 18:34:23 INFO TaskSetManager: Starting task 0.0 in stage 0.0 (TID 0, localhost, PROCESS_LOCAL, 1340 bytes)
15/06/03 18:34:23 INFO Executor: Running task 0.0 in stage 0.0 (TID 0)
15/06/03 18:34:23 INFO CacheManager: Partition rdd_2_0 not found, computing it
15/06/03 18:34:23 INFO HadoopRDD: Input split: file:/Users/usr0101839/myCode/SparkGraphX_study1/data/pageRank/followers.txt:0+66
15/06/03 18:34:23 INFO deprecation: mapred.tip.id is deprecated. Instead, use mapreduce.task.id
15/06/03 18:34:23 INFO deprecation: mapred.task.id is deprecated. Instead, use mapreduce.task.attempt.id
15/06/03 18:34:23 INFO deprecation: mapred.task.is.map is deprecated. Instead, use mapreduce.task.ismap
15/06/03 18:34:23 INFO deprecation: mapred.task.partition is deprecated. Instead, use mapreduce.task.partition
15/06/03 18:34:23 INFO deprecation: mapred.job.id is deprecated. Instead, use mapreduce.job.id
15/06/03 18:34:23 INFO MemoryStore: ensureFreeSpace(2688) called with curMem=162158, maxMem=559153152
15/06/03 18:34:23 INFO MemoryStore: Block rdd_2_0 stored as values in memory (estimated size 2.6 KB, free 533.1 MB)
15/06/03 18:34:23 INFO BlockManagerInfo: Added rdd_2_0 in memory on localhost:60836 (size: 2.6 KB, free: 533.2 MB)
15/06/03 18:34:23 INFO BlockManagerMaster: Updated info of block rdd_2_0
15/06/03 18:34:23 INFO Executor: Finished task 0.0 in stage 0.0 (TID 0). 2399 bytes result sent to driver
15/06/03 18:34:23 INFO TaskSetManager: Finished task 0.0 in stage 0.0 (TID 0) in 100 ms on localhost (1/1)
15/06/03 18:34:23 INFO TaskSchedulerImpl: Removed TaskSet 0.0, whose tasks have all completed, from pool
15/06/03 18:34:23 INFO DAGScheduler: Stage 0 (count at GraphLoader.scala:93) finished in 0.115 s
15/06/03 18:34:23 INFO DAGScheduler: Job 0 finished: count at GraphLoader.scala:93, took 0.185292 s
15/06/03 18:34:23 INFO GraphLoader: It took 451 ms to load the edges
15/06/03 18:34:23 INFO SparkContext: Starting job: reduce at VertexRDDImpl.scala:90
15/06/03 18:34:23 INFO DAGScheduler: Registering RDD 11 (mapPartitions at GraphImpl.scala:235)
15/06/03 18:34:23 INFO DAGScheduler: Registering RDD 5 (mapPartitions at VertexRDD.scala:330)
15/06/03 18:34:23 INFO DAGScheduler: Registering RDD 19 (mapPartitions at VertexRDDImpl.scala:218)
15/06/03 18:34:23 INFO DAGScheduler: Registering RDD 33 (mapPartitions at VertexRDDImpl.scala:218)
15/06/03 18:34:23 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 3 is 84 bytes
15/06/03 18:34:23 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 2 is 84 bytes
15/06/03 18:34:23 INFO DAGScheduler: Registering RDD 37 (mapPartitions at VertexRDDImpl.scala:218)
15/06/03 18:34:23 INFO DAGScheduler: Registering RDD 41 (mapPartitions at GraphImpl.scala:235)
15/06/03 18:34:23 INFO DAGScheduler: Got job 1 (reduce at VertexRDDImpl.scala:90) with 1 output partitions (allowLocal=false)
15/06/03 18:34:23 INFO DAGScheduler: Final stage: Stage 9(reduce at VertexRDDImpl.scala:90)
15/06/03 18:34:23 INFO DAGScheduler: Parents of final stage: List(Stage 5, Stage 6, Stage 8)
15/06/03 18:34:23 INFO DAGScheduler: Missing parents: List(Stage 5, Stage 6, Stage 8)
15/06/03 18:34:23 INFO DAGScheduler: Submitting Stage 5 (VertexRDD.createRoutingTables - vid2pid (aggregation) MapPartitionsRDD[5] at mapPartitions at VertexRDD.scala:330), which has no missing parents
15/06/03 18:34:23 INFO MemoryStore: ensureFreeSpace(3504) called with curMem=164846, maxMem=559153152
15/06/03 18:34:23 INFO MemoryStore: Block broadcast_2 stored as values in memory (estimated size 3.4 KB, free 533.1 MB)
15/06/03 18:34:23 INFO MemoryStore: ensureFreeSpace(2501) called with curMem=168350, maxMem=559153152
15/06/03 18:34:23 INFO MemoryStore: Block broadcast_2_piece0 stored as bytes in memory (estimated size 2.4 KB, free 533.1 MB)
15/06/03 18:34:23 INFO BlockManagerInfo: Added broadcast_2_piece0 in memory on localhost:60836 (size: 2.4 KB, free: 533.2 MB)
15/06/03 18:34:23 INFO BlockManagerMaster: Updated info of block broadcast_2_piece0
15/06/03 18:34:23 INFO SparkContext: Created broadcast 2 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:23 INFO DAGScheduler: Submitting 1 missing tasks from Stage 5 (VertexRDD.createRoutingTables - vid2pid (aggregation) MapPartitionsRDD[5] at mapPartitions at VertexRDD.scala:330)
15/06/03 18:34:23 INFO TaskSchedulerImpl: Adding task set 5.0 with 1 tasks
15/06/03 18:34:23 INFO DAGScheduler: Submitting Stage 6 (GraphImpl.aggregateMessages - preAgg MapPartitionsRDD[11] at mapPartitions at GraphImpl.scala:235), which has no missing parents
15/06/03 18:34:23 INFO MemoryStore: ensureFreeSpace(4024) called with curMem=170851, maxMem=559153152
15/06/03 18:34:23 INFO MemoryStore: Block broadcast_3 stored as values in memory (estimated size 3.9 KB, free 533.1 MB)
15/06/03 18:34:23 INFO MemoryStore: ensureFreeSpace(2849) called with curMem=174875, maxMem=559153152
15/06/03 18:34:23 INFO MemoryStore: Block broadcast_3_piece0 stored as bytes in memory (estimated size 2.8 KB, free 533.1 MB)
15/06/03 18:34:23 INFO TaskSetManager: Starting task 0.0 in stage 5.0 (TID 1, localhost, PROCESS_LOCAL, 1329 bytes)
15/06/03 18:34:23 INFO Executor: Running task 0.0 in stage 5.0 (TID 1)
15/06/03 18:34:23 INFO BlockManagerInfo: Added broadcast_3_piece0 in memory on localhost:60836 (size: 2.8 KB, free: 533.2 MB)
15/06/03 18:34:23 INFO BlockManagerMaster: Updated info of block broadcast_3_piece0
15/06/03 18:34:23 INFO SparkContext: Created broadcast 3 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:23 INFO DAGScheduler: Submitting 1 missing tasks from Stage 6 (GraphImpl.aggregateMessages - preAgg MapPartitionsRDD[11] at mapPartitions at GraphImpl.scala:235)
15/06/03 18:34:23 INFO TaskSchedulerImpl: Adding task set 6.0 with 1 tasks
15/06/03 18:34:23 INFO BlockManager: Found block rdd_2_0 locally
15/06/03 18:34:23 INFO Executor: Finished task 0.0 in stage 5.0 (TID 1). 2003 bytes result sent to driver
15/06/03 18:34:23 INFO TaskSetManager: Starting task 0.0 in stage 6.0 (TID 2, localhost, PROCESS_LOCAL, 1329 bytes)
15/06/03 18:34:23 INFO Executor: Running task 0.0 in stage 6.0 (TID 2)
15/06/03 18:34:23 INFO TaskSetManager: Finished task 0.0 in stage 5.0 (TID 1) in 67 ms on localhost (1/1)
15/06/03 18:34:23 INFO TaskSchedulerImpl: Removed TaskSet 5.0, whose tasks have all completed, from pool
15/06/03 18:34:23 INFO DAGScheduler: Stage 5 (mapPartitions at VertexRDD.scala:330) finished in 0.069 s
15/06/03 18:34:23 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:23 INFO DAGScheduler: running: Set(Stage 6)
15/06/03 18:34:23 INFO DAGScheduler: waiting: Set(Stage 9, Stage 3, Stage 7, Stage 4, Stage 8)
15/06/03 18:34:23 INFO DAGScheduler: failed: Set()
15/06/03 18:34:23 INFO BlockManager: Found block rdd_2_0 locally
15/06/03 18:34:23 INFO DAGScheduler: Missing parents for Stage 9: List(Stage 6, Stage 8)
15/06/03 18:34:23 INFO DAGScheduler: Missing parents for Stage 3: List(Stage 6)
15/06/03 18:34:23 INFO DAGScheduler: Missing parents for Stage 7: List(Stage 6)
15/06/03 18:34:23 INFO DAGScheduler: Missing parents for Stage 4: List(Stage 6)
15/06/03 18:34:23 INFO DAGScheduler: Missing parents for Stage 8: List(Stage 3, Stage 7, Stage 4)
15/06/03 18:34:23 INFO Executor: Finished task 0.0 in stage 6.0 (TID 2). 2003 bytes result sent to driver
15/06/03 18:34:23 INFO TaskSetManager: Finished task 0.0 in stage 6.0 (TID 2) in 26 ms on localhost (1/1)
15/06/03 18:34:23 INFO TaskSchedulerImpl: Removed TaskSet 6.0, whose tasks have all completed, from pool
15/06/03 18:34:23 INFO DAGScheduler: Stage 6 (mapPartitions at GraphImpl.scala:235) finished in 0.087 s
15/06/03 18:34:23 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:23 INFO DAGScheduler: running: Set()
15/06/03 18:34:23 INFO DAGScheduler: waiting: Set(Stage 9, Stage 3, Stage 7, Stage 4, Stage 8)
15/06/03 18:34:23 INFO DAGScheduler: failed: Set()
15/06/03 18:34:23 INFO DAGScheduler: Missing parents for Stage 9: List(Stage 8)
15/06/03 18:34:23 INFO DAGScheduler: Missing parents for Stage 3: List()
15/06/03 18:34:23 INFO DAGScheduler: Missing parents for Stage 7: List()
15/06/03 18:34:23 INFO DAGScheduler: Missing parents for Stage 4: List()
15/06/03 18:34:23 INFO DAGScheduler: Missing parents for Stage 8: List(Stage 3, Stage 7, Stage 4)
15/06/03 18:34:23 INFO DAGScheduler: Submitting Stage 3 (ReplicatedVertexView.upgrade(true, true) - shippedVerts true true (broadcast) MapPartitionsRDD[19] at mapPartitions at VertexRDDImpl.scala:218), which is now runnable
15/06/03 18:34:23 INFO MemoryStore: ensureFreeSpace(4464) called with curMem=177724, maxMem=559153152
15/06/03 18:34:23 INFO MemoryStore: Block broadcast_4 stored as values in memory (estimated size 4.4 KB, free 533.1 MB)
15/06/03 18:34:23 INFO MemoryStore: ensureFreeSpace(2997) called with curMem=182188, maxMem=559153152
15/06/03 18:34:23 INFO MemoryStore: Block broadcast_4_piece0 stored as bytes in memory (estimated size 2.9 KB, free 533.1 MB)
15/06/03 18:34:23 INFO BlockManagerInfo: Added broadcast_4_piece0 in memory on localhost:60836 (size: 2.9 KB, free: 533.2 MB)
15/06/03 18:34:23 INFO BlockManagerMaster: Updated info of block broadcast_4_piece0
15/06/03 18:34:23 INFO SparkContext: Created broadcast 4 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:23 INFO DAGScheduler: Submitting 1 missing tasks from Stage 3 (ReplicatedVertexView.upgrade(true, true) - shippedVerts true true (broadcast) MapPartitionsRDD[19] at mapPartitions at VertexRDDImpl.scala:218)
15/06/03 18:34:23 INFO TaskSchedulerImpl: Adding task set 3.0 with 1 tasks
15/06/03 18:34:23 INFO DAGScheduler: Submitting Stage 4 (ReplicatedVertexView.updateVertices - shippedVerts false false (broadcast) MapPartitionsRDD[33] at mapPartitions at VertexRDDImpl.scala:218), which is now runnable
15/06/03 18:34:23 INFO MemoryStore: ensureFreeSpace(5968) called with curMem=185185, maxMem=559153152
15/06/03 18:34:23 INFO MemoryStore: Block broadcast_5 stored as values in memory (estimated size 5.8 KB, free 533.1 MB)
15/06/03 18:34:23 INFO MemoryStore: ensureFreeSpace(3803) called with curMem=191153, maxMem=559153152
15/06/03 18:34:23 INFO MemoryStore: Block broadcast_5_piece0 stored as bytes in memory (estimated size 3.7 KB, free 533.1 MB)
15/06/03 18:34:23 INFO BlockManagerInfo: Added broadcast_5_piece0 in memory on localhost:60836 (size: 3.7 KB, free: 533.2 MB)
15/06/03 18:34:23 INFO BlockManagerMaster: Updated info of block broadcast_5_piece0
15/06/03 18:34:23 INFO SparkContext: Created broadcast 5 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:23 INFO DAGScheduler: Submitting 1 missing tasks from Stage 4 (ReplicatedVertexView.updateVertices - shippedVerts false false (broadcast) MapPartitionsRDD[33] at mapPartitions at VertexRDDImpl.scala:218)
15/06/03 18:34:23 INFO TaskSchedulerImpl: Adding task set 4.0 with 1 tasks
15/06/03 18:34:23 INFO TaskSetManager: Starting task 0.0 in stage 3.0 (TID 3, localhost, PROCESS_LOCAL, 1494 bytes)
15/06/03 18:34:23 INFO Executor: Running task 0.0 in stage 3.0 (TID 3)
15/06/03 18:34:23 INFO DAGScheduler: Submitting Stage 7 (ReplicatedVertexView.upgrade(true, false) - shippedVerts true false (broadcast) MapPartitionsRDD[37] at mapPartitions at VertexRDDImpl.scala:218), which is now runnable
15/06/03 18:34:23 INFO MemoryStore: ensureFreeSpace(5744) called with curMem=194956, maxMem=559153152
15/06/03 18:34:23 INFO MemoryStore: Block broadcast_6 stored as values in memory (estimated size 5.6 KB, free 533.1 MB)
15/06/03 18:34:23 INFO MemoryStore: ensureFreeSpace(3717) called with curMem=200700, maxMem=559153152
15/06/03 18:34:23 INFO MemoryStore: Block broadcast_6_piece0 stored as bytes in memory (estimated size 3.6 KB, free 533.1 MB)
15/06/03 18:34:23 INFO BlockManagerInfo: Added broadcast_6_piece0 in memory on localhost:60836 (size: 3.6 KB, free: 533.2 MB)
15/06/03 18:34:23 INFO BlockManagerMaster: Updated info of block broadcast_6_piece0
15/06/03 18:34:23 INFO SparkContext: Created broadcast 6 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:23 INFO DAGScheduler: Submitting 1 missing tasks from Stage 7 (ReplicatedVertexView.upgrade(true, false) - shippedVerts true false (broadcast) MapPartitionsRDD[37] at mapPartitions at VertexRDDImpl.scala:218)
15/06/03 18:34:23 INFO TaskSchedulerImpl: Adding task set 7.0 with 1 tasks
15/06/03 18:34:23 INFO CacheManager: Partition rdd_15_0 not found, computing it
15/06/03 18:34:23 INFO CacheManager: Partition rdd_8_0 not found, computing it
15/06/03 18:34:23 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:23 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 5 ms
15/06/03 18:34:23 INFO MemoryStore: ensureFreeSpace(1352) called with curMem=204417, maxMem=559153152
15/06/03 18:34:23 INFO MemoryStore: Block rdd_8_0 stored as values in memory (estimated size 1352.0 B, free 533.1 MB)
15/06/03 18:34:23 INFO BlockManagerInfo: Added rdd_8_0 in memory on localhost:60836 (size: 1352.0 B, free: 533.2 MB)
15/06/03 18:34:23 INFO BlockManagerMaster: Updated info of block rdd_8_0
15/06/03 18:34:23 INFO BlockManager: Found block rdd_8_0 locally
15/06/03 18:34:23 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:23 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:23 INFO MemoryStore: ensureFreeSpace(1352) called with curMem=205769, maxMem=559153152
15/06/03 18:34:23 INFO MemoryStore: Block rdd_15_0 stored as values in memory (estimated size 1352.0 B, free 533.1 MB)
15/06/03 18:34:23 INFO BlockManagerInfo: Added rdd_15_0 in memory on localhost:60836 (size: 1352.0 B, free: 533.2 MB)
15/06/03 18:34:23 INFO BlockManagerMaster: Updated info of block rdd_15_0
15/06/03 18:34:23 INFO Executor: Finished task 0.0 in stage 3.0 (TID 3). 2843 bytes result sent to driver
15/06/03 18:34:23 INFO TaskSetManager: Starting task 0.0 in stage 4.0 (TID 4, localhost, PROCESS_LOCAL, 1526 bytes)
15/06/03 18:34:23 INFO Executor: Running task 0.0 in stage 4.0 (TID 4)
15/06/03 18:34:23 INFO TaskSetManager: Finished task 0.0 in stage 3.0 (TID 3) in 78 ms on localhost (1/1)
15/06/03 18:34:23 INFO TaskSchedulerImpl: Removed TaskSet 3.0, whose tasks have all completed, from pool
15/06/03 18:34:23 INFO DAGScheduler: Stage 3 (mapPartitions at VertexRDDImpl.scala:218) finished in 0.079 s
15/06/03 18:34:23 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:23 INFO DAGScheduler: running: Set(Stage 7, Stage 4)
15/06/03 18:34:23 INFO DAGScheduler: waiting: Set(Stage 9, Stage 8)
15/06/03 18:34:23 INFO DAGScheduler: failed: Set()
15/06/03 18:34:23 INFO DAGScheduler: Missing parents for Stage 9: List(Stage 8)
15/06/03 18:34:23 INFO DAGScheduler: Missing parents for Stage 8: List(Stage 7, Stage 4)
15/06/03 18:34:23 INFO CacheManager: Partition rdd_25_0 not found, computing it
15/06/03 18:34:23 INFO BlockManager: Found block rdd_15_0 locally
15/06/03 18:34:23 INFO MemoryStore: ensureFreeSpace(1496) called with curMem=207121, maxMem=559153152
15/06/03 18:34:23 INFO MemoryStore: Block rdd_25_0 stored as values in memory (estimated size 1496.0 B, free 533.1 MB)
15/06/03 18:34:23 INFO BlockManagerInfo: Added rdd_25_0 in memory on localhost:60836 (size: 1496.0 B, free: 533.2 MB)
15/06/03 18:34:23 INFO BlockManagerMaster: Updated info of block rdd_25_0
15/06/03 18:34:23 INFO CacheManager: Partition rdd_29_0 not found, computing it
15/06/03 18:34:23 INFO BlockManager: Found block rdd_25_0 locally
15/06/03 18:34:23 INFO MemoryStore: ensureFreeSpace(1496) called with curMem=208617, maxMem=559153152
15/06/03 18:34:23 INFO MemoryStore: Block rdd_29_0 stored as values in memory (estimated size 1496.0 B, free 533.0 MB)
15/06/03 18:34:23 INFO BlockManagerInfo: Added rdd_29_0 in memory on localhost:60836 (size: 1496.0 B, free: 533.2 MB)
15/06/03 18:34:23 INFO BlockManagerMaster: Updated info of block rdd_29_0
15/06/03 18:34:23 INFO Executor: Finished task 0.0 in stage 4.0 (TID 4). 2627 bytes result sent to driver
15/06/03 18:34:23 INFO TaskSetManager: Starting task 0.0 in stage 7.0 (TID 5, localhost, PROCESS_LOCAL, 1494 bytes)
15/06/03 18:34:23 INFO TaskSetManager: Finished task 0.0 in stage 4.0 (TID 4) in 16 ms on localhost (1/1)
15/06/03 18:34:23 INFO TaskSchedulerImpl: Removed TaskSet 4.0, whose tasks have all completed, from pool
15/06/03 18:34:23 INFO DAGScheduler: Stage 4 (mapPartitions at VertexRDDImpl.scala:218) finished in 0.086 s
15/06/03 18:34:23 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:23 INFO DAGScheduler: running: Set(Stage 7)
15/06/03 18:34:23 INFO DAGScheduler: waiting: Set(Stage 9, Stage 8)
15/06/03 18:34:23 INFO DAGScheduler: failed: Set()
15/06/03 18:34:23 INFO Executor: Running task 0.0 in stage 7.0 (TID 5)
15/06/03 18:34:23 INFO DAGScheduler: Missing parents for Stage 9: List(Stage 8)
15/06/03 18:34:23 INFO DAGScheduler: Missing parents for Stage 8: List(Stage 7)
15/06/03 18:34:23 INFO BlockManager: Found block rdd_29_0 locally
15/06/03 18:34:23 INFO Executor: Finished task 0.0 in stage 7.0 (TID 5). 2003 bytes result sent to driver
15/06/03 18:34:23 INFO TaskSetManager: Finished task 0.0 in stage 7.0 (TID 5) in 19 ms on localhost (1/1)
15/06/03 18:34:23 INFO TaskSchedulerImpl: Removed TaskSet 7.0, whose tasks have all completed, from pool
15/06/03 18:34:23 INFO DAGScheduler: Stage 7 (mapPartitions at VertexRDDImpl.scala:218) finished in 0.098 s
15/06/03 18:34:23 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:23 INFO DAGScheduler: running: Set()
15/06/03 18:34:23 INFO DAGScheduler: waiting: Set(Stage 9, Stage 8)
15/06/03 18:34:23 INFO DAGScheduler: failed: Set()
15/06/03 18:34:23 INFO DAGScheduler: Missing parents for Stage 9: List(Stage 8)
15/06/03 18:34:23 INFO DAGScheduler: Missing parents for Stage 8: List()
15/06/03 18:34:23 INFO DAGScheduler: Submitting Stage 8 (GraphImpl.aggregateMessages - preAgg MapPartitionsRDD[41] at mapPartitions at GraphImpl.scala:235), which is now runnable
15/06/03 18:34:23 INFO MemoryStore: ensureFreeSpace(6480) called with curMem=210113, maxMem=559153152
15/06/03 18:34:23 INFO MemoryStore: Block broadcast_7 stored as values in memory (estimated size 6.3 KB, free 533.0 MB)
15/06/03 18:34:23 INFO MemoryStore: ensureFreeSpace(4341) called with curMem=216593, maxMem=559153152
15/06/03 18:34:23 INFO MemoryStore: Block broadcast_7_piece0 stored as bytes in memory (estimated size 4.2 KB, free 533.0 MB)
15/06/03 18:34:23 INFO BlockManagerInfo: Added broadcast_7_piece0 in memory on localhost:60836 (size: 4.2 KB, free: 533.2 MB)
15/06/03 18:34:23 INFO BlockManagerMaster: Updated info of block broadcast_7_piece0
15/06/03 18:34:23 INFO SparkContext: Created broadcast 7 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:23 INFO DAGScheduler: Submitting 1 missing tasks from Stage 8 (GraphImpl.aggregateMessages - preAgg MapPartitionsRDD[41] at mapPartitions at GraphImpl.scala:235)
15/06/03 18:34:23 INFO TaskSchedulerImpl: Adding task set 8.0 with 1 tasks
15/06/03 18:34:23 INFO TaskSetManager: Starting task 0.0 in stage 8.0 (TID 6, localhost, PROCESS_LOCAL, 1894 bytes)
15/06/03 18:34:23 INFO Executor: Running task 0.0 in stage 8.0 (TID 6)
15/06/03 18:34:23 INFO CacheManager: Partition rdd_35_0 not found, computing it
15/06/03 18:34:23 INFO CacheManager: Partition rdd_27_0 not found, computing it
15/06/03 18:34:23 INFO BlockManager: Found block rdd_2_0 locally
15/06/03 18:34:23 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:23 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
15/06/03 18:34:23 INFO MemoryStore: ensureFreeSpace(2776) called with curMem=220934, maxMem=559153152
15/06/03 18:34:23 INFO MemoryStore: Block rdd_27_0 stored as values in memory (estimated size 2.7 KB, free 533.0 MB)
15/06/03 18:34:23 INFO BlockManagerInfo: Added rdd_27_0 in memory on localhost:60836 (size: 2.7 KB, free: 533.2 MB)
15/06/03 18:34:23 INFO BlockManagerMaster: Updated info of block rdd_27_0
15/06/03 18:34:23 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:23 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:23 INFO MemoryStore: ensureFreeSpace(2776) called with curMem=223710, maxMem=559153152
15/06/03 18:34:23 INFO MemoryStore: Block rdd_35_0 stored as values in memory (estimated size 2.7 KB, free 533.0 MB)
15/06/03 18:34:23 INFO BlockManagerInfo: Added rdd_35_0 in memory on localhost:60836 (size: 2.7 KB, free: 533.2 MB)
15/06/03 18:34:23 INFO BlockManagerMaster: Updated info of block rdd_35_0
15/06/03 18:34:23 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:23 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:23 INFO Executor: Finished task 0.0 in stage 8.0 (TID 6). 2843 bytes result sent to driver
15/06/03 18:34:23 INFO TaskSetManager: Finished task 0.0 in stage 8.0 (TID 6) in 29 ms on localhost (1/1)
15/06/03 18:34:23 INFO TaskSchedulerImpl: Removed TaskSet 8.0, whose tasks have all completed, from pool
15/06/03 18:34:23 INFO DAGScheduler: Stage 8 (mapPartitions at GraphImpl.scala:235) finished in 0.029 s
15/06/03 18:34:23 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:23 INFO DAGScheduler: running: Set()
15/06/03 18:34:23 INFO DAGScheduler: waiting: Set(Stage 9)
15/06/03 18:34:23 INFO DAGScheduler: failed: Set()
15/06/03 18:34:23 INFO DAGScheduler: Missing parents for Stage 9: List()
15/06/03 18:34:23 INFO DAGScheduler: Submitting Stage 9 (MapPartitionsRDD[45] at map at VertexRDDImpl.scala:90), which is now runnable
15/06/03 18:34:23 INFO MemoryStore: ensureFreeSpace(6272) called with curMem=226486, maxMem=559153152
15/06/03 18:34:23 INFO MemoryStore: Block broadcast_8 stored as values in memory (estimated size 6.1 KB, free 533.0 MB)
15/06/03 18:34:23 INFO MemoryStore: ensureFreeSpace(3985) called with curMem=232758, maxMem=559153152
15/06/03 18:34:23 INFO MemoryStore: Block broadcast_8_piece0 stored as bytes in memory (estimated size 3.9 KB, free 533.0 MB)
15/06/03 18:34:23 INFO BlockManagerInfo: Added broadcast_8_piece0 in memory on localhost:60836 (size: 3.9 KB, free: 533.2 MB)
15/06/03 18:34:23 INFO BlockManagerMaster: Updated info of block broadcast_8_piece0
15/06/03 18:34:23 INFO SparkContext: Created broadcast 8 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:23 INFO DAGScheduler: Submitting 1 missing tasks from Stage 9 (MapPartitionsRDD[45] at map at VertexRDDImpl.scala:90)
15/06/03 18:34:23 INFO TaskSchedulerImpl: Adding task set 9.0 with 1 tasks
15/06/03 18:34:23 INFO TaskSetManager: Starting task 0.0 in stage 9.0 (TID 7, localhost, PROCESS_LOCAL, 1546 bytes)
15/06/03 18:34:23 INFO Executor: Running task 0.0 in stage 9.0 (TID 7)
15/06/03 18:34:23 INFO BlockManager: Found block rdd_29_0 locally
15/06/03 18:34:23 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:23 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:23 INFO Executor: Finished task 0.0 in stage 9.0 (TID 7). 2124 bytes result sent to driver
15/06/03 18:34:23 INFO TaskSetManager: Finished task 0.0 in stage 9.0 (TID 7) in 5 ms on localhost (1/1)
15/06/03 18:34:23 INFO TaskSchedulerImpl: Removed TaskSet 9.0, whose tasks have all completed, from pool
15/06/03 18:34:23 INFO DAGScheduler: Stage 9 (reduce at VertexRDDImpl.scala:90) finished in 0.006 s
15/06/03 18:34:23 INFO DAGScheduler: Job 1 finished: reduce at VertexRDDImpl.scala:90, took 0.303204 s
15/06/03 18:34:23 INFO SparkContext: Starting job: reduce at VertexRDDImpl.scala:90
15/06/03 18:34:23 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 2 is 145 bytes
15/06/03 18:34:23 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 3 is 145 bytes
15/06/03 18:34:23 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 5 is 145 bytes
15/06/03 18:34:23 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 4 is 145 bytes
15/06/03 18:34:23 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 1 is 145 bytes
15/06/03 18:34:23 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 0 is 145 bytes
15/06/03 18:34:23 INFO DAGScheduler: Registering RDD 52 (mapPartitions at VertexRDDImpl.scala:218)
15/06/03 18:34:23 INFO DAGScheduler: Registering RDD 56 (mapPartitions at VertexRDDImpl.scala:222)
15/06/03 18:34:23 INFO DAGScheduler: Registering RDD 60 (mapPartitions at GraphImpl.scala:235)
15/06/03 18:34:23 INFO DAGScheduler: Got job 2 (reduce at VertexRDDImpl.scala:90) with 1 output partitions (allowLocal=false)
15/06/03 18:34:23 INFO DAGScheduler: Final stage: Stage 27(reduce at VertexRDDImpl.scala:90)
15/06/03 18:34:23 INFO DAGScheduler: Parents of final stage: List(Stage 24, Stage 21, Stage 22, Stage 26)
15/06/03 18:34:23 INFO DAGScheduler: Missing parents: List(Stage 26)
15/06/03 18:34:23 INFO DAGScheduler: Submitting Stage 18 (ReplicatedVertexView.updateVertices - shippedVerts true false (broadcast) MapPartitionsRDD[52] at mapPartitions at VertexRDDImpl.scala:218), which has no missing parents
15/06/03 18:34:23 INFO MemoryStore: ensureFreeSpace(6704) called with curMem=236743, maxMem=559153152
15/06/03 18:34:23 INFO MemoryStore: Block broadcast_9 stored as values in memory (estimated size 6.5 KB, free 533.0 MB)
15/06/03 18:34:23 INFO MemoryStore: ensureFreeSpace(4179) called with curMem=243447, maxMem=559153152
15/06/03 18:34:23 INFO MemoryStore: Block broadcast_9_piece0 stored as bytes in memory (estimated size 4.1 KB, free 533.0 MB)
15/06/03 18:34:23 INFO BlockManagerInfo: Added broadcast_9_piece0 in memory on localhost:60836 (size: 4.1 KB, free: 533.2 MB)
15/06/03 18:34:23 INFO BlockManagerMaster: Updated info of block broadcast_9_piece0
15/06/03 18:34:23 INFO SparkContext: Created broadcast 9 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:23 INFO DAGScheduler: Submitting 1 missing tasks from Stage 18 (ReplicatedVertexView.updateVertices - shippedVerts true false (broadcast) MapPartitionsRDD[52] at mapPartitions at VertexRDDImpl.scala:218)
15/06/03 18:34:23 INFO TaskSchedulerImpl: Adding task set 18.0 with 1 tasks
15/06/03 18:34:23 INFO DAGScheduler: Submitting Stage 25 (ReplicatedVertexView.withActiveSet - shippedActives (broadcast) MapPartitionsRDD[56] at mapPartitions at VertexRDDImpl.scala:222), which has no missing parents
15/06/03 18:34:23 INFO TaskSetManager: Starting task 0.0 in stage 18.0 (TID 8, localhost, PROCESS_LOCAL, 1631 bytes)
15/06/03 18:34:23 INFO Executor: Running task 0.0 in stage 18.0 (TID 8)
15/06/03 18:34:23 INFO MemoryStore: ensureFreeSpace(6304) called with curMem=247626, maxMem=559153152
15/06/03 18:34:23 INFO MemoryStore: Block broadcast_10 stored as values in memory (estimated size 6.2 KB, free 533.0 MB)
15/06/03 18:34:23 INFO MemoryStore: ensureFreeSpace(3975) called with curMem=253930, maxMem=559153152
15/06/03 18:34:23 INFO MemoryStore: Block broadcast_10_piece0 stored as bytes in memory (estimated size 3.9 KB, free 533.0 MB)
15/06/03 18:34:23 INFO BlockManager: Found block rdd_29_0 locally
15/06/03 18:34:23 INFO BlockManagerInfo: Added broadcast_10_piece0 in memory on localhost:60836 (size: 3.9 KB, free: 533.2 MB)
15/06/03 18:34:23 INFO BlockManagerMaster: Updated info of block broadcast_10_piece0
15/06/03 18:34:23 INFO CacheManager: Partition rdd_48_0 not found, computing it
15/06/03 18:34:23 INFO SparkContext: Created broadcast 10 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:23 INFO BlockManager: Found block rdd_29_0 locally
15/06/03 18:34:23 INFO DAGScheduler: Submitting 1 missing tasks from Stage 25 (ReplicatedVertexView.withActiveSet - shippedActives (broadcast) MapPartitionsRDD[56] at mapPartitions at VertexRDDImpl.scala:222)
15/06/03 18:34:23 INFO TaskSchedulerImpl: Adding task set 25.0 with 1 tasks
15/06/03 18:34:23 INFO CacheManager: Partition rdd_46_0 not found, computing it
15/06/03 18:34:23 INFO BlockManager: Found block rdd_29_0 locally
15/06/03 18:34:23 INFO BlockManager: Found block rdd_29_0 locally
15/06/03 18:34:23 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:23 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:23 INFO MemoryStore: ensureFreeSpace(1544) called with curMem=257905, maxMem=559153152
15/06/03 18:34:23 INFO MemoryStore: Block rdd_46_0 stored as values in memory (estimated size 1544.0 B, free 533.0 MB)
15/06/03 18:34:23 INFO BlockManagerInfo: Added rdd_46_0 in memory on localhost:60836 (size: 1544.0 B, free: 533.2 MB)
15/06/03 18:34:23 INFO BlockManagerMaster: Updated info of block rdd_46_0
15/06/03 18:34:23 INFO MemoryStore: ensureFreeSpace(1496) called with curMem=259449, maxMem=559153152
15/06/03 18:34:23 INFO MemoryStore: Block rdd_48_0 stored as values in memory (estimated size 1496.0 B, free 533.0 MB)
15/06/03 18:34:23 INFO BlockManagerInfo: Added rdd_48_0 in memory on localhost:60836 (size: 1496.0 B, free: 533.2 MB)
15/06/03 18:34:23 INFO BlockManagerMaster: Updated info of block rdd_48_0
15/06/03 18:34:23 INFO Executor: Finished task 0.0 in stage 18.0 (TID 8). 2843 bytes result sent to driver
15/06/03 18:34:23 INFO TaskSetManager: Starting task 0.0 in stage 25.0 (TID 9, localhost, PROCESS_LOCAL, 1567 bytes)
15/06/03 18:34:23 INFO Executor: Running task 0.0 in stage 25.0 (TID 9)
15/06/03 18:34:23 INFO TaskSetManager: Finished task 0.0 in stage 18.0 (TID 8) in 18 ms on localhost (1/1)
15/06/03 18:34:23 INFO TaskSchedulerImpl: Removed TaskSet 18.0, whose tasks have all completed, from pool
15/06/03 18:34:23 INFO DAGScheduler: Stage 18 (mapPartitions at VertexRDDImpl.scala:218) finished in 0.019 s
15/06/03 18:34:23 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:23 INFO DAGScheduler: running: Set(Stage 25)
15/06/03 18:34:23 INFO DAGScheduler: waiting: Set(Stage 27, Stage 26)
15/06/03 18:34:23 INFO DAGScheduler: failed: Set()
15/06/03 18:34:23 INFO DAGScheduler: Missing parents for Stage 27: List(Stage 26)
15/06/03 18:34:23 INFO BlockManager: Found block rdd_46_0 locally
15/06/03 18:34:23 INFO DAGScheduler: Missing parents for Stage 26: List(Stage 25)
15/06/03 18:34:23 INFO Executor: Finished task 0.0 in stage 25.0 (TID 9). 2003 bytes result sent to driver
15/06/03 18:34:23 INFO TaskSetManager: Finished task 0.0 in stage 25.0 (TID 9) in 19 ms on localhost (1/1)
15/06/03 18:34:23 INFO TaskSchedulerImpl: Removed TaskSet 25.0, whose tasks have all completed, from pool
15/06/03 18:34:23 INFO DAGScheduler: Stage 25 (mapPartitions at VertexRDDImpl.scala:222) finished in 0.033 s
15/06/03 18:34:23 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:23 INFO DAGScheduler: running: Set()
15/06/03 18:34:23 INFO DAGScheduler: waiting: Set(Stage 27, Stage 26)
15/06/03 18:34:23 INFO DAGScheduler: failed: Set()
15/06/03 18:34:23 INFO DAGScheduler: Missing parents for Stage 27: List(Stage 26)
15/06/03 18:34:23 INFO DAGScheduler: Missing parents for Stage 26: List()
15/06/03 18:34:23 INFO DAGScheduler: Submitting Stage 26 (GraphImpl.aggregateMessages - preAgg MapPartitionsRDD[60] at mapPartitions at GraphImpl.scala:235), which is now runnable
15/06/03 18:34:23 INFO MemoryStore: ensureFreeSpace(7080) called with curMem=260945, maxMem=559153152
15/06/03 18:34:23 INFO MemoryStore: Block broadcast_11 stored as values in memory (estimated size 6.9 KB, free 533.0 MB)
15/06/03 18:34:24 INFO MemoryStore: ensureFreeSpace(4505) called with curMem=268025, maxMem=559153152
15/06/03 18:34:24 INFO MemoryStore: Block broadcast_11_piece0 stored as bytes in memory (estimated size 4.4 KB, free 533.0 MB)
15/06/03 18:34:24 INFO BlockManagerInfo: Added broadcast_11_piece0 in memory on localhost:60836 (size: 4.4 KB, free: 533.2 MB)
15/06/03 18:34:24 INFO BlockManagerMaster: Updated info of block broadcast_11_piece0
15/06/03 18:34:24 INFO SparkContext: Created broadcast 11 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:24 INFO DAGScheduler: Submitting 1 missing tasks from Stage 26 (GraphImpl.aggregateMessages - preAgg MapPartitionsRDD[60] at mapPartitions at GraphImpl.scala:235)
15/06/03 18:34:24 INFO TaskSchedulerImpl: Adding task set 26.0 with 1 tasks
15/06/03 18:34:24 INFO TaskSetManager: Starting task 0.0 in stage 26.0 (TID 10, localhost, PROCESS_LOCAL, 1976 bytes)
15/06/03 18:34:24 INFO Executor: Running task 0.0 in stage 26.0 (TID 10)
15/06/03 18:34:24 INFO CacheManager: Partition rdd_54_0 not found, computing it
15/06/03 18:34:24 INFO BlockManager: Found block rdd_35_0 locally
15/06/03 18:34:24 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:24 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:24 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:24 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:24 INFO BlockManager: Removing broadcast 9
15/06/03 18:34:24 INFO BlockManager: Removing block broadcast_9
15/06/03 18:34:24 INFO MemoryStore: ensureFreeSpace(2976) called with curMem=272530, maxMem=559153152
15/06/03 18:34:24 INFO MemoryStore: Block broadcast_9 of size 6704 dropped from memory (free 558887326)
15/06/03 18:34:24 INFO BlockManager: Removing block broadcast_9_piece0
15/06/03 18:34:24 INFO MemoryStore: Block broadcast_9_piece0 of size 4179 dropped from memory (free 558891505)
15/06/03 18:34:24 INFO MemoryStore: Block rdd_54_0 stored as values in memory (estimated size 2.9 KB, free 533.0 MB)
15/06/03 18:34:24 INFO BlockManagerInfo: Removed broadcast_9_piece0 on localhost:60836 in memory (size: 4.1 KB, free: 533.2 MB)
15/06/03 18:34:24 INFO BlockManagerMaster: Updated info of block broadcast_9_piece0
15/06/03 18:34:24 INFO BlockManagerInfo: Added rdd_54_0 in memory on localhost:60836 (size: 2.9 KB, free: 533.2 MB)
15/06/03 18:34:24 INFO BlockManagerMaster: Updated info of block rdd_54_0
15/06/03 18:34:24 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:24 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:24 INFO ContextCleaner: Cleaned broadcast 9
15/06/03 18:34:24 INFO BlockManager: Removing broadcast 8
15/06/03 18:34:24 INFO BlockManager: Removing block broadcast_8
15/06/03 18:34:24 INFO MemoryStore: Block broadcast_8 of size 6272 dropped from memory (free 558894801)
15/06/03 18:34:24 INFO BlockManager: Removing block broadcast_8_piece0
15/06/03 18:34:24 INFO MemoryStore: Block broadcast_8_piece0 of size 3985 dropped from memory (free 558898786)
15/06/03 18:34:24 INFO BlockManagerInfo: Removed broadcast_8_piece0 on localhost:60836 in memory (size: 3.9 KB, free: 533.2 MB)
15/06/03 18:34:24 INFO BlockManagerMaster: Updated info of block broadcast_8_piece0
15/06/03 18:34:24 INFO ContextCleaner: Cleaned broadcast 8
15/06/03 18:34:24 INFO BlockManager: Removing broadcast 7
15/06/03 18:34:24 INFO BlockManager: Removing block broadcast_7
15/06/03 18:34:24 INFO MemoryStore: Block broadcast_7 of size 6480 dropped from memory (free 558905266)
15/06/03 18:34:24 INFO BlockManager: Removing block broadcast_7_piece0
15/06/03 18:34:24 INFO MemoryStore: Block broadcast_7_piece0 of size 4341 dropped from memory (free 558909607)
15/06/03 18:34:24 INFO BlockManagerInfo: Removed broadcast_7_piece0 on localhost:60836 in memory (size: 4.2 KB, free: 533.2 MB)
15/06/03 18:34:24 INFO BlockManagerMaster: Updated info of block broadcast_7_piece0
15/06/03 18:34:24 INFO ContextCleaner: Cleaned broadcast 7
15/06/03 18:34:24 INFO BlockManager: Removing broadcast 6
15/06/03 18:34:24 INFO BlockManager: Removing block broadcast_6_piece0
15/06/03 18:34:24 INFO MemoryStore: Block broadcast_6_piece0 of size 3717 dropped from memory (free 558913324)
15/06/03 18:34:24 INFO BlockManagerInfo: Removed broadcast_6_piece0 on localhost:60836 in memory (size: 3.6 KB, free: 533.2 MB)
15/06/03 18:34:24 INFO BlockManagerMaster: Updated info of block broadcast_6_piece0
15/06/03 18:34:24 INFO BlockManager: Removing block broadcast_6
15/06/03 18:34:24 INFO MemoryStore: Block broadcast_6 of size 5744 dropped from memory (free 558919068)
15/06/03 18:34:24 INFO ContextCleaner: Cleaned broadcast 6
15/06/03 18:34:24 INFO BlockManager: Removing broadcast 5
15/06/03 18:34:24 INFO BlockManager: Removing block broadcast_5_piece0
15/06/03 18:34:24 INFO MemoryStore: Block broadcast_5_piece0 of size 3803 dropped from memory (free 558922871)
15/06/03 18:34:24 INFO BlockManagerInfo: Removed broadcast_5_piece0 on localhost:60836 in memory (size: 3.7 KB, free: 533.2 MB)
15/06/03 18:34:24 INFO BlockManagerMaster: Updated info of block broadcast_5_piece0
15/06/03 18:34:24 INFO BlockManager: Removing block broadcast_5
15/06/03 18:34:24 INFO MemoryStore: Block broadcast_5 of size 5968 dropped from memory (free 558928839)
15/06/03 18:34:24 INFO ContextCleaner: Cleaned broadcast 5
15/06/03 18:34:24 INFO BlockManager: Removing broadcast 4
15/06/03 18:34:24 INFO BlockManager: Removing block broadcast_4
15/06/03 18:34:24 INFO MemoryStore: Block broadcast_4 of size 4464 dropped from memory (free 558933303)
15/06/03 18:34:24 INFO BlockManager: Removing block broadcast_4_piece0
15/06/03 18:34:24 INFO MemoryStore: Block broadcast_4_piece0 of size 2997 dropped from memory (free 558936300)
15/06/03 18:34:24 INFO BlockManagerInfo: Removed broadcast_4_piece0 on localhost:60836 in memory (size: 2.9 KB, free: 533.2 MB)
15/06/03 18:34:24 INFO BlockManagerMaster: Updated info of block broadcast_4_piece0
15/06/03 18:34:24 INFO Executor: Finished task 0.0 in stage 26.0 (TID 10). 2788 bytes result sent to driver
15/06/03 18:34:24 INFO ContextCleaner: Cleaned broadcast 4
15/06/03 18:34:24 INFO BlockManager: Removing broadcast 3
15/06/03 18:34:24 INFO BlockManager: Removing block broadcast_3_piece0
15/06/03 18:34:24 INFO MemoryStore: Block broadcast_3_piece0 of size 2849 dropped from memory (free 558939149)
15/06/03 18:34:24 INFO TaskSetManager: Finished task 0.0 in stage 26.0 (TID 10) in 30 ms on localhost (1/1)
15/06/03 18:34:24 INFO TaskSchedulerImpl: Removed TaskSet 26.0, whose tasks have all completed, from pool
15/06/03 18:34:24 INFO DAGScheduler: Stage 26 (mapPartitions at GraphImpl.scala:235) finished in 0.030 s
15/06/03 18:34:24 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:24 INFO DAGScheduler: running: Set()
15/06/03 18:34:24 INFO DAGScheduler: waiting: Set(Stage 27)
15/06/03 18:34:24 INFO DAGScheduler: failed: Set()
15/06/03 18:34:24 INFO BlockManagerInfo: Removed broadcast_3_piece0 on localhost:60836 in memory (size: 2.8 KB, free: 533.2 MB)
15/06/03 18:34:24 INFO BlockManagerMaster: Updated info of block broadcast_3_piece0
15/06/03 18:34:24 INFO BlockManager: Removing block broadcast_3
15/06/03 18:34:24 INFO MemoryStore: Block broadcast_3 of size 4024 dropped from memory (free 558943173)
15/06/03 18:34:24 INFO ContextCleaner: Cleaned broadcast 3
15/06/03 18:34:24 INFO BlockManager: Removing broadcast 2
15/06/03 18:34:24 INFO BlockManager: Removing block broadcast_2
15/06/03 18:34:24 INFO MemoryStore: Block broadcast_2 of size 3504 dropped from memory (free 558946677)
15/06/03 18:34:24 INFO DAGScheduler: Missing parents for Stage 27: List()
15/06/03 18:34:24 INFO BlockManager: Removing block broadcast_2_piece0
15/06/03 18:34:24 INFO MemoryStore: Block broadcast_2_piece0 of size 2501 dropped from memory (free 558949178)
15/06/03 18:34:24 INFO DAGScheduler: Submitting Stage 27 (MapPartitionsRDD[64] at map at VertexRDDImpl.scala:90), which is now runnable
15/06/03 18:34:24 INFO BlockManagerInfo: Removed broadcast_2_piece0 on localhost:60836 in memory (size: 2.4 KB, free: 533.2 MB)
15/06/03 18:34:24 INFO BlockManagerMaster: Updated info of block broadcast_2_piece0
15/06/03 18:34:24 INFO ContextCleaner: Cleaned broadcast 2
15/06/03 18:34:24 INFO BlockManager: Removing broadcast 1
15/06/03 18:34:24 INFO BlockManager: Removing block broadcast_1_piece0
15/06/03 18:34:24 INFO MemoryStore: ensureFreeSpace(6992) called with curMem=203974, maxMem=559153152
15/06/03 18:34:24 INFO MemoryStore: Block broadcast_1_piece0 of size 2091 dropped from memory (free 558951269)
15/06/03 18:34:24 INFO MemoryStore: Block broadcast_12 stored as values in memory (estimated size 6.8 KB, free 533.1 MB)
15/06/03 18:34:24 INFO BlockManagerInfo: Removed broadcast_1_piece0 on localhost:60836 in memory (size: 2.0 KB, free: 533.2 MB)
15/06/03 18:34:24 INFO BlockManagerMaster: Updated info of block broadcast_1_piece0
15/06/03 18:34:24 INFO MemoryStore: ensureFreeSpace(4344) called with curMem=208875, maxMem=559153152
15/06/03 18:34:24 INFO BlockManager: Removing block broadcast_1
15/06/03 18:34:24 INFO MemoryStore: Block broadcast_1 of size 2880 dropped from memory (free 558942813)
15/06/03 18:34:24 INFO MemoryStore: Block broadcast_12_piece0 stored as bytes in memory (estimated size 4.2 KB, free 533.0 MB)
15/06/03 18:34:24 INFO ContextCleaner: Cleaned broadcast 1
15/06/03 18:34:24 INFO BlockManagerInfo: Added broadcast_12_piece0 in memory on localhost:60836 (size: 4.2 KB, free: 533.2 MB)
15/06/03 18:34:24 INFO BlockManagerMaster: Updated info of block broadcast_12_piece0
15/06/03 18:34:24 INFO SparkContext: Created broadcast 12 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:24 INFO DAGScheduler: Submitting 1 missing tasks from Stage 27 (MapPartitionsRDD[64] at map at VertexRDDImpl.scala:90)
15/06/03 18:34:24 INFO TaskSchedulerImpl: Adding task set 27.0 with 1 tasks
15/06/03 18:34:24 INFO TaskSetManager: Starting task 0.0 in stage 27.0 (TID 11, localhost, PROCESS_LOCAL, 1651 bytes)
15/06/03 18:34:24 INFO Executor: Running task 0.0 in stage 27.0 (TID 11)
15/06/03 18:34:24 INFO CacheManager: Partition rdd_62_0 not found, computing it
15/06/03 18:34:24 INFO BlockManager: Found block rdd_48_0 locally
15/06/03 18:34:24 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:24 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:24 INFO MemoryStore: ensureFreeSpace(1664) called with curMem=210339, maxMem=559153152
15/06/03 18:34:24 INFO MemoryStore: Block rdd_62_0 stored as values in memory (estimated size 1664.0 B, free 533.0 MB)
15/06/03 18:34:24 INFO BlockManagerInfo: Added rdd_62_0 in memory on localhost:60836 (size: 1664.0 B, free: 533.2 MB)
15/06/03 18:34:24 INFO BlockManagerMaster: Updated info of block rdd_62_0
15/06/03 18:34:24 INFO Executor: Finished task 0.0 in stage 27.0 (TID 11). 2693 bytes result sent to driver
15/06/03 18:34:24 INFO TaskSetManager: Finished task 0.0 in stage 27.0 (TID 11) in 8 ms on localhost (1/1)
15/06/03 18:34:24 INFO TaskSchedulerImpl: Removed TaskSet 27.0, whose tasks have all completed, from pool
15/06/03 18:34:24 INFO DAGScheduler: Stage 27 (reduce at VertexRDDImpl.scala:90) finished in 0.009 s
15/06/03 18:34:24 INFO DAGScheduler: Job 2 finished: reduce at VertexRDDImpl.scala:90, took 0.148632 s
15/06/03 18:34:24 INFO Pregel: Pregel finished iteration 0
15/06/03 18:34:24 INFO ZippedPartitionsRDD2: Removing RDD 43 from persistence list
15/06/03 18:34:24 INFO ZippedPartitionsRDD2: Removing RDD 46 from persistence list
15/06/03 18:34:24 INFO BlockManager: Removing RDD 43
15/06/03 18:34:24 INFO BlockManager: Removing RDD 46
15/06/03 18:34:24 INFO MapPartitionsRDD: Removing RDD 29 from persistence list
15/06/03 18:34:24 INFO BlockManager: Removing RDD 29
15/06/03 18:34:24 INFO ZippedPartitionsRDD2: Removing RDD 35 from persistence list
15/06/03 18:34:24 INFO BlockManager: Removing block rdd_46_0
15/06/03 18:34:24 INFO MemoryStore: Block rdd_46_0 of size 1544 dropped from memory (free 558942693)
15/06/03 18:34:24 INFO BlockManager: Removing RDD 35
15/06/03 18:34:24 INFO BlockManager: Removing block rdd_35_0
15/06/03 18:34:24 INFO MemoryStore: Block rdd_35_0 of size 2776 dropped from memory (free 558945469)
15/06/03 18:34:24 INFO BlockManager: Removing block rdd_29_0
15/06/03 18:34:24 INFO MemoryStore: Block rdd_29_0 of size 1496 dropped from memory (free 558946965)
15/06/03 18:34:24 INFO SparkContext: Starting job: reduce at VertexRDDImpl.scala:90
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 2 is 145 bytes
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 3 is 145 bytes
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 5 is 145 bytes
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 4 is 145 bytes
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 1 is 145 bytes
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 0 is 145 bytes
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 8 is 145 bytes
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 7 is 145 bytes
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 6 is 145 bytes
15/06/03 18:34:24 INFO DAGScheduler: Registering RDD 71 (mapPartitions at VertexRDDImpl.scala:218)
15/06/03 18:34:24 INFO DAGScheduler: Registering RDD 75 (mapPartitions at VertexRDDImpl.scala:222)
15/06/03 18:34:24 INFO DAGScheduler: Registering RDD 79 (mapPartitions at GraphImpl.scala:235)
15/06/03 18:34:24 INFO DAGScheduler: Got job 3 (reduce at VertexRDDImpl.scala:90) with 1 output partitions (allowLocal=false)
15/06/03 18:34:24 INFO DAGScheduler: Final stage: Stage 57(reduce at VertexRDDImpl.scala:90)
15/06/03 18:34:24 INFO DAGScheduler: Parents of final stage: List(Stage 52, Stage 49, Stage 56, Stage 50, Stage 54)
15/06/03 18:34:24 INFO DAGScheduler: Missing parents: List(Stage 56)
15/06/03 18:34:24 INFO DAGScheduler: Submitting Stage 45 (ReplicatedVertexView.updateVertices - shippedVerts true false (broadcast) MapPartitionsRDD[71] at mapPartitions at VertexRDDImpl.scala:218), which has no missing parents
15/06/03 18:34:24 INFO MemoryStore: ensureFreeSpace(7232) called with curMem=206187, maxMem=559153152
15/06/03 18:34:24 INFO MemoryStore: Block broadcast_13 stored as values in memory (estimated size 7.1 KB, free 533.0 MB)
15/06/03 18:34:24 INFO MemoryStore: ensureFreeSpace(4438) called with curMem=213419, maxMem=559153152
15/06/03 18:34:24 INFO MemoryStore: Block broadcast_13_piece0 stored as bytes in memory (estimated size 4.3 KB, free 533.0 MB)
15/06/03 18:34:24 INFO BlockManagerInfo: Added broadcast_13_piece0 in memory on localhost:60836 (size: 4.3 KB, free: 533.2 MB)
15/06/03 18:34:24 INFO BlockManagerMaster: Updated info of block broadcast_13_piece0
15/06/03 18:34:24 INFO SparkContext: Created broadcast 13 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:24 INFO DAGScheduler: Submitting 1 missing tasks from Stage 45 (ReplicatedVertexView.updateVertices - shippedVerts true false (broadcast) MapPartitionsRDD[71] at mapPartitions at VertexRDDImpl.scala:218)
15/06/03 18:34:24 INFO TaskSchedulerImpl: Adding task set 45.0 with 1 tasks
15/06/03 18:34:24 INFO TaskSetManager: Starting task 0.0 in stage 45.0 (TID 12, localhost, PROCESS_LOCAL, 1736 bytes)
15/06/03 18:34:24 INFO Executor: Running task 0.0 in stage 45.0 (TID 12)
15/06/03 18:34:24 INFO DAGScheduler: Submitting Stage 55 (ReplicatedVertexView.withActiveSet - shippedActives (broadcast) MapPartitionsRDD[75] at mapPartitions at VertexRDDImpl.scala:222), which has no missing parents
15/06/03 18:34:24 INFO MemoryStore: ensureFreeSpace(6888) called with curMem=217857, maxMem=559153152
15/06/03 18:34:24 INFO MemoryStore: Block broadcast_14 stored as values in memory (estimated size 6.7 KB, free 533.0 MB)
15/06/03 18:34:24 INFO MemoryStore: ensureFreeSpace(4286) called with curMem=224745, maxMem=559153152
15/06/03 18:34:24 INFO MemoryStore: Block broadcast_14_piece0 stored as bytes in memory (estimated size 4.2 KB, free 533.0 MB)
15/06/03 18:34:24 INFO BlockManager: Found block rdd_48_0 locally
15/06/03 18:34:24 INFO BlockManagerInfo: Added broadcast_14_piece0 in memory on localhost:60836 (size: 4.2 KB, free: 533.2 MB)
15/06/03 18:34:24 INFO CacheManager: Partition rdd_67_0 not found, computing it
15/06/03 18:34:24 INFO BlockManager: Found block rdd_48_0 locally
15/06/03 18:34:24 INFO BlockManagerMaster: Updated info of block broadcast_14_piece0
15/06/03 18:34:24 INFO CacheManager: Partition rdd_65_0 not found, computing it
15/06/03 18:34:24 INFO SparkContext: Created broadcast 14 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:24 INFO BlockManager: Found block rdd_48_0 locally
15/06/03 18:34:24 INFO BlockManager: Found block rdd_62_0 locally
15/06/03 18:34:24 INFO DAGScheduler: Submitting 1 missing tasks from Stage 55 (ReplicatedVertexView.withActiveSet - shippedActives (broadcast) MapPartitionsRDD[75] at mapPartitions at VertexRDDImpl.scala:222)
15/06/03 18:34:24 INFO TaskSchedulerImpl: Adding task set 55.0 with 1 tasks
15/06/03 18:34:24 INFO MemoryStore: ensureFreeSpace(1544) called with curMem=229031, maxMem=559153152
15/06/03 18:34:24 INFO MemoryStore: Block rdd_65_0 stored as values in memory (estimated size 1544.0 B, free 533.0 MB)
15/06/03 18:34:24 INFO BlockManagerInfo: Added rdd_65_0 in memory on localhost:60836 (size: 1544.0 B, free: 533.2 MB)
15/06/03 18:34:24 INFO BlockManagerMaster: Updated info of block rdd_65_0
15/06/03 18:34:24 INFO MemoryStore: ensureFreeSpace(1496) called with curMem=230575, maxMem=559153152
15/06/03 18:34:24 INFO MemoryStore: Block rdd_67_0 stored as values in memory (estimated size 1496.0 B, free 533.0 MB)
15/06/03 18:34:24 INFO BlockManagerInfo: Added rdd_67_0 in memory on localhost:60836 (size: 1496.0 B, free: 533.2 MB)
15/06/03 18:34:24 INFO BlockManagerMaster: Updated info of block rdd_67_0
15/06/03 18:34:24 INFO Executor: Finished task 0.0 in stage 45.0 (TID 12). 2627 bytes result sent to driver
15/06/03 18:34:24 INFO TaskSetManager: Starting task 0.0 in stage 55.0 (TID 13, localhost, PROCESS_LOCAL, 1672 bytes)
15/06/03 18:34:24 INFO Executor: Running task 0.0 in stage 55.0 (TID 13)
15/06/03 18:34:24 INFO TaskSetManager: Finished task 0.0 in stage 45.0 (TID 12) in 24 ms on localhost (1/1)
15/06/03 18:34:24 INFO TaskSchedulerImpl: Removed TaskSet 45.0, whose tasks have all completed, from pool
15/06/03 18:34:24 INFO DAGScheduler: Stage 45 (mapPartitions at VertexRDDImpl.scala:218) finished in 0.025 s
15/06/03 18:34:24 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:24 INFO DAGScheduler: running: Set(Stage 55)
15/06/03 18:34:24 INFO DAGScheduler: waiting: Set(Stage 56, Stage 57)
15/06/03 18:34:24 INFO DAGScheduler: failed: Set()
15/06/03 18:34:24 INFO DAGScheduler: Missing parents for Stage 56: List(Stage 55)
15/06/03 18:34:24 INFO BlockManager: Found block rdd_65_0 locally
15/06/03 18:34:24 INFO DAGScheduler: Missing parents for Stage 57: List(Stage 56)
15/06/03 18:34:24 INFO Executor: Finished task 0.0 in stage 55.0 (TID 13). 2003 bytes result sent to driver
15/06/03 18:34:24 INFO TaskSetManager: Finished task 0.0 in stage 55.0 (TID 13) in 13 ms on localhost (1/1)
15/06/03 18:34:24 INFO TaskSchedulerImpl: Removed TaskSet 55.0, whose tasks have all completed, from pool
15/06/03 18:34:24 INFO DAGScheduler: Stage 55 (mapPartitions at VertexRDDImpl.scala:222) finished in 0.031 s
15/06/03 18:34:24 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:24 INFO DAGScheduler: running: Set()
15/06/03 18:34:24 INFO DAGScheduler: waiting: Set(Stage 56, Stage 57)
15/06/03 18:34:24 INFO DAGScheduler: failed: Set()
15/06/03 18:34:24 INFO DAGScheduler: Missing parents for Stage 56: List()
15/06/03 18:34:24 INFO DAGScheduler: Missing parents for Stage 57: List(Stage 56)
15/06/03 18:34:24 INFO DAGScheduler: Submitting Stage 56 (GraphImpl.aggregateMessages - preAgg MapPartitionsRDD[79] at mapPartitions at GraphImpl.scala:235), which is now runnable
15/06/03 18:34:24 INFO MemoryStore: ensureFreeSpace(7304) called with curMem=232071, maxMem=559153152
15/06/03 18:34:24 INFO MemoryStore: Block broadcast_15 stored as values in memory (estimated size 7.1 KB, free 533.0 MB)
15/06/03 18:34:24 INFO MemoryStore: ensureFreeSpace(4628) called with curMem=239375, maxMem=559153152
15/06/03 18:34:24 INFO MemoryStore: Block broadcast_15_piece0 stored as bytes in memory (estimated size 4.5 KB, free 533.0 MB)
15/06/03 18:34:24 INFO BlockManagerInfo: Added broadcast_15_piece0 in memory on localhost:60836 (size: 4.5 KB, free: 533.2 MB)
15/06/03 18:34:24 INFO BlockManagerMaster: Updated info of block broadcast_15_piece0
15/06/03 18:34:24 INFO SparkContext: Created broadcast 15 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:24 INFO DAGScheduler: Submitting 1 missing tasks from Stage 56 (GraphImpl.aggregateMessages - preAgg MapPartitionsRDD[79] at mapPartitions at GraphImpl.scala:235)
15/06/03 18:34:24 INFO TaskSchedulerImpl: Adding task set 56.0 with 1 tasks
15/06/03 18:34:24 INFO TaskSetManager: Starting task 0.0 in stage 56.0 (TID 14, localhost, PROCESS_LOCAL, 2017 bytes)
15/06/03 18:34:24 INFO Executor: Running task 0.0 in stage 56.0 (TID 14)
15/06/03 18:34:24 INFO CacheManager: Partition rdd_73_0 not found, computing it
15/06/03 18:34:24 INFO BlockManager: Found block rdd_54_0 locally
15/06/03 18:34:24 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:24 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:24 INFO MemoryStore: ensureFreeSpace(2976) called with curMem=244003, maxMem=559153152
15/06/03 18:34:24 INFO MemoryStore: Block rdd_73_0 stored as values in memory (estimated size 2.9 KB, free 533.0 MB)
15/06/03 18:34:24 INFO BlockManagerInfo: Added rdd_73_0 in memory on localhost:60836 (size: 2.9 KB, free: 533.2 MB)
15/06/03 18:34:24 INFO BlockManagerMaster: Updated info of block rdd_73_0
15/06/03 18:34:24 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:24 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:24 INFO Executor: Finished task 0.0 in stage 56.0 (TID 14). 2788 bytes result sent to driver
15/06/03 18:34:24 INFO TaskSetManager: Finished task 0.0 in stage 56.0 (TID 14) in 17 ms on localhost (1/1)
15/06/03 18:34:24 INFO TaskSchedulerImpl: Removed TaskSet 56.0, whose tasks have all completed, from pool
15/06/03 18:34:24 INFO DAGScheduler: Stage 56 (mapPartitions at GraphImpl.scala:235) finished in 0.018 s
15/06/03 18:34:24 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:24 INFO DAGScheduler: running: Set()
15/06/03 18:34:24 INFO DAGScheduler: waiting: Set(Stage 57)
15/06/03 18:34:24 INFO DAGScheduler: failed: Set()
15/06/03 18:34:24 INFO DAGScheduler: Missing parents for Stage 57: List()
15/06/03 18:34:24 INFO DAGScheduler: Submitting Stage 57 (MapPartitionsRDD[83] at map at VertexRDDImpl.scala:90), which is now runnable
15/06/03 18:34:24 INFO MemoryStore: ensureFreeSpace(7520) called with curMem=246979, maxMem=559153152
15/06/03 18:34:24 INFO MemoryStore: Block broadcast_16 stored as values in memory (estimated size 7.3 KB, free 533.0 MB)
15/06/03 18:34:24 INFO MemoryStore: ensureFreeSpace(4584) called with curMem=254499, maxMem=559153152
15/06/03 18:34:24 INFO MemoryStore: Block broadcast_16_piece0 stored as bytes in memory (estimated size 4.5 KB, free 533.0 MB)
15/06/03 18:34:24 INFO BlockManagerInfo: Added broadcast_16_piece0 in memory on localhost:60836 (size: 4.5 KB, free: 533.2 MB)
15/06/03 18:34:24 INFO BlockManagerMaster: Updated info of block broadcast_16_piece0
15/06/03 18:34:24 INFO SparkContext: Created broadcast 16 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:24 INFO DAGScheduler: Submitting 1 missing tasks from Stage 57 (MapPartitionsRDD[83] at map at VertexRDDImpl.scala:90)
15/06/03 18:34:24 INFO TaskSchedulerImpl: Adding task set 57.0 with 1 tasks
15/06/03 18:34:24 INFO TaskSetManager: Starting task 0.0 in stage 57.0 (TID 15, localhost, PROCESS_LOCAL, 1756 bytes)
15/06/03 18:34:24 INFO Executor: Running task 0.0 in stage 57.0 (TID 15)
15/06/03 18:34:24 INFO CacheManager: Partition rdd_81_0 not found, computing it
15/06/03 18:34:24 INFO BlockManager: Found block rdd_67_0 locally
15/06/03 18:34:24 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:24 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:24 INFO MemoryStore: ensureFreeSpace(1664) called with curMem=259083, maxMem=559153152
15/06/03 18:34:24 INFO MemoryStore: Block rdd_81_0 stored as values in memory (estimated size 1664.0 B, free 533.0 MB)
15/06/03 18:34:24 INFO BlockManagerInfo: Added rdd_81_0 in memory on localhost:60836 (size: 1664.0 B, free: 533.2 MB)
15/06/03 18:34:24 INFO BlockManagerMaster: Updated info of block rdd_81_0
15/06/03 18:34:24 INFO Executor: Finished task 0.0 in stage 57.0 (TID 15). 2693 bytes result sent to driver
15/06/03 18:34:24 INFO TaskSetManager: Finished task 0.0 in stage 57.0 (TID 15) in 10 ms on localhost (1/1)
15/06/03 18:34:24 INFO TaskSchedulerImpl: Removed TaskSet 57.0, whose tasks have all completed, from pool
15/06/03 18:34:24 INFO DAGScheduler: Stage 57 (reduce at VertexRDDImpl.scala:90) finished in 0.010 s
15/06/03 18:34:24 INFO DAGScheduler: Job 3 finished: reduce at VertexRDDImpl.scala:90, took 0.097724 s
15/06/03 18:34:24 INFO Pregel: Pregel finished iteration 1
15/06/03 18:34:24 INFO ZippedPartitionsRDD2: Removing RDD 62 from persistence list
15/06/03 18:34:24 INFO BlockManager: Removing RDD 62
15/06/03 18:34:24 INFO ZippedPartitionsRDD2: Removing RDD 65 from persistence list
15/06/03 18:34:24 INFO BlockManager: Removing block rdd_62_0
15/06/03 18:34:24 INFO MemoryStore: Block rdd_62_0 of size 1664 dropped from memory (free 558894069)
15/06/03 18:34:24 INFO BlockManager: Removing RDD 65
15/06/03 18:34:24 INFO BlockManager: Removing block rdd_65_0
15/06/03 18:34:24 INFO ZippedPartitionsRDD2: Removing RDD 48 from persistence list
15/06/03 18:34:24 INFO MemoryStore: Block rdd_65_0 of size 1544 dropped from memory (free 558895613)
15/06/03 18:34:24 INFO BlockManager: Removing RDD 48
15/06/03 18:34:24 INFO ZippedPartitionsRDD2: Removing RDD 54 from persistence list
15/06/03 18:34:24 INFO BlockManager: Removing block rdd_48_0
15/06/03 18:34:24 INFO BlockManager: Removing RDD 54
15/06/03 18:34:24 INFO MemoryStore: Block rdd_48_0 of size 1496 dropped from memory (free 558897109)
15/06/03 18:34:24 INFO BlockManager: Removing block rdd_54_0
15/06/03 18:34:24 INFO MemoryStore: Block rdd_54_0 of size 2976 dropped from memory (free 558900085)
15/06/03 18:34:24 INFO SparkContext: Starting job: reduce at VertexRDDImpl.scala:90
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 2 is 145 bytes
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 3 is 145 bytes
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 5 is 145 bytes
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 4 is 145 bytes
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 1 is 145 bytes
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 0 is 145 bytes
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 8 is 145 bytes
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 7 is 145 bytes
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 6 is 145 bytes
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 11 is 145 bytes
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 10 is 145 bytes
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 9 is 145 bytes
15/06/03 18:34:24 INFO DAGScheduler: Registering RDD 90 (mapPartitions at VertexRDDImpl.scala:218)
15/06/03 18:34:24 INFO DAGScheduler: Registering RDD 94 (mapPartitions at VertexRDDImpl.scala:222)
15/06/03 18:34:24 INFO DAGScheduler: Registering RDD 98 (mapPartitions at GraphImpl.scala:235)
15/06/03 18:34:24 INFO DAGScheduler: Got job 4 (reduce at VertexRDDImpl.scala:90) with 1 output partitions (allowLocal=false)
15/06/03 18:34:24 INFO DAGScheduler: Final stage: Stage 102(reduce at VertexRDDImpl.scala:90)
15/06/03 18:34:24 INFO DAGScheduler: Parents of final stage: List(Stage 99, Stage 101, Stage 93, Stage 97, Stage 95, Stage 92)
15/06/03 18:34:24 INFO DAGScheduler: Missing parents: List(Stage 101)
15/06/03 18:34:24 INFO DAGScheduler: Submitting Stage 87 (ReplicatedVertexView.updateVertices - shippedVerts true false (broadcast) MapPartitionsRDD[90] at mapPartitions at VertexRDDImpl.scala:218), which has no missing parents
15/06/03 18:34:24 INFO MemoryStore: ensureFreeSpace(7760) called with curMem=253067, maxMem=559153152
15/06/03 18:34:24 INFO MemoryStore: Block broadcast_17 stored as values in memory (estimated size 7.6 KB, free 533.0 MB)
15/06/03 18:34:24 INFO MemoryStore: ensureFreeSpace(4673) called with curMem=260827, maxMem=559153152
15/06/03 18:34:24 INFO MemoryStore: Block broadcast_17_piece0 stored as bytes in memory (estimated size 4.6 KB, free 533.0 MB)
15/06/03 18:34:24 INFO BlockManagerInfo: Added broadcast_17_piece0 in memory on localhost:60836 (size: 4.6 KB, free: 533.2 MB)
15/06/03 18:34:24 INFO BlockManagerMaster: Updated info of block broadcast_17_piece0
15/06/03 18:34:24 INFO SparkContext: Created broadcast 17 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:24 INFO DAGScheduler: Submitting 1 missing tasks from Stage 87 (ReplicatedVertexView.updateVertices - shippedVerts true false (broadcast) MapPartitionsRDD[90] at mapPartitions at VertexRDDImpl.scala:218)
15/06/03 18:34:24 INFO TaskSchedulerImpl: Adding task set 87.0 with 1 tasks
15/06/03 18:34:24 INFO DAGScheduler: Submitting Stage 100 (ReplicatedVertexView.withActiveSet - shippedActives (broadcast) MapPartitionsRDD[94] at mapPartitions at VertexRDDImpl.scala:222), which has no missing parents
15/06/03 18:34:24 INFO TaskSetManager: Starting task 0.0 in stage 87.0 (TID 16, localhost, PROCESS_LOCAL, 1841 bytes)
15/06/03 18:34:24 INFO Executor: Running task 0.0 in stage 87.0 (TID 16)
15/06/03 18:34:24 INFO MemoryStore: ensureFreeSpace(7416) called with curMem=265500, maxMem=559153152
15/06/03 18:34:24 INFO MemoryStore: Block broadcast_18 stored as values in memory (estimated size 7.2 KB, free 533.0 MB)
15/06/03 18:34:24 INFO BlockManager: Found block rdd_67_0 locally
15/06/03 18:34:24 INFO CacheManager: Partition rdd_86_0 not found, computing it
15/06/03 18:34:24 INFO BlockManager: Found block rdd_67_0 locally
15/06/03 18:34:24 INFO CacheManager: Partition rdd_84_0 not found, computing it
15/06/03 18:34:24 INFO MemoryStore: ensureFreeSpace(4548) called with curMem=272916, maxMem=559153152
15/06/03 18:34:24 INFO BlockManager: Found block rdd_67_0 locally
15/06/03 18:34:24 INFO BlockManager: Found block rdd_81_0 locally
15/06/03 18:34:24 INFO MemoryStore: Block broadcast_18_piece0 stored as bytes in memory (estimated size 4.4 KB, free 533.0 MB)
15/06/03 18:34:24 INFO BlockManagerInfo: Added broadcast_18_piece0 in memory on localhost:60836 (size: 4.4 KB, free: 533.2 MB)
15/06/03 18:34:24 INFO MemoryStore: ensureFreeSpace(1544) called with curMem=277464, maxMem=559153152
15/06/03 18:34:24 INFO BlockManagerMaster: Updated info of block broadcast_18_piece0
15/06/03 18:34:24 INFO MemoryStore: Block rdd_84_0 stored as values in memory (estimated size 1544.0 B, free 533.0 MB)
15/06/03 18:34:24 INFO SparkContext: Created broadcast 18 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:24 INFO DAGScheduler: Submitting 1 missing tasks from Stage 100 (ReplicatedVertexView.withActiveSet - shippedActives (broadcast) MapPartitionsRDD[94] at mapPartitions at VertexRDDImpl.scala:222)
15/06/03 18:34:24 INFO BlockManagerInfo: Added rdd_84_0 in memory on localhost:60836 (size: 1544.0 B, free: 533.2 MB)
15/06/03 18:34:24 INFO TaskSchedulerImpl: Adding task set 100.0 with 1 tasks
15/06/03 18:34:24 INFO BlockManagerMaster: Updated info of block rdd_84_0
15/06/03 18:34:24 INFO MemoryStore: ensureFreeSpace(1496) called with curMem=279008, maxMem=559153152
15/06/03 18:34:24 INFO MemoryStore: Block rdd_86_0 stored as values in memory (estimated size 1496.0 B, free 533.0 MB)
15/06/03 18:34:24 INFO BlockManagerInfo: Added rdd_86_0 in memory on localhost:60836 (size: 1496.0 B, free: 533.2 MB)
15/06/03 18:34:24 INFO BlockManagerMaster: Updated info of block rdd_86_0
15/06/03 18:34:24 INFO Executor: Finished task 0.0 in stage 87.0 (TID 16). 2627 bytes result sent to driver
15/06/03 18:34:24 INFO TaskSetManager: Starting task 0.0 in stage 100.0 (TID 17, localhost, PROCESS_LOCAL, 1777 bytes)
15/06/03 18:34:24 INFO Executor: Running task 0.0 in stage 100.0 (TID 17)
15/06/03 18:34:24 INFO TaskSetManager: Finished task 0.0 in stage 87.0 (TID 16) in 17 ms on localhost (1/1)
15/06/03 18:34:24 INFO TaskSchedulerImpl: Removed TaskSet 87.0, whose tasks have all completed, from pool
15/06/03 18:34:24 INFO DAGScheduler: Stage 87 (mapPartitions at VertexRDDImpl.scala:218) finished in 0.018 s
15/06/03 18:34:24 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:24 INFO DAGScheduler: running: Set(Stage 100)
15/06/03 18:34:24 INFO DAGScheduler: waiting: Set(Stage 102, Stage 101)
15/06/03 18:34:24 INFO DAGScheduler: failed: Set()
15/06/03 18:34:24 INFO DAGScheduler: Missing parents for Stage 102: List(Stage 101)
15/06/03 18:34:24 INFO BlockManager: Found block rdd_84_0 locally
15/06/03 18:34:24 INFO DAGScheduler: Missing parents for Stage 101: List(Stage 100)
15/06/03 18:34:24 INFO Executor: Finished task 0.0 in stage 100.0 (TID 17). 2003 bytes result sent to driver
15/06/03 18:34:24 INFO TaskSetManager: Finished task 0.0 in stage 100.0 (TID 17) in 13 ms on localhost (1/1)
15/06/03 18:34:24 INFO TaskSchedulerImpl: Removed TaskSet 100.0, whose tasks have all completed, from pool
15/06/03 18:34:24 INFO DAGScheduler: Stage 100 (mapPartitions at VertexRDDImpl.scala:222) finished in 0.024 s
15/06/03 18:34:24 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:24 INFO DAGScheduler: running: Set()
15/06/03 18:34:24 INFO DAGScheduler: waiting: Set(Stage 102, Stage 101)
15/06/03 18:34:24 INFO DAGScheduler: failed: Set()
15/06/03 18:34:24 INFO DAGScheduler: Missing parents for Stage 102: List(Stage 101)
15/06/03 18:34:24 INFO DAGScheduler: Missing parents for Stage 101: List()
15/06/03 18:34:24 INFO DAGScheduler: Submitting Stage 101 (GraphImpl.aggregateMessages - preAgg MapPartitionsRDD[98] at mapPartitions at GraphImpl.scala:235), which is now runnable
15/06/03 18:34:24 INFO MemoryStore: ensureFreeSpace(7536) called with curMem=280504, maxMem=559153152
15/06/03 18:34:24 INFO MemoryStore: Block broadcast_19 stored as values in memory (estimated size 7.4 KB, free 533.0 MB)
15/06/03 18:34:24 INFO MemoryStore: ensureFreeSpace(4796) called with curMem=288040, maxMem=559153152
15/06/03 18:34:24 INFO MemoryStore: Block broadcast_19_piece0 stored as bytes in memory (estimated size 4.7 KB, free 533.0 MB)
15/06/03 18:34:24 INFO BlockManagerInfo: Added broadcast_19_piece0 in memory on localhost:60836 (size: 4.7 KB, free: 533.2 MB)
15/06/03 18:34:24 INFO BlockManagerMaster: Updated info of block broadcast_19_piece0
15/06/03 18:34:24 INFO SparkContext: Created broadcast 19 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:24 INFO DAGScheduler: Submitting 1 missing tasks from Stage 101 (GraphImpl.aggregateMessages - preAgg MapPartitionsRDD[98] at mapPartitions at GraphImpl.scala:235)
15/06/03 18:34:24 INFO TaskSchedulerImpl: Adding task set 101.0 with 1 tasks
15/06/03 18:34:24 INFO TaskSetManager: Starting task 0.0 in stage 101.0 (TID 18, localhost, PROCESS_LOCAL, 2058 bytes)
15/06/03 18:34:24 INFO Executor: Running task 0.0 in stage 101.0 (TID 18)
15/06/03 18:34:24 INFO CacheManager: Partition rdd_92_0 not found, computing it
15/06/03 18:34:24 INFO BlockManager: Found block rdd_73_0 locally
15/06/03 18:34:24 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:24 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:24 INFO MemoryStore: ensureFreeSpace(2976) called with curMem=292836, maxMem=559153152
15/06/03 18:34:24 INFO MemoryStore: Block rdd_92_0 stored as values in memory (estimated size 2.9 KB, free 533.0 MB)
15/06/03 18:34:24 INFO BlockManagerInfo: Added rdd_92_0 in memory on localhost:60836 (size: 2.9 KB, free: 533.2 MB)
15/06/03 18:34:24 INFO BlockManagerMaster: Updated info of block rdd_92_0
15/06/03 18:34:24 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:24 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
15/06/03 18:34:24 INFO Executor: Finished task 0.0 in stage 101.0 (TID 18). 2788 bytes result sent to driver
15/06/03 18:34:24 INFO TaskSetManager: Finished task 0.0 in stage 101.0 (TID 18) in 17 ms on localhost (1/1)
15/06/03 18:34:24 INFO TaskSchedulerImpl: Removed TaskSet 101.0, whose tasks have all completed, from pool
15/06/03 18:34:24 INFO DAGScheduler: Stage 101 (mapPartitions at GraphImpl.scala:235) finished in 0.017 s
15/06/03 18:34:24 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:24 INFO DAGScheduler: running: Set()
15/06/03 18:34:24 INFO DAGScheduler: waiting: Set(Stage 102)
15/06/03 18:34:24 INFO DAGScheduler: failed: Set()
15/06/03 18:34:24 INFO DAGScheduler: Missing parents for Stage 102: List()
15/06/03 18:34:24 INFO DAGScheduler: Submitting Stage 102 (MapPartitionsRDD[102] at map at VertexRDDImpl.scala:90), which is now runnable
15/06/03 18:34:24 INFO MemoryStore: ensureFreeSpace(8040) called with curMem=295812, maxMem=559153152
15/06/03 18:34:24 INFO MemoryStore: Block broadcast_20 stored as values in memory (estimated size 7.9 KB, free 533.0 MB)
15/06/03 18:34:24 INFO MemoryStore: ensureFreeSpace(4815) called with curMem=303852, maxMem=559153152
15/06/03 18:34:24 INFO MemoryStore: Block broadcast_20_piece0 stored as bytes in memory (estimated size 4.7 KB, free 533.0 MB)
15/06/03 18:34:24 INFO BlockManagerInfo: Added broadcast_20_piece0 in memory on localhost:60836 (size: 4.7 KB, free: 533.2 MB)
15/06/03 18:34:24 INFO BlockManagerMaster: Updated info of block broadcast_20_piece0
15/06/03 18:34:24 INFO SparkContext: Created broadcast 20 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:24 INFO DAGScheduler: Submitting 1 missing tasks from Stage 102 (MapPartitionsRDD[102] at map at VertexRDDImpl.scala:90)
15/06/03 18:34:24 INFO TaskSchedulerImpl: Adding task set 102.0 with 1 tasks
15/06/03 18:34:24 INFO TaskSetManager: Starting task 0.0 in stage 102.0 (TID 19, localhost, PROCESS_LOCAL, 1861 bytes)
15/06/03 18:34:24 INFO Executor: Running task 0.0 in stage 102.0 (TID 19)
15/06/03 18:34:24 INFO CacheManager: Partition rdd_100_0 not found, computing it
15/06/03 18:34:24 INFO BlockManager: Found block rdd_86_0 locally
15/06/03 18:34:24 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:24 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:24 INFO MemoryStore: ensureFreeSpace(1664) called with curMem=308667, maxMem=559153152
15/06/03 18:34:24 INFO MemoryStore: Block rdd_100_0 stored as values in memory (estimated size 1664.0 B, free 533.0 MB)
15/06/03 18:34:24 INFO BlockManagerInfo: Added rdd_100_0 in memory on localhost:60836 (size: 1664.0 B, free: 533.2 MB)
15/06/03 18:34:24 INFO BlockManagerMaster: Updated info of block rdd_100_0
15/06/03 18:34:24 INFO Executor: Finished task 0.0 in stage 102.0 (TID 19). 2693 bytes result sent to driver
15/06/03 18:34:24 INFO TaskSetManager: Finished task 0.0 in stage 102.0 (TID 19) in 10 ms on localhost (1/1)
15/06/03 18:34:24 INFO TaskSchedulerImpl: Removed TaskSet 102.0, whose tasks have all completed, from pool
15/06/03 18:34:24 INFO DAGScheduler: Stage 102 (reduce at VertexRDDImpl.scala:90) finished in 0.010 s
15/06/03 18:34:24 INFO DAGScheduler: Job 4 finished: reduce at VertexRDDImpl.scala:90, took 0.100709 s
15/06/03 18:34:24 INFO Pregel: Pregel finished iteration 2
15/06/03 18:34:24 INFO ZippedPartitionsRDD2: Removing RDD 81 from persistence list
15/06/03 18:34:24 INFO BlockManager: Removing RDD 81
15/06/03 18:34:24 INFO ZippedPartitionsRDD2: Removing RDD 84 from persistence list
15/06/03 18:34:24 INFO BlockManager: Removing block rdd_81_0
15/06/03 18:34:24 INFO MemoryStore: Block rdd_81_0 of size 1664 dropped from memory (free 558844485)
15/06/03 18:34:24 INFO BlockManager: Removing RDD 84
15/06/03 18:34:24 INFO ZippedPartitionsRDD2: Removing RDD 67 from persistence list
15/06/03 18:34:24 INFO BlockManager: Removing block rdd_84_0
15/06/03 18:34:24 INFO MemoryStore: Block rdd_84_0 of size 1544 dropped from memory (free 558846029)
15/06/03 18:34:24 INFO BlockManager: Removing RDD 67
15/06/03 18:34:24 INFO ZippedPartitionsRDD2: Removing RDD 73 from persistence list
15/06/03 18:34:24 INFO BlockManager: Removing block rdd_67_0
15/06/03 18:34:24 INFO MemoryStore: Block rdd_67_0 of size 1496 dropped from memory (free 558847525)
15/06/03 18:34:24 INFO BlockManager: Removing RDD 73
15/06/03 18:34:24 INFO BlockManager: Removing block rdd_73_0
15/06/03 18:34:24 INFO MemoryStore: Block rdd_73_0 of size 2976 dropped from memory (free 558850501)
15/06/03 18:34:24 INFO SparkContext: Starting job: reduce at VertexRDDImpl.scala:90
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 2 is 145 bytes
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 3 is 145 bytes
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 5 is 145 bytes
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 4 is 145 bytes
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 1 is 145 bytes
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 0 is 145 bytes
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 8 is 145 bytes
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 7 is 145 bytes
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 6 is 145 bytes
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 11 is 145 bytes
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 10 is 145 bytes
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 9 is 145 bytes
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 14 is 145 bytes
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 13 is 145 bytes
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 12 is 145 bytes
15/06/03 18:34:24 INFO DAGScheduler: Registering RDD 109 (mapPartitions at VertexRDDImpl.scala:218)
15/06/03 18:34:24 INFO DAGScheduler: Registering RDD 113 (mapPartitions at VertexRDDImpl.scala:222)
15/06/03 18:34:24 INFO DAGScheduler: Registering RDD 117 (mapPartitions at GraphImpl.scala:235)
15/06/03 18:34:24 INFO DAGScheduler: Got job 5 (reduce at VertexRDDImpl.scala:90) with 1 output partitions (allowLocal=false)
15/06/03 18:34:24 INFO DAGScheduler: Final stage: Stage 165(reduce at VertexRDDImpl.scala:90)
15/06/03 18:34:24 INFO DAGScheduler: Parents of final stage: List(Stage 153, Stage 154, Stage 158, Stage 162, Stage 156, Stage 160, Stage 164)
15/06/03 18:34:24 INFO DAGScheduler: Missing parents: List(Stage 164)
15/06/03 18:34:24 INFO DAGScheduler: Submitting Stage 147 (ReplicatedVertexView.updateVertices - shippedVerts true false (broadcast) MapPartitionsRDD[109] at mapPartitions at VertexRDDImpl.scala:218), which has no missing parents
15/06/03 18:34:24 INFO MemoryStore: ensureFreeSpace(8280) called with curMem=302651, maxMem=559153152
15/06/03 18:34:24 INFO MemoryStore: Block broadcast_21 stored as values in memory (estimated size 8.1 KB, free 533.0 MB)
15/06/03 18:34:24 INFO MemoryStore: ensureFreeSpace(4909) called with curMem=310931, maxMem=559153152
15/06/03 18:34:24 INFO MemoryStore: Block broadcast_21_piece0 stored as bytes in memory (estimated size 4.8 KB, free 532.9 MB)
15/06/03 18:34:24 INFO BlockManagerInfo: Added broadcast_21_piece0 in memory on localhost:60836 (size: 4.8 KB, free: 533.2 MB)
15/06/03 18:34:24 INFO BlockManagerMaster: Updated info of block broadcast_21_piece0
15/06/03 18:34:24 INFO SparkContext: Created broadcast 21 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:24 INFO DAGScheduler: Submitting 1 missing tasks from Stage 147 (ReplicatedVertexView.updateVertices - shippedVerts true false (broadcast) MapPartitionsRDD[109] at mapPartitions at VertexRDDImpl.scala:218)
15/06/03 18:34:24 INFO TaskSchedulerImpl: Adding task set 147.0 with 1 tasks
15/06/03 18:34:24 INFO DAGScheduler: Submitting Stage 163 (ReplicatedVertexView.withActiveSet - shippedActives (broadcast) MapPartitionsRDD[113] at mapPartitions at VertexRDDImpl.scala:222), which has no missing parents
15/06/03 18:34:24 INFO TaskSetManager: Starting task 0.0 in stage 147.0 (TID 20, localhost, PROCESS_LOCAL, 1946 bytes)
15/06/03 18:34:24 INFO Executor: Running task 0.0 in stage 147.0 (TID 20)
15/06/03 18:34:24 INFO MemoryStore: ensureFreeSpace(7944) called with curMem=315840, maxMem=559153152
15/06/03 18:34:24 INFO MemoryStore: Block broadcast_22 stored as values in memory (estimated size 7.8 KB, free 532.9 MB)
15/06/03 18:34:24 INFO BlockManager: Found block rdd_86_0 locally
15/06/03 18:34:24 INFO MemoryStore: ensureFreeSpace(4778) called with curMem=323784, maxMem=559153152
15/06/03 18:34:24 INFO MemoryStore: Block broadcast_22_piece0 stored as bytes in memory (estimated size 4.7 KB, free 532.9 MB)
15/06/03 18:34:24 INFO CacheManager: Partition rdd_105_0 not found, computing it
15/06/03 18:34:24 INFO BlockManager: Found block rdd_86_0 locally
15/06/03 18:34:24 INFO BlockManagerInfo: Added broadcast_22_piece0 in memory on localhost:60836 (size: 4.7 KB, free: 533.2 MB)
15/06/03 18:34:24 INFO BlockManagerMaster: Updated info of block broadcast_22_piece0
15/06/03 18:34:24 INFO CacheManager: Partition rdd_103_0 not found, computing it
15/06/03 18:34:24 INFO BlockManager: Found block rdd_86_0 locally
15/06/03 18:34:24 INFO SparkContext: Created broadcast 22 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:24 INFO BlockManager: Found block rdd_100_0 locally
15/06/03 18:34:24 INFO DAGScheduler: Submitting 1 missing tasks from Stage 163 (ReplicatedVertexView.withActiveSet - shippedActives (broadcast) MapPartitionsRDD[113] at mapPartitions at VertexRDDImpl.scala:222)
15/06/03 18:34:24 INFO TaskSchedulerImpl: Adding task set 163.0 with 1 tasks
15/06/03 18:34:24 INFO MemoryStore: ensureFreeSpace(1544) called with curMem=328562, maxMem=559153152
15/06/03 18:34:24 INFO MemoryStore: Block rdd_103_0 stored as values in memory (estimated size 1544.0 B, free 532.9 MB)
15/06/03 18:34:24 INFO BlockManagerInfo: Added rdd_103_0 in memory on localhost:60836 (size: 1544.0 B, free: 533.2 MB)
15/06/03 18:34:24 INFO BlockManagerMaster: Updated info of block rdd_103_0
15/06/03 18:34:24 INFO MemoryStore: ensureFreeSpace(1496) called with curMem=330106, maxMem=559153152
15/06/03 18:34:24 INFO MemoryStore: Block rdd_105_0 stored as values in memory (estimated size 1496.0 B, free 532.9 MB)
15/06/03 18:34:24 INFO BlockManagerInfo: Added rdd_105_0 in memory on localhost:60836 (size: 1496.0 B, free: 533.2 MB)
15/06/03 18:34:24 INFO BlockManagerMaster: Updated info of block rdd_105_0
15/06/03 18:34:24 INFO Executor: Finished task 0.0 in stage 147.0 (TID 20). 2627 bytes result sent to driver
15/06/03 18:34:24 INFO TaskSetManager: Starting task 0.0 in stage 163.0 (TID 21, localhost, PROCESS_LOCAL, 1882 bytes)
15/06/03 18:34:24 INFO TaskSetManager: Finished task 0.0 in stage 147.0 (TID 20) in 28 ms on localhost (1/1)
15/06/03 18:34:24 INFO Executor: Running task 0.0 in stage 163.0 (TID 21)
15/06/03 18:34:24 INFO TaskSchedulerImpl: Removed TaskSet 147.0, whose tasks have all completed, from pool
15/06/03 18:34:24 INFO DAGScheduler: Stage 147 (mapPartitions at VertexRDDImpl.scala:218) finished in 0.029 s
15/06/03 18:34:24 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:24 INFO DAGScheduler: running: Set(Stage 163)
15/06/03 18:34:24 INFO DAGScheduler: waiting: Set(Stage 165, Stage 164)
15/06/03 18:34:24 INFO DAGScheduler: failed: Set()
15/06/03 18:34:24 INFO DAGScheduler: Missing parents for Stage 165: List(Stage 164)
15/06/03 18:34:24 INFO BlockManager: Found block rdd_103_0 locally
15/06/03 18:34:24 INFO DAGScheduler: Missing parents for Stage 164: List(Stage 163)
15/06/03 18:34:24 INFO Executor: Finished task 0.0 in stage 163.0 (TID 21). 2003 bytes result sent to driver
15/06/03 18:34:24 INFO TaskSetManager: Finished task 0.0 in stage 163.0 (TID 21) in 12 ms on localhost (1/1)
15/06/03 18:34:24 INFO TaskSchedulerImpl: Removed TaskSet 163.0, whose tasks have all completed, from pool
15/06/03 18:34:24 INFO DAGScheduler: Stage 163 (mapPartitions at VertexRDDImpl.scala:222) finished in 0.030 s
15/06/03 18:34:24 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:24 INFO DAGScheduler: running: Set()
15/06/03 18:34:24 INFO DAGScheduler: waiting: Set(Stage 165, Stage 164)
15/06/03 18:34:24 INFO DAGScheduler: failed: Set()
15/06/03 18:34:24 INFO DAGScheduler: Missing parents for Stage 165: List(Stage 164)
15/06/03 18:34:24 INFO DAGScheduler: Missing parents for Stage 164: List()
15/06/03 18:34:24 INFO DAGScheduler: Submitting Stage 164 (GraphImpl.aggregateMessages - preAgg MapPartitionsRDD[117] at mapPartitions at GraphImpl.scala:235), which is now runnable
15/06/03 18:34:24 INFO MemoryStore: ensureFreeSpace(7760) called with curMem=331602, maxMem=559153152
15/06/03 18:34:24 INFO MemoryStore: Block broadcast_23 stored as values in memory (estimated size 7.6 KB, free 532.9 MB)
15/06/03 18:34:24 INFO MemoryStore: ensureFreeSpace(4856) called with curMem=339362, maxMem=559153152
15/06/03 18:34:24 INFO MemoryStore: Block broadcast_23_piece0 stored as bytes in memory (estimated size 4.7 KB, free 532.9 MB)
15/06/03 18:34:24 INFO BlockManagerInfo: Added broadcast_23_piece0 in memory on localhost:60836 (size: 4.7 KB, free: 533.2 MB)
15/06/03 18:34:24 INFO BlockManagerMaster: Updated info of block broadcast_23_piece0
15/06/03 18:34:24 INFO SparkContext: Created broadcast 23 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:24 INFO DAGScheduler: Submitting 1 missing tasks from Stage 164 (GraphImpl.aggregateMessages - preAgg MapPartitionsRDD[117] at mapPartitions at GraphImpl.scala:235)
15/06/03 18:34:24 INFO TaskSchedulerImpl: Adding task set 164.0 with 1 tasks
15/06/03 18:34:24 INFO TaskSetManager: Starting task 0.0 in stage 164.0 (TID 22, localhost, PROCESS_LOCAL, 2099 bytes)
15/06/03 18:34:24 INFO Executor: Running task 0.0 in stage 164.0 (TID 22)
15/06/03 18:34:24 INFO CacheManager: Partition rdd_111_0 not found, computing it
15/06/03 18:34:24 INFO BlockManager: Found block rdd_92_0 locally
15/06/03 18:34:24 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:24 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:24 INFO MemoryStore: ensureFreeSpace(2976) called with curMem=344218, maxMem=559153152
15/06/03 18:34:24 INFO MemoryStore: Block rdd_111_0 stored as values in memory (estimated size 2.9 KB, free 532.9 MB)
15/06/03 18:34:24 INFO BlockManagerInfo: Added rdd_111_0 in memory on localhost:60836 (size: 2.9 KB, free: 533.2 MB)
15/06/03 18:34:24 INFO BlockManagerMaster: Updated info of block rdd_111_0
15/06/03 18:34:24 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:24 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:24 INFO Executor: Finished task 0.0 in stage 164.0 (TID 22). 2788 bytes result sent to driver
15/06/03 18:34:24 INFO TaskSetManager: Finished task 0.0 in stage 164.0 (TID 22) in 21 ms on localhost (1/1)
15/06/03 18:34:24 INFO TaskSchedulerImpl: Removed TaskSet 164.0, whose tasks have all completed, from pool
15/06/03 18:34:24 INFO DAGScheduler: Stage 164 (mapPartitions at GraphImpl.scala:235) finished in 0.021 s
15/06/03 18:34:24 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:24 INFO DAGScheduler: running: Set()
15/06/03 18:34:24 INFO DAGScheduler: waiting: Set(Stage 165)
15/06/03 18:34:24 INFO DAGScheduler: failed: Set()
15/06/03 18:34:24 INFO DAGScheduler: Missing parents for Stage 165: List()
15/06/03 18:34:24 INFO DAGScheduler: Submitting Stage 165 (MapPartitionsRDD[121] at map at VertexRDDImpl.scala:90), which is now runnable
15/06/03 18:34:24 INFO MemoryStore: ensureFreeSpace(8568) called with curMem=347194, maxMem=559153152
15/06/03 18:34:24 INFO MemoryStore: Block broadcast_24 stored as values in memory (estimated size 8.4 KB, free 532.9 MB)
15/06/03 18:34:24 INFO MemoryStore: ensureFreeSpace(5006) called with curMem=355762, maxMem=559153152
15/06/03 18:34:24 INFO MemoryStore: Block broadcast_24_piece0 stored as bytes in memory (estimated size 4.9 KB, free 532.9 MB)
15/06/03 18:34:24 INFO BlockManagerInfo: Added broadcast_24_piece0 in memory on localhost:60836 (size: 4.9 KB, free: 533.1 MB)
15/06/03 18:34:24 INFO BlockManagerMaster: Updated info of block broadcast_24_piece0
15/06/03 18:34:24 INFO SparkContext: Created broadcast 24 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:24 INFO DAGScheduler: Submitting 1 missing tasks from Stage 165 (MapPartitionsRDD[121] at map at VertexRDDImpl.scala:90)
15/06/03 18:34:24 INFO TaskSchedulerImpl: Adding task set 165.0 with 1 tasks
15/06/03 18:34:24 INFO TaskSetManager: Starting task 0.0 in stage 165.0 (TID 23, localhost, PROCESS_LOCAL, 1966 bytes)
15/06/03 18:34:24 INFO Executor: Running task 0.0 in stage 165.0 (TID 23)
15/06/03 18:34:24 INFO CacheManager: Partition rdd_119_0 not found, computing it
15/06/03 18:34:24 INFO BlockManager: Found block rdd_105_0 locally
15/06/03 18:34:24 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:24 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:24 INFO MemoryStore: ensureFreeSpace(1664) called with curMem=360768, maxMem=559153152
15/06/03 18:34:24 INFO MemoryStore: Block rdd_119_0 stored as values in memory (estimated size 1664.0 B, free 532.9 MB)
15/06/03 18:34:24 INFO BlockManagerInfo: Added rdd_119_0 in memory on localhost:60836 (size: 1664.0 B, free: 533.1 MB)
15/06/03 18:34:24 INFO BlockManagerMaster: Updated info of block rdd_119_0
15/06/03 18:34:24 INFO Executor: Finished task 0.0 in stage 165.0 (TID 23). 2693 bytes result sent to driver
15/06/03 18:34:24 INFO TaskSetManager: Finished task 0.0 in stage 165.0 (TID 23) in 7 ms on localhost (1/1)
15/06/03 18:34:24 INFO TaskSchedulerImpl: Removed TaskSet 165.0, whose tasks have all completed, from pool
15/06/03 18:34:24 INFO DAGScheduler: Stage 165 (reduce at VertexRDDImpl.scala:90) finished in 0.007 s
15/06/03 18:34:24 INFO DAGScheduler: Job 5 finished: reduce at VertexRDDImpl.scala:90, took 0.115658 s
15/06/03 18:34:24 INFO Pregel: Pregel finished iteration 3
15/06/03 18:34:24 INFO ZippedPartitionsRDD2: Removing RDD 100 from persistence list
15/06/03 18:34:24 INFO BlockManager: Removing RDD 100
15/06/03 18:34:24 INFO BlockManager: Removing block rdd_100_0
15/06/03 18:34:24 INFO MemoryStore: Block rdd_100_0 of size 1664 dropped from memory (free 558792384)
15/06/03 18:34:24 INFO ZippedPartitionsRDD2: Removing RDD 103 from persistence list
15/06/03 18:34:24 INFO BlockManager: Removing RDD 103
15/06/03 18:34:24 INFO BlockManager: Removing block rdd_103_0
15/06/03 18:34:24 INFO ZippedPartitionsRDD2: Removing RDD 86 from persistence list
15/06/03 18:34:24 INFO MemoryStore: Block rdd_103_0 of size 1544 dropped from memory (free 558793928)
15/06/03 18:34:24 INFO BlockManager: Removing RDD 86
15/06/03 18:34:24 INFO BlockManager: Removing block rdd_86_0
15/06/03 18:34:24 INFO MemoryStore: Block rdd_86_0 of size 1496 dropped from memory (free 558795424)
15/06/03 18:34:24 INFO ZippedPartitionsRDD2: Removing RDD 92 from persistence list
15/06/03 18:34:24 INFO BlockManager: Removing RDD 92
15/06/03 18:34:24 INFO BlockManager: Removing block rdd_92_0
15/06/03 18:34:24 INFO MemoryStore: Block rdd_92_0 of size 2976 dropped from memory (free 558798400)
15/06/03 18:34:24 INFO SparkContext: Starting job: reduce at VertexRDDImpl.scala:90
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 2 is 145 bytes
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 3 is 145 bytes
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 5 is 145 bytes
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 4 is 145 bytes
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 1 is 145 bytes
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 0 is 145 bytes
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 8 is 145 bytes
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 7 is 145 bytes
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 6 is 145 bytes
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 11 is 145 bytes
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 10 is 145 bytes
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 9 is 145 bytes
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 14 is 145 bytes
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 13 is 145 bytes
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 12 is 145 bytes
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 17 is 145 bytes
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 16 is 145 bytes
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 15 is 145 bytes
15/06/03 18:34:24 INFO DAGScheduler: Registering RDD 128 (mapPartitions at VertexRDDImpl.scala:218)
15/06/03 18:34:24 INFO DAGScheduler: Registering RDD 132 (mapPartitions at VertexRDDImpl.scala:222)
15/06/03 18:34:24 INFO DAGScheduler: Registering RDD 136 (mapPartitions at GraphImpl.scala:235)
15/06/03 18:34:24 INFO DAGScheduler: Got job 6 (reduce at VertexRDDImpl.scala:90) with 1 output partitions (allowLocal=false)
15/06/03 18:34:24 INFO DAGScheduler: Final stage: Stage 249(reduce at VertexRDDImpl.scala:90)
15/06/03 18:34:24 INFO DAGScheduler: Parents of final stage: List(Stage 248, Stage 238, Stage 235, Stage 242, Stage 246, Stage 240, Stage 244, Stage 236)
15/06/03 18:34:24 INFO DAGScheduler: Missing parents: List(Stage 248)
15/06/03 18:34:24 INFO DAGScheduler: Submitting Stage 228 (ReplicatedVertexView.updateVertices - shippedVerts true false (broadcast) MapPartitionsRDD[128] at mapPartitions at VertexRDDImpl.scala:218), which has no missing parents
15/06/03 18:34:24 INFO MemoryStore: ensureFreeSpace(8808) called with curMem=354752, maxMem=559153152
15/06/03 18:34:24 INFO MemoryStore: Block broadcast_25 stored as values in memory (estimated size 8.6 KB, free 532.9 MB)
15/06/03 18:34:24 INFO MemoryStore: ensureFreeSpace(5103) called with curMem=363560, maxMem=559153152
15/06/03 18:34:24 INFO MemoryStore: Block broadcast_25_piece0 stored as bytes in memory (estimated size 5.0 KB, free 532.9 MB)
15/06/03 18:34:24 INFO BlockManagerInfo: Added broadcast_25_piece0 in memory on localhost:60836 (size: 5.0 KB, free: 533.1 MB)
15/06/03 18:34:24 INFO BlockManagerMaster: Updated info of block broadcast_25_piece0
15/06/03 18:34:24 INFO SparkContext: Created broadcast 25 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:24 INFO DAGScheduler: Submitting 1 missing tasks from Stage 228 (ReplicatedVertexView.updateVertices - shippedVerts true false (broadcast) MapPartitionsRDD[128] at mapPartitions at VertexRDDImpl.scala:218)
15/06/03 18:34:24 INFO TaskSchedulerImpl: Adding task set 228.0 with 1 tasks
15/06/03 18:34:24 INFO TaskSetManager: Starting task 0.0 in stage 228.0 (TID 24, localhost, PROCESS_LOCAL, 2051 bytes)
15/06/03 18:34:24 INFO DAGScheduler: Submitting Stage 247 (ReplicatedVertexView.withActiveSet - shippedActives (broadcast) MapPartitionsRDD[132] at mapPartitions at VertexRDDImpl.scala:222), which has no missing parents
15/06/03 18:34:24 INFO Executor: Running task 0.0 in stage 228.0 (TID 24)
15/06/03 18:34:24 INFO MemoryStore: ensureFreeSpace(8464) called with curMem=368663, maxMem=559153152
15/06/03 18:34:24 INFO MemoryStore: Block broadcast_26 stored as values in memory (estimated size 8.3 KB, free 532.9 MB)
15/06/03 18:34:24 INFO BlockManager: Found block rdd_105_0 locally
15/06/03 18:34:24 INFO CacheManager: Partition rdd_124_0 not found, computing it
15/06/03 18:34:24 INFO BlockManager: Found block rdd_105_0 locally
15/06/03 18:34:24 INFO MemoryStore: ensureFreeSpace(4955) called with curMem=377127, maxMem=559153152
15/06/03 18:34:24 INFO CacheManager: Partition rdd_122_0 not found, computing it
15/06/03 18:34:24 INFO BlockManager: Found block rdd_105_0 locally
15/06/03 18:34:24 INFO BlockManager: Found block rdd_119_0 locally
15/06/03 18:34:24 INFO MemoryStore: Block broadcast_26_piece0 stored as bytes in memory (estimated size 4.8 KB, free 532.9 MB)
15/06/03 18:34:24 INFO BlockManagerInfo: Added broadcast_26_piece0 in memory on localhost:60836 (size: 4.8 KB, free: 533.1 MB)
15/06/03 18:34:24 INFO MemoryStore: ensureFreeSpace(1544) called with curMem=382082, maxMem=559153152
15/06/03 18:34:24 INFO BlockManagerMaster: Updated info of block broadcast_26_piece0
15/06/03 18:34:24 INFO MemoryStore: Block rdd_122_0 stored as values in memory (estimated size 1544.0 B, free 532.9 MB)
15/06/03 18:34:24 INFO SparkContext: Created broadcast 26 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:24 INFO BlockManagerInfo: Added rdd_122_0 in memory on localhost:60836 (size: 1544.0 B, free: 533.1 MB)
15/06/03 18:34:24 INFO BlockManagerMaster: Updated info of block rdd_122_0
15/06/03 18:34:24 INFO DAGScheduler: Submitting 1 missing tasks from Stage 247 (ReplicatedVertexView.withActiveSet - shippedActives (broadcast) MapPartitionsRDD[132] at mapPartitions at VertexRDDImpl.scala:222)
15/06/03 18:34:24 INFO TaskSchedulerImpl: Adding task set 247.0 with 1 tasks
15/06/03 18:34:24 INFO MemoryStore: ensureFreeSpace(1496) called with curMem=383626, maxMem=559153152
15/06/03 18:34:24 INFO MemoryStore: Block rdd_124_0 stored as values in memory (estimated size 1496.0 B, free 532.9 MB)
15/06/03 18:34:24 INFO BlockManagerInfo: Added rdd_124_0 in memory on localhost:60836 (size: 1496.0 B, free: 533.1 MB)
15/06/03 18:34:24 INFO BlockManagerMaster: Updated info of block rdd_124_0
15/06/03 18:34:24 INFO Executor: Finished task 0.0 in stage 228.0 (TID 24). 2627 bytes result sent to driver
15/06/03 18:34:24 INFO TaskSetManager: Starting task 0.0 in stage 247.0 (TID 25, localhost, PROCESS_LOCAL, 1987 bytes)
15/06/03 18:34:24 INFO TaskSetManager: Finished task 0.0 in stage 228.0 (TID 24) in 25 ms on localhost (1/1)
15/06/03 18:34:24 INFO Executor: Running task 0.0 in stage 247.0 (TID 25)
15/06/03 18:34:24 INFO TaskSchedulerImpl: Removed TaskSet 228.0, whose tasks have all completed, from pool
15/06/03 18:34:24 INFO DAGScheduler: Stage 228 (mapPartitions at VertexRDDImpl.scala:218) finished in 0.028 s
15/06/03 18:34:24 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:24 INFO DAGScheduler: running: Set(Stage 247)
15/06/03 18:34:24 INFO DAGScheduler: waiting: Set(Stage 248, Stage 249)
15/06/03 18:34:24 INFO DAGScheduler: failed: Set()
15/06/03 18:34:24 INFO BlockManager: Found block rdd_122_0 locally
15/06/03 18:34:24 INFO DAGScheduler: Missing parents for Stage 248: List(Stage 247)
15/06/03 18:34:24 INFO DAGScheduler: Missing parents for Stage 249: List(Stage 248)
15/06/03 18:34:24 INFO Executor: Finished task 0.0 in stage 247.0 (TID 25). 2003 bytes result sent to driver
15/06/03 18:34:24 INFO TaskSetManager: Finished task 0.0 in stage 247.0 (TID 25) in 12 ms on localhost (1/1)
15/06/03 18:34:24 INFO TaskSchedulerImpl: Removed TaskSet 247.0, whose tasks have all completed, from pool
15/06/03 18:34:24 INFO DAGScheduler: Stage 247 (mapPartitions at VertexRDDImpl.scala:222) finished in 0.031 s
15/06/03 18:34:24 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:24 INFO DAGScheduler: running: Set()
15/06/03 18:34:24 INFO DAGScheduler: waiting: Set(Stage 248, Stage 249)
15/06/03 18:34:24 INFO DAGScheduler: failed: Set()
15/06/03 18:34:24 INFO DAGScheduler: Missing parents for Stage 248: List()
15/06/03 18:34:24 INFO DAGScheduler: Missing parents for Stage 249: List(Stage 248)
15/06/03 18:34:24 INFO DAGScheduler: Submitting Stage 248 (GraphImpl.aggregateMessages - preAgg MapPartitionsRDD[136] at mapPartitions at GraphImpl.scala:235), which is now runnable
15/06/03 18:34:24 INFO MemoryStore: ensureFreeSpace(7992) called with curMem=385122, maxMem=559153152
15/06/03 18:34:24 INFO MemoryStore: Block broadcast_27 stored as values in memory (estimated size 7.8 KB, free 532.9 MB)
15/06/03 18:34:24 INFO MemoryStore: ensureFreeSpace(4921) called with curMem=393114, maxMem=559153152
15/06/03 18:34:24 INFO MemoryStore: Block broadcast_27_piece0 stored as bytes in memory (estimated size 4.8 KB, free 532.9 MB)
15/06/03 18:34:24 INFO BlockManagerInfo: Added broadcast_27_piece0 in memory on localhost:60836 (size: 4.8 KB, free: 533.1 MB)
15/06/03 18:34:24 INFO BlockManagerMaster: Updated info of block broadcast_27_piece0
15/06/03 18:34:24 INFO SparkContext: Created broadcast 27 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:24 INFO DAGScheduler: Submitting 1 missing tasks from Stage 248 (GraphImpl.aggregateMessages - preAgg MapPartitionsRDD[136] at mapPartitions at GraphImpl.scala:235)
15/06/03 18:34:24 INFO TaskSchedulerImpl: Adding task set 248.0 with 1 tasks
15/06/03 18:34:24 INFO TaskSetManager: Starting task 0.0 in stage 248.0 (TID 26, localhost, PROCESS_LOCAL, 2140 bytes)
15/06/03 18:34:24 INFO Executor: Running task 0.0 in stage 248.0 (TID 26)
15/06/03 18:34:24 INFO CacheManager: Partition rdd_130_0 not found, computing it
15/06/03 18:34:24 INFO BlockManager: Found block rdd_111_0 locally
15/06/03 18:34:24 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:24 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:24 INFO MemoryStore: ensureFreeSpace(2976) called with curMem=398035, maxMem=559153152
15/06/03 18:34:24 INFO MemoryStore: Block rdd_130_0 stored as values in memory (estimated size 2.9 KB, free 532.9 MB)
15/06/03 18:34:24 INFO BlockManagerInfo: Added rdd_130_0 in memory on localhost:60836 (size: 2.9 KB, free: 533.1 MB)
15/06/03 18:34:24 INFO BlockManagerMaster: Updated info of block rdd_130_0
15/06/03 18:34:24 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:24 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:24 INFO Executor: Finished task 0.0 in stage 248.0 (TID 26). 2788 bytes result sent to driver
15/06/03 18:34:24 INFO TaskSetManager: Finished task 0.0 in stage 248.0 (TID 26) in 15 ms on localhost (1/1)
15/06/03 18:34:24 INFO TaskSchedulerImpl: Removed TaskSet 248.0, whose tasks have all completed, from pool
15/06/03 18:34:24 INFO DAGScheduler: Stage 248 (mapPartitions at GraphImpl.scala:235) finished in 0.015 s
15/06/03 18:34:24 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:24 INFO DAGScheduler: running: Set()
15/06/03 18:34:24 INFO DAGScheduler: waiting: Set(Stage 249)
15/06/03 18:34:24 INFO DAGScheduler: failed: Set()
15/06/03 18:34:24 INFO DAGScheduler: Missing parents for Stage 249: List()
15/06/03 18:34:24 INFO DAGScheduler: Submitting Stage 249 (MapPartitionsRDD[140] at map at VertexRDDImpl.scala:90), which is now runnable
15/06/03 18:34:24 INFO MemoryStore: ensureFreeSpace(9096) called with curMem=401011, maxMem=559153152
15/06/03 18:34:24 INFO MemoryStore: Block broadcast_28 stored as values in memory (estimated size 8.9 KB, free 532.9 MB)
15/06/03 18:34:24 INFO MemoryStore: ensureFreeSpace(5212) called with curMem=410107, maxMem=559153152
15/06/03 18:34:24 INFO MemoryStore: Block broadcast_28_piece0 stored as bytes in memory (estimated size 5.1 KB, free 532.9 MB)
15/06/03 18:34:24 INFO BlockManagerInfo: Added broadcast_28_piece0 in memory on localhost:60836 (size: 5.1 KB, free: 533.1 MB)
15/06/03 18:34:24 INFO BlockManagerMaster: Updated info of block broadcast_28_piece0
15/06/03 18:34:24 INFO SparkContext: Created broadcast 28 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:24 INFO DAGScheduler: Submitting 1 missing tasks from Stage 249 (MapPartitionsRDD[140] at map at VertexRDDImpl.scala:90)
15/06/03 18:34:24 INFO TaskSchedulerImpl: Adding task set 249.0 with 1 tasks
15/06/03 18:34:24 INFO TaskSetManager: Starting task 0.0 in stage 249.0 (TID 27, localhost, PROCESS_LOCAL, 2071 bytes)
15/06/03 18:34:24 INFO Executor: Running task 0.0 in stage 249.0 (TID 27)
15/06/03 18:34:24 INFO CacheManager: Partition rdd_138_0 not found, computing it
15/06/03 18:34:24 INFO BlockManager: Found block rdd_124_0 locally
15/06/03 18:34:24 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:24 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
15/06/03 18:34:24 INFO MemoryStore: ensureFreeSpace(1664) called with curMem=415319, maxMem=559153152
15/06/03 18:34:24 INFO MemoryStore: Block rdd_138_0 stored as values in memory (estimated size 1664.0 B, free 532.9 MB)
15/06/03 18:34:24 INFO BlockManagerInfo: Added rdd_138_0 in memory on localhost:60836 (size: 1664.0 B, free: 533.1 MB)
15/06/03 18:34:24 INFO BlockManagerMaster: Updated info of block rdd_138_0
15/06/03 18:34:24 INFO Executor: Finished task 0.0 in stage 249.0 (TID 27). 2693 bytes result sent to driver
15/06/03 18:34:24 INFO TaskSetManager: Finished task 0.0 in stage 249.0 (TID 27) in 9 ms on localhost (1/1)
15/06/03 18:34:24 INFO TaskSchedulerImpl: Removed TaskSet 249.0, whose tasks have all completed, from pool
15/06/03 18:34:24 INFO DAGScheduler: Stage 249 (reduce at VertexRDDImpl.scala:90) finished in 0.009 s
15/06/03 18:34:24 INFO DAGScheduler: Job 6 finished: reduce at VertexRDDImpl.scala:90, took 0.115397 s
15/06/03 18:34:24 INFO Pregel: Pregel finished iteration 4
15/06/03 18:34:24 INFO ZippedPartitionsRDD2: Removing RDD 119 from persistence list
15/06/03 18:34:24 INFO BlockManager: Removing RDD 119
15/06/03 18:34:24 INFO ZippedPartitionsRDD2: Removing RDD 122 from persistence list
15/06/03 18:34:24 INFO BlockManager: Removing block rdd_119_0
15/06/03 18:34:24 INFO MemoryStore: Block rdd_119_0 of size 1664 dropped from memory (free 558737833)
15/06/03 18:34:24 INFO BlockManager: Removing RDD 122
15/06/03 18:34:24 INFO ZippedPartitionsRDD2: Removing RDD 105 from persistence list
15/06/03 18:34:24 INFO BlockManager: Removing block rdd_122_0
15/06/03 18:34:24 INFO MemoryStore: Block rdd_122_0 of size 1544 dropped from memory (free 558739377)
15/06/03 18:34:24 INFO BlockManager: Removing RDD 105
15/06/03 18:34:24 INFO BlockManager: Removing block rdd_105_0
15/06/03 18:34:24 INFO MemoryStore: Block rdd_105_0 of size 1496 dropped from memory (free 558740873)
15/06/03 18:34:24 INFO ZippedPartitionsRDD2: Removing RDD 111 from persistence list
15/06/03 18:34:24 INFO BlockManager: Removing RDD 111
15/06/03 18:34:24 INFO BlockManager: Removing block rdd_111_0
15/06/03 18:34:24 INFO MemoryStore: Block rdd_111_0 of size 2976 dropped from memory (free 558743849)
15/06/03 18:34:24 INFO SparkContext: Starting job: reduce at VertexRDDImpl.scala:90
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 2 is 145 bytes
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 3 is 145 bytes
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 5 is 145 bytes
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 4 is 145 bytes
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 1 is 145 bytes
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 0 is 145 bytes
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 8 is 145 bytes
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 7 is 145 bytes
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 6 is 145 bytes
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 11 is 145 bytes
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 10 is 145 bytes
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 9 is 145 bytes
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 14 is 145 bytes
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 13 is 145 bytes
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 12 is 145 bytes
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 17 is 145 bytes
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 16 is 145 bytes
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 15 is 145 bytes
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 20 is 145 bytes
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 19 is 145 bytes
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 18 is 145 bytes
15/06/03 18:34:24 INFO DAGScheduler: Registering RDD 147 (mapPartitions at VertexRDDImpl.scala:218)
15/06/03 18:34:24 INFO DAGScheduler: Registering RDD 151 (mapPartitions at VertexRDDImpl.scala:222)
15/06/03 18:34:24 INFO DAGScheduler: Registering RDD 155 (mapPartitions at GraphImpl.scala:235)
15/06/03 18:34:24 INFO DAGScheduler: Got job 7 (reduce at VertexRDDImpl.scala:90) with 1 output partitions (allowLocal=false)
15/06/03 18:34:24 INFO DAGScheduler: Final stage: Stage 357(reduce at VertexRDDImpl.scala:90)
15/06/03 18:34:24 INFO DAGScheduler: Parents of final stage: List(Stage 356, Stage 350, Stage 342, Stage 354, Stage 344, Stage 341, Stage 348, Stage 352, Stage 346)
15/06/03 18:34:24 INFO DAGScheduler: Missing parents: List(Stage 356)
15/06/03 18:34:24 INFO DAGScheduler: Submitting Stage 333 (ReplicatedVertexView.updateVertices - shippedVerts true false (broadcast) MapPartitionsRDD[147] at mapPartitions at VertexRDDImpl.scala:218), which has no missing parents
15/06/03 18:34:24 INFO MemoryStore: ensureFreeSpace(9336) called with curMem=409303, maxMem=559153152
15/06/03 18:34:24 INFO MemoryStore: Block broadcast_29 stored as values in memory (estimated size 9.1 KB, free 532.9 MB)
15/06/03 18:34:24 INFO MemoryStore: ensureFreeSpace(5298) called with curMem=418639, maxMem=559153152
15/06/03 18:34:24 INFO MemoryStore: Block broadcast_29_piece0 stored as bytes in memory (estimated size 5.2 KB, free 532.8 MB)
15/06/03 18:34:24 INFO BlockManagerInfo: Added broadcast_29_piece0 in memory on localhost:60836 (size: 5.2 KB, free: 533.1 MB)
15/06/03 18:34:24 INFO BlockManagerMaster: Updated info of block broadcast_29_piece0
15/06/03 18:34:24 INFO SparkContext: Created broadcast 29 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:24 INFO DAGScheduler: Submitting 1 missing tasks from Stage 333 (ReplicatedVertexView.updateVertices - shippedVerts true false (broadcast) MapPartitionsRDD[147] at mapPartitions at VertexRDDImpl.scala:218)
15/06/03 18:34:24 INFO TaskSchedulerImpl: Adding task set 333.0 with 1 tasks
15/06/03 18:34:24 INFO DAGScheduler: Submitting Stage 355 (ReplicatedVertexView.withActiveSet - shippedActives (broadcast) MapPartitionsRDD[151] at mapPartitions at VertexRDDImpl.scala:222), which has no missing parents
15/06/03 18:34:24 INFO TaskSetManager: Starting task 0.0 in stage 333.0 (TID 28, localhost, PROCESS_LOCAL, 2156 bytes)
15/06/03 18:34:24 INFO Executor: Running task 0.0 in stage 333.0 (TID 28)
15/06/03 18:34:24 INFO MemoryStore: ensureFreeSpace(8992) called with curMem=423937, maxMem=559153152
15/06/03 18:34:24 INFO MemoryStore: Block broadcast_30 stored as values in memory (estimated size 8.8 KB, free 532.8 MB)
15/06/03 18:34:24 INFO MemoryStore: ensureFreeSpace(5165) called with curMem=432929, maxMem=559153152
15/06/03 18:34:24 INFO MemoryStore: Block broadcast_30_piece0 stored as bytes in memory (estimated size 5.0 KB, free 532.8 MB)
15/06/03 18:34:24 INFO BlockManagerInfo: Added broadcast_30_piece0 in memory on localhost:60836 (size: 5.0 KB, free: 533.1 MB)
15/06/03 18:34:24 INFO BlockManagerMaster: Updated info of block broadcast_30_piece0
15/06/03 18:34:24 INFO SparkContext: Created broadcast 30 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:24 INFO DAGScheduler: Submitting 1 missing tasks from Stage 355 (ReplicatedVertexView.withActiveSet - shippedActives (broadcast) MapPartitionsRDD[151] at mapPartitions at VertexRDDImpl.scala:222)
15/06/03 18:34:24 INFO TaskSchedulerImpl: Adding task set 355.0 with 1 tasks
15/06/03 18:34:24 INFO BlockManager: Found block rdd_124_0 locally
15/06/03 18:34:24 INFO CacheManager: Partition rdd_143_0 not found, computing it
15/06/03 18:34:24 INFO BlockManager: Found block rdd_124_0 locally
15/06/03 18:34:24 INFO CacheManager: Partition rdd_141_0 not found, computing it
15/06/03 18:34:24 INFO BlockManager: Found block rdd_124_0 locally
15/06/03 18:34:24 INFO BlockManager: Found block rdd_138_0 locally
15/06/03 18:34:24 INFO MemoryStore: ensureFreeSpace(1544) called with curMem=438094, maxMem=559153152
15/06/03 18:34:24 INFO MemoryStore: Block rdd_141_0 stored as values in memory (estimated size 1544.0 B, free 532.8 MB)
15/06/03 18:34:24 INFO BlockManagerInfo: Added rdd_141_0 in memory on localhost:60836 (size: 1544.0 B, free: 533.1 MB)
15/06/03 18:34:24 INFO BlockManagerMaster: Updated info of block rdd_141_0
15/06/03 18:34:24 INFO MemoryStore: ensureFreeSpace(1496) called with curMem=439638, maxMem=559153152
15/06/03 18:34:24 INFO MemoryStore: Block rdd_143_0 stored as values in memory (estimated size 1496.0 B, free 532.8 MB)
15/06/03 18:34:24 INFO BlockManagerInfo: Added rdd_143_0 in memory on localhost:60836 (size: 1496.0 B, free: 533.1 MB)
15/06/03 18:34:24 INFO BlockManagerMaster: Updated info of block rdd_143_0
15/06/03 18:34:24 INFO Executor: Finished task 0.0 in stage 333.0 (TID 28). 2627 bytes result sent to driver
15/06/03 18:34:24 INFO TaskSetManager: Starting task 0.0 in stage 355.0 (TID 29, localhost, PROCESS_LOCAL, 2092 bytes)
15/06/03 18:34:24 INFO Executor: Running task 0.0 in stage 355.0 (TID 29)
15/06/03 18:34:24 INFO TaskSetManager: Finished task 0.0 in stage 333.0 (TID 28) in 28 ms on localhost (1/1)
15/06/03 18:34:24 INFO TaskSchedulerImpl: Removed TaskSet 333.0, whose tasks have all completed, from pool
15/06/03 18:34:24 INFO DAGScheduler: Stage 333 (mapPartitions at VertexRDDImpl.scala:218) finished in 0.029 s
15/06/03 18:34:24 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:24 INFO DAGScheduler: running: Set(Stage 355)
15/06/03 18:34:24 INFO DAGScheduler: waiting: Set(Stage 356, Stage 357)
15/06/03 18:34:24 INFO DAGScheduler: failed: Set()
15/06/03 18:34:24 INFO DAGScheduler: Missing parents for Stage 356: List(Stage 355)
15/06/03 18:34:24 INFO BlockManager: Found block rdd_141_0 locally
15/06/03 18:34:24 INFO DAGScheduler: Missing parents for Stage 357: List(Stage 356)
15/06/03 18:34:24 INFO Executor: Finished task 0.0 in stage 355.0 (TID 29). 2003 bytes result sent to driver
15/06/03 18:34:24 INFO TaskSetManager: Finished task 0.0 in stage 355.0 (TID 29) in 26 ms on localhost (1/1)
15/06/03 18:34:24 INFO TaskSchedulerImpl: Removed TaskSet 355.0, whose tasks have all completed, from pool
15/06/03 18:34:24 INFO DAGScheduler: Stage 355 (mapPartitions at VertexRDDImpl.scala:222) finished in 0.048 s
15/06/03 18:34:24 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:24 INFO DAGScheduler: running: Set()
15/06/03 18:34:24 INFO DAGScheduler: waiting: Set(Stage 356, Stage 357)
15/06/03 18:34:24 INFO DAGScheduler: failed: Set()
15/06/03 18:34:24 INFO DAGScheduler: Missing parents for Stage 356: List()
15/06/03 18:34:24 INFO DAGScheduler: Missing parents for Stage 357: List(Stage 356)
15/06/03 18:34:24 INFO DAGScheduler: Submitting Stage 356 (GraphImpl.aggregateMessages - preAgg MapPartitionsRDD[155] at mapPartitions at GraphImpl.scala:235), which is now runnable
15/06/03 18:34:24 INFO MemoryStore: ensureFreeSpace(8216) called with curMem=441134, maxMem=559153152
15/06/03 18:34:24 INFO MemoryStore: Block broadcast_31 stored as values in memory (estimated size 8.0 KB, free 532.8 MB)
15/06/03 18:34:24 INFO MemoryStore: ensureFreeSpace(4954) called with curMem=449350, maxMem=559153152
15/06/03 18:34:24 INFO MemoryStore: Block broadcast_31_piece0 stored as bytes in memory (estimated size 4.8 KB, free 532.8 MB)
15/06/03 18:34:24 INFO BlockManagerInfo: Added broadcast_31_piece0 in memory on localhost:60836 (size: 4.8 KB, free: 533.1 MB)
15/06/03 18:34:24 INFO BlockManagerMaster: Updated info of block broadcast_31_piece0
15/06/03 18:34:24 INFO SparkContext: Created broadcast 31 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:24 INFO DAGScheduler: Submitting 1 missing tasks from Stage 356 (GraphImpl.aggregateMessages - preAgg MapPartitionsRDD[155] at mapPartitions at GraphImpl.scala:235)
15/06/03 18:34:24 INFO TaskSchedulerImpl: Adding task set 356.0 with 1 tasks
15/06/03 18:34:24 INFO TaskSetManager: Starting task 0.0 in stage 356.0 (TID 30, localhost, PROCESS_LOCAL, 2181 bytes)
15/06/03 18:34:24 INFO Executor: Running task 0.0 in stage 356.0 (TID 30)
15/06/03 18:34:24 INFO CacheManager: Partition rdd_149_0 not found, computing it
15/06/03 18:34:24 INFO BlockManager: Found block rdd_130_0 locally
15/06/03 18:34:24 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:24 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:24 INFO MemoryStore: ensureFreeSpace(2976) called with curMem=454304, maxMem=559153152
15/06/03 18:34:24 INFO MemoryStore: Block rdd_149_0 stored as values in memory (estimated size 2.9 KB, free 532.8 MB)
15/06/03 18:34:24 INFO BlockManagerInfo: Added rdd_149_0 in memory on localhost:60836 (size: 2.9 KB, free: 533.1 MB)
15/06/03 18:34:24 INFO BlockManagerMaster: Updated info of block rdd_149_0
15/06/03 18:34:24 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:24 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:24 INFO Executor: Finished task 0.0 in stage 356.0 (TID 30). 2788 bytes result sent to driver
15/06/03 18:34:24 INFO TaskSetManager: Finished task 0.0 in stage 356.0 (TID 30) in 25 ms on localhost (1/1)
15/06/03 18:34:24 INFO TaskSchedulerImpl: Removed TaskSet 356.0, whose tasks have all completed, from pool
15/06/03 18:34:24 INFO DAGScheduler: Stage 356 (mapPartitions at GraphImpl.scala:235) finished in 0.027 s
15/06/03 18:34:24 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:24 INFO DAGScheduler: running: Set()
15/06/03 18:34:24 INFO DAGScheduler: waiting: Set(Stage 357)
15/06/03 18:34:24 INFO DAGScheduler: failed: Set()
15/06/03 18:34:24 INFO DAGScheduler: Missing parents for Stage 357: List()
15/06/03 18:34:24 INFO DAGScheduler: Submitting Stage 357 (MapPartitionsRDD[159] at map at VertexRDDImpl.scala:90), which is now runnable
15/06/03 18:34:24 INFO MemoryStore: ensureFreeSpace(9624) called with curMem=457280, maxMem=559153152
15/06/03 18:34:24 INFO MemoryStore: Block broadcast_32 stored as values in memory (estimated size 9.4 KB, free 532.8 MB)
15/06/03 18:34:24 INFO MemoryStore: ensureFreeSpace(5377) called with curMem=466904, maxMem=559153152
15/06/03 18:34:24 INFO MemoryStore: Block broadcast_32_piece0 stored as bytes in memory (estimated size 5.3 KB, free 532.8 MB)
15/06/03 18:34:24 INFO BlockManagerInfo: Added broadcast_32_piece0 in memory on localhost:60836 (size: 5.3 KB, free: 533.1 MB)
15/06/03 18:34:24 INFO BlockManagerMaster: Updated info of block broadcast_32_piece0
15/06/03 18:34:24 INFO SparkContext: Created broadcast 32 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:24 INFO DAGScheduler: Submitting 1 missing tasks from Stage 357 (MapPartitionsRDD[159] at map at VertexRDDImpl.scala:90)
15/06/03 18:34:24 INFO TaskSchedulerImpl: Adding task set 357.0 with 1 tasks
15/06/03 18:34:24 INFO TaskSetManager: Starting task 0.0 in stage 357.0 (TID 31, localhost, PROCESS_LOCAL, 2176 bytes)
15/06/03 18:34:24 INFO Executor: Running task 0.0 in stage 357.0 (TID 31)
15/06/03 18:34:24 INFO CacheManager: Partition rdd_157_0 not found, computing it
15/06/03 18:34:24 INFO BlockManager: Found block rdd_143_0 locally
15/06/03 18:34:24 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:24 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:24 INFO MemoryStore: ensureFreeSpace(1664) called with curMem=472281, maxMem=559153152
15/06/03 18:34:24 INFO MemoryStore: Block rdd_157_0 stored as values in memory (estimated size 1664.0 B, free 532.8 MB)
15/06/03 18:34:24 INFO BlockManagerInfo: Added rdd_157_0 in memory on localhost:60836 (size: 1664.0 B, free: 533.1 MB)
15/06/03 18:34:24 INFO BlockManagerMaster: Updated info of block rdd_157_0
15/06/03 18:34:24 INFO Executor: Finished task 0.0 in stage 357.0 (TID 31). 2693 bytes result sent to driver
15/06/03 18:34:24 INFO TaskSetManager: Finished task 0.0 in stage 357.0 (TID 31) in 7 ms on localhost (1/1)
15/06/03 18:34:24 INFO TaskSchedulerImpl: Removed TaskSet 357.0, whose tasks have all completed, from pool
15/06/03 18:34:24 INFO DAGScheduler: Stage 357 (reduce at VertexRDDImpl.scala:90) finished in 0.007 s
15/06/03 18:34:24 INFO DAGScheduler: Job 7 finished: reduce at VertexRDDImpl.scala:90, took 0.156334 s
15/06/03 18:34:24 INFO Pregel: Pregel finished iteration 5
15/06/03 18:34:24 INFO ZippedPartitionsRDD2: Removing RDD 138 from persistence list
15/06/03 18:34:24 INFO BlockManager: Removing RDD 138
15/06/03 18:34:24 INFO ZippedPartitionsRDD2: Removing RDD 141 from persistence list
15/06/03 18:34:24 INFO BlockManager: Removing block rdd_138_0
15/06/03 18:34:24 INFO MemoryStore: Block rdd_138_0 of size 1664 dropped from memory (free 558680871)
15/06/03 18:34:24 INFO BlockManager: Removing RDD 141
15/06/03 18:34:24 INFO ZippedPartitionsRDD2: Removing RDD 124 from persistence list
15/06/03 18:34:24 INFO BlockManager: Removing block rdd_141_0
15/06/03 18:34:24 INFO MemoryStore: Block rdd_141_0 of size 1544 dropped from memory (free 558682415)
15/06/03 18:34:24 INFO BlockManager: Removing RDD 124
15/06/03 18:34:24 INFO ZippedPartitionsRDD2: Removing RDD 130 from persistence list
15/06/03 18:34:24 INFO BlockManager: Removing block rdd_124_0
15/06/03 18:34:24 INFO MemoryStore: Block rdd_124_0 of size 1496 dropped from memory (free 558683911)
15/06/03 18:34:24 INFO BlockManager: Removing RDD 130
15/06/03 18:34:24 INFO BlockManager: Removing block rdd_130_0
15/06/03 18:34:24 INFO MemoryStore: Block rdd_130_0 of size 2976 dropped from memory (free 558686887)
15/06/03 18:34:24 INFO SparkContext: Starting job: reduce at VertexRDDImpl.scala:90
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 2 is 145 bytes
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 3 is 145 bytes
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 5 is 145 bytes
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 4 is 145 bytes
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 1 is 145 bytes
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 0 is 145 bytes
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 8 is 145 bytes
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 7 is 145 bytes
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 6 is 145 bytes
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 11 is 145 bytes
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 10 is 145 bytes
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 9 is 145 bytes
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 14 is 145 bytes
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 13 is 145 bytes
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 12 is 145 bytes
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 17 is 145 bytes
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 16 is 145 bytes
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 15 is 145 bytes
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 20 is 145 bytes
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 19 is 145 bytes
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 18 is 145 bytes
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 23 is 145 bytes
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 22 is 145 bytes
15/06/03 18:34:24 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 21 is 145 bytes
15/06/03 18:34:24 INFO DAGScheduler: Registering RDD 166 (mapPartitions at VertexRDDImpl.scala:218)
15/06/03 18:34:24 INFO DAGScheduler: Registering RDD 170 (mapPartitions at VertexRDDImpl.scala:222)
15/06/03 18:34:24 INFO DAGScheduler: Registering RDD 174 (mapPartitions at GraphImpl.scala:235)
15/06/03 18:34:24 INFO DAGScheduler: Got job 8 (reduce at VertexRDDImpl.scala:90) with 1 output partitions (allowLocal=false)
15/06/03 18:34:24 INFO DAGScheduler: Final stage: Stage 492(reduce at VertexRDDImpl.scala:90)
15/06/03 18:34:24 INFO DAGScheduler: Parents of final stage: List(Stage 491, Stage 477, Stage 474, Stage 489, Stage 481, Stage 475, Stage 485, Stage 479, Stage 483, Stage 487)
15/06/03 18:34:24 INFO DAGScheduler: Missing parents: List(Stage 491)
15/06/03 18:34:24 INFO DAGScheduler: Submitting Stage 465 (ReplicatedVertexView.updateVertices - shippedVerts true false (broadcast) MapPartitionsRDD[166] at mapPartitions at VertexRDDImpl.scala:218), which has no missing parents
15/06/03 18:34:24 INFO MemoryStore: ensureFreeSpace(9864) called with curMem=466265, maxMem=559153152
15/06/03 18:34:24 INFO MemoryStore: Block broadcast_33 stored as values in memory (estimated size 9.6 KB, free 532.8 MB)
15/06/03 18:34:24 INFO MemoryStore: ensureFreeSpace(5498) called with curMem=476129, maxMem=559153152
15/06/03 18:34:24 INFO MemoryStore: Block broadcast_33_piece0 stored as bytes in memory (estimated size 5.4 KB, free 532.8 MB)
15/06/03 18:34:24 INFO BlockManagerInfo: Added broadcast_33_piece0 in memory on localhost:60836 (size: 5.4 KB, free: 533.1 MB)
15/06/03 18:34:24 INFO BlockManagerMaster: Updated info of block broadcast_33_piece0
15/06/03 18:34:24 INFO SparkContext: Created broadcast 33 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:24 INFO DAGScheduler: Submitting 1 missing tasks from Stage 465 (ReplicatedVertexView.updateVertices - shippedVerts true false (broadcast) MapPartitionsRDD[166] at mapPartitions at VertexRDDImpl.scala:218)
15/06/03 18:34:24 INFO TaskSchedulerImpl: Adding task set 465.0 with 1 tasks
15/06/03 18:34:24 INFO DAGScheduler: Submitting Stage 490 (ReplicatedVertexView.withActiveSet - shippedActives (broadcast) MapPartitionsRDD[170] at mapPartitions at VertexRDDImpl.scala:222), which has no missing parents
15/06/03 18:34:24 INFO TaskSetManager: Starting task 0.0 in stage 465.0 (TID 32, localhost, PROCESS_LOCAL, 2261 bytes)
15/06/03 18:34:24 INFO Executor: Running task 0.0 in stage 465.0 (TID 32)
15/06/03 18:34:24 INFO MemoryStore: ensureFreeSpace(9520) called with curMem=481627, maxMem=559153152
15/06/03 18:34:24 INFO MemoryStore: Block broadcast_34 stored as values in memory (estimated size 9.3 KB, free 532.8 MB)
15/06/03 18:34:24 INFO MemoryStore: ensureFreeSpace(5357) called with curMem=491147, maxMem=559153152
15/06/03 18:34:24 INFO BlockManager: Found block rdd_143_0 locally
15/06/03 18:34:24 INFO MemoryStore: Block broadcast_34_piece0 stored as bytes in memory (estimated size 5.2 KB, free 532.8 MB)
15/06/03 18:34:24 INFO CacheManager: Partition rdd_162_0 not found, computing it
15/06/03 18:34:24 INFO BlockManager: Found block rdd_143_0 locally
15/06/03 18:34:24 INFO BlockManagerInfo: Added broadcast_34_piece0 in memory on localhost:60836 (size: 5.2 KB, free: 533.1 MB)
15/06/03 18:34:24 INFO BlockManagerMaster: Updated info of block broadcast_34_piece0
15/06/03 18:34:24 INFO CacheManager: Partition rdd_160_0 not found, computing it
15/06/03 18:34:24 INFO BlockManager: Found block rdd_143_0 locally
15/06/03 18:34:24 INFO BlockManager: Found block rdd_157_0 locally
15/06/03 18:34:24 INFO SparkContext: Created broadcast 34 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:24 INFO DAGScheduler: Submitting 1 missing tasks from Stage 490 (ReplicatedVertexView.withActiveSet - shippedActives (broadcast) MapPartitionsRDD[170] at mapPartitions at VertexRDDImpl.scala:222)
15/06/03 18:34:24 INFO TaskSchedulerImpl: Adding task set 490.0 with 1 tasks
15/06/03 18:34:24 INFO MemoryStore: ensureFreeSpace(1544) called with curMem=496504, maxMem=559153152
15/06/03 18:34:24 INFO MemoryStore: Block rdd_160_0 stored as values in memory (estimated size 1544.0 B, free 532.8 MB)
15/06/03 18:34:24 INFO BlockManagerInfo: Added rdd_160_0 in memory on localhost:60836 (size: 1544.0 B, free: 533.1 MB)
15/06/03 18:34:24 INFO BlockManagerMaster: Updated info of block rdd_160_0
15/06/03 18:34:24 INFO MemoryStore: ensureFreeSpace(1496) called with curMem=498048, maxMem=559153152
15/06/03 18:34:24 INFO MemoryStore: Block rdd_162_0 stored as values in memory (estimated size 1496.0 B, free 532.8 MB)
15/06/03 18:34:24 INFO BlockManagerInfo: Added rdd_162_0 in memory on localhost:60836 (size: 1496.0 B, free: 533.1 MB)
15/06/03 18:34:24 INFO BlockManagerMaster: Updated info of block rdd_162_0
15/06/03 18:34:24 INFO Executor: Finished task 0.0 in stage 465.0 (TID 32). 2627 bytes result sent to driver
15/06/03 18:34:24 INFO TaskSetManager: Starting task 0.0 in stage 490.0 (TID 33, localhost, PROCESS_LOCAL, 2197 bytes)
15/06/03 18:34:24 INFO Executor: Running task 0.0 in stage 490.0 (TID 33)
15/06/03 18:34:24 INFO TaskSetManager: Finished task 0.0 in stage 465.0 (TID 32) in 25 ms on localhost (1/1)
15/06/03 18:34:24 INFO TaskSchedulerImpl: Removed TaskSet 465.0, whose tasks have all completed, from pool
15/06/03 18:34:24 INFO DAGScheduler: Stage 465 (mapPartitions at VertexRDDImpl.scala:218) finished in 0.027 s
15/06/03 18:34:24 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:24 INFO DAGScheduler: running: Set(Stage 490)
15/06/03 18:34:24 INFO DAGScheduler: waiting: Set(Stage 491, Stage 492)
15/06/03 18:34:24 INFO DAGScheduler: failed: Set()
15/06/03 18:34:24 INFO DAGScheduler: Missing parents for Stage 491: List(Stage 490)
15/06/03 18:34:24 INFO DAGScheduler: Missing parents for Stage 492: List(Stage 491)
15/06/03 18:34:24 INFO BlockManager: Found block rdd_160_0 locally
15/06/03 18:34:24 INFO Executor: Finished task 0.0 in stage 490.0 (TID 33). 2003 bytes result sent to driver
15/06/03 18:34:24 INFO TaskSetManager: Finished task 0.0 in stage 490.0 (TID 33) in 15 ms on localhost (1/1)
15/06/03 18:34:24 INFO TaskSchedulerImpl: Removed TaskSet 490.0, whose tasks have all completed, from pool
15/06/03 18:34:24 INFO DAGScheduler: Stage 490 (mapPartitions at VertexRDDImpl.scala:222) finished in 0.032 s
15/06/03 18:34:24 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:24 INFO DAGScheduler: running: Set()
15/06/03 18:34:24 INFO DAGScheduler: waiting: Set(Stage 491, Stage 492)
15/06/03 18:34:24 INFO DAGScheduler: failed: Set()
15/06/03 18:34:24 INFO DAGScheduler: Missing parents for Stage 491: List()
15/06/03 18:34:24 INFO DAGScheduler: Missing parents for Stage 492: List(Stage 491)
15/06/03 18:34:24 INFO DAGScheduler: Submitting Stage 491 (GraphImpl.aggregateMessages - preAgg MapPartitionsRDD[174] at mapPartitions at GraphImpl.scala:235), which is now runnable
15/06/03 18:34:24 INFO MemoryStore: ensureFreeSpace(8448) called with curMem=499544, maxMem=559153152
15/06/03 18:34:24 INFO MemoryStore: Block broadcast_35 stored as values in memory (estimated size 8.3 KB, free 532.8 MB)
15/06/03 18:34:24 INFO MemoryStore: ensureFreeSpace(5095) called with curMem=507992, maxMem=559153152
15/06/03 18:34:24 INFO MemoryStore: Block broadcast_35_piece0 stored as bytes in memory (estimated size 5.0 KB, free 532.8 MB)
15/06/03 18:34:24 INFO BlockManagerInfo: Added broadcast_35_piece0 in memory on localhost:60836 (size: 5.0 KB, free: 533.1 MB)
15/06/03 18:34:24 INFO BlockManagerMaster: Updated info of block broadcast_35_piece0
15/06/03 18:34:24 INFO SparkContext: Created broadcast 35 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:24 INFO DAGScheduler: Submitting 1 missing tasks from Stage 491 (GraphImpl.aggregateMessages - preAgg MapPartitionsRDD[174] at mapPartitions at GraphImpl.scala:235)
15/06/03 18:34:24 INFO TaskSchedulerImpl: Adding task set 491.0 with 1 tasks
15/06/03 18:34:24 INFO TaskSetManager: Starting task 0.0 in stage 491.0 (TID 34, localhost, PROCESS_LOCAL, 2222 bytes)
15/06/03 18:34:24 INFO Executor: Running task 0.0 in stage 491.0 (TID 34)
15/06/03 18:34:24 INFO CacheManager: Partition rdd_168_0 not found, computing it
15/06/03 18:34:24 INFO BlockManager: Found block rdd_149_0 locally
15/06/03 18:34:24 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:24 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:24 INFO MemoryStore: ensureFreeSpace(2976) called with curMem=513087, maxMem=559153152
15/06/03 18:34:24 INFO MemoryStore: Block rdd_168_0 stored as values in memory (estimated size 2.9 KB, free 532.8 MB)
15/06/03 18:34:24 INFO BlockManagerInfo: Added rdd_168_0 in memory on localhost:60836 (size: 2.9 KB, free: 533.1 MB)
15/06/03 18:34:24 INFO BlockManagerMaster: Updated info of block rdd_168_0
15/06/03 18:34:24 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:24 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:25 INFO Executor: Finished task 0.0 in stage 491.0 (TID 34). 2788 bytes result sent to driver
15/06/03 18:34:25 INFO TaskSetManager: Finished task 0.0 in stage 491.0 (TID 34) in 15 ms on localhost (1/1)
15/06/03 18:34:25 INFO TaskSchedulerImpl: Removed TaskSet 491.0, whose tasks have all completed, from pool
15/06/03 18:34:25 INFO DAGScheduler: Stage 491 (mapPartitions at GraphImpl.scala:235) finished in 0.015 s
15/06/03 18:34:25 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:25 INFO DAGScheduler: running: Set()
15/06/03 18:34:25 INFO DAGScheduler: waiting: Set(Stage 492)
15/06/03 18:34:25 INFO DAGScheduler: failed: Set()
15/06/03 18:34:25 INFO DAGScheduler: Missing parents for Stage 492: List()
15/06/03 18:34:25 INFO DAGScheduler: Submitting Stage 492 (MapPartitionsRDD[178] at map at VertexRDDImpl.scala:90), which is now runnable
15/06/03 18:34:25 INFO MemoryStore: ensureFreeSpace(10152) called with curMem=516063, maxMem=559153152
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_36 stored as values in memory (estimated size 9.9 KB, free 532.7 MB)
15/06/03 18:34:25 INFO MemoryStore: ensureFreeSpace(5614) called with curMem=526215, maxMem=559153152
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_36_piece0 stored as bytes in memory (estimated size 5.5 KB, free 532.7 MB)
15/06/03 18:34:25 INFO BlockManagerInfo: Added broadcast_36_piece0 in memory on localhost:60836 (size: 5.5 KB, free: 533.1 MB)
15/06/03 18:34:25 INFO BlockManagerMaster: Updated info of block broadcast_36_piece0
15/06/03 18:34:25 INFO SparkContext: Created broadcast 36 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:25 INFO DAGScheduler: Submitting 1 missing tasks from Stage 492 (MapPartitionsRDD[178] at map at VertexRDDImpl.scala:90)
15/06/03 18:34:25 INFO TaskSchedulerImpl: Adding task set 492.0 with 1 tasks
15/06/03 18:34:25 INFO TaskSetManager: Starting task 0.0 in stage 492.0 (TID 35, localhost, PROCESS_LOCAL, 2281 bytes)
15/06/03 18:34:25 INFO Executor: Running task 0.0 in stage 492.0 (TID 35)
15/06/03 18:34:25 INFO CacheManager: Partition rdd_176_0 not found, computing it
15/06/03 18:34:25 INFO BlockManager: Found block rdd_162_0 locally
15/06/03 18:34:25 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:25 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:25 INFO MemoryStore: ensureFreeSpace(1664) called with curMem=531829, maxMem=559153152
15/06/03 18:34:25 INFO MemoryStore: Block rdd_176_0 stored as values in memory (estimated size 1664.0 B, free 532.7 MB)
15/06/03 18:34:25 INFO BlockManagerInfo: Added rdd_176_0 in memory on localhost:60836 (size: 1664.0 B, free: 533.1 MB)
15/06/03 18:34:25 INFO BlockManagerMaster: Updated info of block rdd_176_0
15/06/03 18:34:25 INFO Executor: Finished task 0.0 in stage 492.0 (TID 35). 2693 bytes result sent to driver
15/06/03 18:34:25 INFO TaskSetManager: Finished task 0.0 in stage 492.0 (TID 35) in 7 ms on localhost (1/1)
15/06/03 18:34:25 INFO TaskSchedulerImpl: Removed TaskSet 492.0, whose tasks have all completed, from pool
15/06/03 18:34:25 INFO DAGScheduler: Stage 492 (reduce at VertexRDDImpl.scala:90) finished in 0.008 s
15/06/03 18:34:25 INFO DAGScheduler: Job 8 finished: reduce at VertexRDDImpl.scala:90, took 0.129802 s
15/06/03 18:34:25 INFO Pregel: Pregel finished iteration 6
15/06/03 18:34:25 INFO ZippedPartitionsRDD2: Removing RDD 157 from persistence list
15/06/03 18:34:25 INFO BlockManager: Removing RDD 157
15/06/03 18:34:25 INFO ZippedPartitionsRDD2: Removing RDD 160 from persistence list
15/06/03 18:34:25 INFO BlockManager: Removing block rdd_157_0
15/06/03 18:34:25 INFO MemoryStore: Block rdd_157_0 of size 1664 dropped from memory (free 558621323)
15/06/03 18:34:25 INFO BlockManager: Removing RDD 160
15/06/03 18:34:25 INFO ZippedPartitionsRDD2: Removing RDD 143 from persistence list
15/06/03 18:34:25 INFO BlockManager: Removing block rdd_160_0
15/06/03 18:34:25 INFO MemoryStore: Block rdd_160_0 of size 1544 dropped from memory (free 558622867)
15/06/03 18:34:25 INFO BlockManager: Removing RDD 143
15/06/03 18:34:25 INFO ZippedPartitionsRDD2: Removing RDD 149 from persistence list
15/06/03 18:34:25 INFO BlockManager: Removing block rdd_143_0
15/06/03 18:34:25 INFO MemoryStore: Block rdd_143_0 of size 1496 dropped from memory (free 558624363)
15/06/03 18:34:25 INFO BlockManager: Removing RDD 149
15/06/03 18:34:25 INFO BlockManager: Removing block rdd_149_0
15/06/03 18:34:25 INFO MemoryStore: Block rdd_149_0 of size 2976 dropped from memory (free 558627339)
15/06/03 18:34:25 INFO SparkContext: Starting job: reduce at VertexRDDImpl.scala:90
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 2 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 3 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 5 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 4 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 1 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 0 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 8 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 7 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 6 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 11 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 10 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 9 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 14 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 13 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 12 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 17 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 16 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 15 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 20 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 19 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 18 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 23 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 22 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 21 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 26 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 25 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 24 is 145 bytes
15/06/03 18:34:25 INFO DAGScheduler: Registering RDD 185 (mapPartitions at VertexRDDImpl.scala:218)
15/06/03 18:34:25 INFO DAGScheduler: Registering RDD 189 (mapPartitions at VertexRDDImpl.scala:222)
15/06/03 18:34:25 INFO DAGScheduler: Registering RDD 193 (mapPartitions at GraphImpl.scala:235)
15/06/03 18:34:25 INFO DAGScheduler: Got job 9 (reduce at VertexRDDImpl.scala:90) with 1 output partitions (allowLocal=false)
15/06/03 18:34:25 INFO DAGScheduler: Final stage: Stage 657(reduce at VertexRDDImpl.scala:90)
15/06/03 18:34:25 INFO DAGScheduler: Parents of final stage: List(Stage 640, Stage 644, Stage 648, Stage 637, Stage 652, Stage 646, Stage 656, Stage 638, Stage 650, Stage 642, Stage 654)
15/06/03 18:34:25 INFO DAGScheduler: Missing parents: List(Stage 656)
15/06/03 18:34:25 INFO DAGScheduler: Submitting Stage 627 (ReplicatedVertexView.updateVertices - shippedVerts true false (broadcast) MapPartitionsRDD[185] at mapPartitions at VertexRDDImpl.scala:218), which has no missing parents
15/06/03 18:34:25 INFO MemoryStore: ensureFreeSpace(10392) called with curMem=525813, maxMem=559153152
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_37 stored as values in memory (estimated size 10.1 KB, free 532.7 MB)
15/06/03 18:34:25 INFO MemoryStore: ensureFreeSpace(5692) called with curMem=536205, maxMem=559153152
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_37_piece0 stored as bytes in memory (estimated size 5.6 KB, free 532.7 MB)
15/06/03 18:34:25 INFO BlockManagerInfo: Added broadcast_37_piece0 in memory on localhost:60836 (size: 5.6 KB, free: 533.1 MB)
15/06/03 18:34:25 INFO BlockManagerMaster: Updated info of block broadcast_37_piece0
15/06/03 18:34:25 INFO SparkContext: Created broadcast 37 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:25 INFO DAGScheduler: Submitting 1 missing tasks from Stage 627 (ReplicatedVertexView.updateVertices - shippedVerts true false (broadcast) MapPartitionsRDD[185] at mapPartitions at VertexRDDImpl.scala:218)
15/06/03 18:34:25 INFO TaskSchedulerImpl: Adding task set 627.0 with 1 tasks
15/06/03 18:34:25 INFO DAGScheduler: Submitting Stage 655 (ReplicatedVertexView.withActiveSet - shippedActives (broadcast) MapPartitionsRDD[189] at mapPartitions at VertexRDDImpl.scala:222), which has no missing parents
15/06/03 18:34:25 INFO TaskSetManager: Starting task 0.0 in stage 627.0 (TID 36, localhost, PROCESS_LOCAL, 2366 bytes)
15/06/03 18:34:25 INFO Executor: Running task 0.0 in stage 627.0 (TID 36)
15/06/03 18:34:25 INFO MemoryStore: ensureFreeSpace(10048) called with curMem=541897, maxMem=559153152
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_38 stored as values in memory (estimated size 9.8 KB, free 532.7 MB)
15/06/03 18:34:25 INFO MemoryStore: ensureFreeSpace(5563) called with curMem=551945, maxMem=559153152
15/06/03 18:34:25 INFO BlockManager: Found block rdd_162_0 locally
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_38_piece0 stored as bytes in memory (estimated size 5.4 KB, free 532.7 MB)
15/06/03 18:34:25 INFO CacheManager: Partition rdd_181_0 not found, computing it
15/06/03 18:34:25 INFO BlockManagerInfo: Added broadcast_38_piece0 in memory on localhost:60836 (size: 5.4 KB, free: 533.1 MB)
15/06/03 18:34:25 INFO BlockManager: Found block rdd_162_0 locally
15/06/03 18:34:25 INFO BlockManagerMaster: Updated info of block broadcast_38_piece0
15/06/03 18:34:25 INFO CacheManager: Partition rdd_179_0 not found, computing it
15/06/03 18:34:25 INFO BlockManager: Found block rdd_162_0 locally
15/06/03 18:34:25 INFO BlockManager: Found block rdd_176_0 locally
15/06/03 18:34:25 INFO SparkContext: Created broadcast 38 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:25 INFO DAGScheduler: Submitting 1 missing tasks from Stage 655 (ReplicatedVertexView.withActiveSet - shippedActives (broadcast) MapPartitionsRDD[189] at mapPartitions at VertexRDDImpl.scala:222)
15/06/03 18:34:25 INFO TaskSchedulerImpl: Adding task set 655.0 with 1 tasks
15/06/03 18:34:25 INFO MemoryStore: ensureFreeSpace(1544) called with curMem=557508, maxMem=559153152
15/06/03 18:34:25 INFO MemoryStore: Block rdd_179_0 stored as values in memory (estimated size 1544.0 B, free 532.7 MB)
15/06/03 18:34:25 INFO BlockManagerInfo: Added rdd_179_0 in memory on localhost:60836 (size: 1544.0 B, free: 533.1 MB)
15/06/03 18:34:25 INFO BlockManagerMaster: Updated info of block rdd_179_0
15/06/03 18:34:25 INFO MemoryStore: ensureFreeSpace(1496) called with curMem=559052, maxMem=559153152
15/06/03 18:34:25 INFO MemoryStore: Block rdd_181_0 stored as values in memory (estimated size 1496.0 B, free 532.7 MB)
15/06/03 18:34:25 INFO BlockManagerInfo: Added rdd_181_0 in memory on localhost:60836 (size: 1496.0 B, free: 533.1 MB)
15/06/03 18:34:25 INFO BlockManagerMaster: Updated info of block rdd_181_0
15/06/03 18:34:25 INFO Executor: Finished task 0.0 in stage 627.0 (TID 36). 2627 bytes result sent to driver
15/06/03 18:34:25 INFO TaskSetManager: Starting task 0.0 in stage 655.0 (TID 37, localhost, PROCESS_LOCAL, 2302 bytes)
15/06/03 18:34:25 INFO Executor: Running task 0.0 in stage 655.0 (TID 37)
15/06/03 18:34:25 INFO TaskSetManager: Finished task 0.0 in stage 627.0 (TID 36) in 18 ms on localhost (1/1)
15/06/03 18:34:25 INFO TaskSchedulerImpl: Removed TaskSet 627.0, whose tasks have all completed, from pool
15/06/03 18:34:25 INFO DAGScheduler: Stage 627 (mapPartitions at VertexRDDImpl.scala:218) finished in 0.019 s
15/06/03 18:34:25 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:25 INFO DAGScheduler: running: Set(Stage 655)
15/06/03 18:34:25 INFO DAGScheduler: waiting: Set(Stage 656, Stage 657)
15/06/03 18:34:25 INFO DAGScheduler: failed: Set()
15/06/03 18:34:25 INFO BlockManager: Found block rdd_179_0 locally
15/06/03 18:34:25 INFO DAGScheduler: Missing parents for Stage 656: List(Stage 655)
15/06/03 18:34:25 INFO DAGScheduler: Missing parents for Stage 657: List(Stage 656)
15/06/03 18:34:25 INFO Executor: Finished task 0.0 in stage 655.0 (TID 37). 2003 bytes result sent to driver
15/06/03 18:34:25 INFO TaskSetManager: Finished task 0.0 in stage 655.0 (TID 37) in 12 ms on localhost (1/1)
15/06/03 18:34:25 INFO TaskSchedulerImpl: Removed TaskSet 655.0, whose tasks have all completed, from pool
15/06/03 18:34:25 INFO DAGScheduler: Stage 655 (mapPartitions at VertexRDDImpl.scala:222) finished in 0.022 s
15/06/03 18:34:25 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:25 INFO DAGScheduler: running: Set()
15/06/03 18:34:25 INFO DAGScheduler: waiting: Set(Stage 656, Stage 657)
15/06/03 18:34:25 INFO DAGScheduler: failed: Set()
15/06/03 18:34:25 INFO DAGScheduler: Missing parents for Stage 656: List()
15/06/03 18:34:25 INFO DAGScheduler: Missing parents for Stage 657: List(Stage 656)
15/06/03 18:34:25 INFO DAGScheduler: Submitting Stage 656 (GraphImpl.aggregateMessages - preAgg MapPartitionsRDD[193] at mapPartitions at GraphImpl.scala:235), which is now runnable
15/06/03 18:34:25 INFO MemoryStore: ensureFreeSpace(8672) called with curMem=560548, maxMem=559153152
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_39 stored as values in memory (estimated size 8.5 KB, free 532.7 MB)
15/06/03 18:34:25 INFO MemoryStore: ensureFreeSpace(5273) called with curMem=569220, maxMem=559153152
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_39_piece0 stored as bytes in memory (estimated size 5.1 KB, free 532.7 MB)
15/06/03 18:34:25 INFO BlockManagerInfo: Added broadcast_39_piece0 in memory on localhost:60836 (size: 5.1 KB, free: 533.1 MB)
15/06/03 18:34:25 INFO BlockManagerMaster: Updated info of block broadcast_39_piece0
15/06/03 18:34:25 INFO SparkContext: Created broadcast 39 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:25 INFO DAGScheduler: Submitting 1 missing tasks from Stage 656 (GraphImpl.aggregateMessages - preAgg MapPartitionsRDD[193] at mapPartitions at GraphImpl.scala:235)
15/06/03 18:34:25 INFO TaskSchedulerImpl: Adding task set 656.0 with 1 tasks
15/06/03 18:34:25 INFO TaskSetManager: Starting task 0.0 in stage 656.0 (TID 38, localhost, PROCESS_LOCAL, 2263 bytes)
15/06/03 18:34:25 INFO Executor: Running task 0.0 in stage 656.0 (TID 38)
15/06/03 18:34:25 INFO CacheManager: Partition rdd_187_0 not found, computing it
15/06/03 18:34:25 INFO BlockManager: Found block rdd_168_0 locally
15/06/03 18:34:25 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:25 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:25 INFO MemoryStore: ensureFreeSpace(2976) called with curMem=574493, maxMem=559153152
15/06/03 18:34:25 INFO MemoryStore: Block rdd_187_0 stored as values in memory (estimated size 2.9 KB, free 532.7 MB)
15/06/03 18:34:25 INFO BlockManagerInfo: Added rdd_187_0 in memory on localhost:60836 (size: 2.9 KB, free: 533.1 MB)
15/06/03 18:34:25 INFO BlockManagerMaster: Updated info of block rdd_187_0
15/06/03 18:34:25 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:25 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:25 INFO Executor: Finished task 0.0 in stage 656.0 (TID 38). 2788 bytes result sent to driver
15/06/03 18:34:25 INFO TaskSetManager: Finished task 0.0 in stage 656.0 (TID 38) in 16 ms on localhost (1/1)
15/06/03 18:34:25 INFO TaskSchedulerImpl: Removed TaskSet 656.0, whose tasks have all completed, from pool
15/06/03 18:34:25 INFO DAGScheduler: Stage 656 (mapPartitions at GraphImpl.scala:235) finished in 0.016 s
15/06/03 18:34:25 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:25 INFO DAGScheduler: running: Set()
15/06/03 18:34:25 INFO DAGScheduler: waiting: Set(Stage 657)
15/06/03 18:34:25 INFO DAGScheduler: failed: Set()
15/06/03 18:34:25 INFO DAGScheduler: Missing parents for Stage 657: List()
15/06/03 18:34:25 INFO DAGScheduler: Submitting Stage 657 (MapPartitionsRDD[197] at map at VertexRDDImpl.scala:90), which is now runnable
15/06/03 18:34:25 INFO MemoryStore: ensureFreeSpace(10680) called with curMem=577469, maxMem=559153152
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_40 stored as values in memory (estimated size 10.4 KB, free 532.7 MB)
15/06/03 18:34:25 INFO MemoryStore: ensureFreeSpace(5827) called with curMem=588149, maxMem=559153152
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_40_piece0 stored as bytes in memory (estimated size 5.7 KB, free 532.7 MB)
15/06/03 18:34:25 INFO BlockManagerInfo: Added broadcast_40_piece0 in memory on localhost:60836 (size: 5.7 KB, free: 533.1 MB)
15/06/03 18:34:25 INFO BlockManagerMaster: Updated info of block broadcast_40_piece0
15/06/03 18:34:25 INFO SparkContext: Created broadcast 40 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:25 INFO DAGScheduler: Submitting 1 missing tasks from Stage 657 (MapPartitionsRDD[197] at map at VertexRDDImpl.scala:90)
15/06/03 18:34:25 INFO TaskSchedulerImpl: Adding task set 657.0 with 1 tasks
15/06/03 18:34:25 INFO TaskSetManager: Starting task 0.0 in stage 657.0 (TID 39, localhost, PROCESS_LOCAL, 2386 bytes)
15/06/03 18:34:25 INFO Executor: Running task 0.0 in stage 657.0 (TID 39)
15/06/03 18:34:25 INFO CacheManager: Partition rdd_195_0 not found, computing it
15/06/03 18:34:25 INFO BlockManager: Found block rdd_181_0 locally
15/06/03 18:34:25 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:25 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:25 INFO MemoryStore: ensureFreeSpace(1664) called with curMem=593976, maxMem=559153152
15/06/03 18:34:25 INFO MemoryStore: Block rdd_195_0 stored as values in memory (estimated size 1664.0 B, free 532.7 MB)
15/06/03 18:34:25 INFO BlockManagerInfo: Added rdd_195_0 in memory on localhost:60836 (size: 1664.0 B, free: 533.1 MB)
15/06/03 18:34:25 INFO BlockManagerMaster: Updated info of block rdd_195_0
15/06/03 18:34:25 INFO Executor: Finished task 0.0 in stage 657.0 (TID 39). 2693 bytes result sent to driver
15/06/03 18:34:25 INFO TaskSetManager: Finished task 0.0 in stage 657.0 (TID 39) in 9 ms on localhost (1/1)
15/06/03 18:34:25 INFO TaskSchedulerImpl: Removed TaskSet 657.0, whose tasks have all completed, from pool
15/06/03 18:34:25 INFO DAGScheduler: Stage 657 (reduce at VertexRDDImpl.scala:90) finished in 0.009 s
15/06/03 18:34:25 INFO DAGScheduler: Job 9 finished: reduce at VertexRDDImpl.scala:90, took 0.125086 s
15/06/03 18:34:25 INFO Pregel: Pregel finished iteration 7
15/06/03 18:34:25 INFO ZippedPartitionsRDD2: Removing RDD 176 from persistence list
15/06/03 18:34:25 INFO BlockManager: Removing RDD 176
15/06/03 18:34:25 INFO ZippedPartitionsRDD2: Removing RDD 179 from persistence list
15/06/03 18:34:25 INFO BlockManager: Removing block rdd_176_0
15/06/03 18:34:25 INFO MemoryStore: Block rdd_176_0 of size 1664 dropped from memory (free 558559176)
15/06/03 18:34:25 INFO BlockManager: Removing RDD 179
15/06/03 18:34:25 INFO ZippedPartitionsRDD2: Removing RDD 162 from persistence list
15/06/03 18:34:25 INFO BlockManager: Removing block rdd_179_0
15/06/03 18:34:25 INFO MemoryStore: Block rdd_179_0 of size 1544 dropped from memory (free 558560720)
15/06/03 18:34:25 INFO BlockManager: Removing RDD 162
15/06/03 18:34:25 INFO BlockManager: Removing block rdd_162_0
15/06/03 18:34:25 INFO MemoryStore: Block rdd_162_0 of size 1496 dropped from memory (free 558562216)
15/06/03 18:34:25 INFO ZippedPartitionsRDD2: Removing RDD 168 from persistence list
15/06/03 18:34:25 INFO BlockManager: Removing RDD 168
15/06/03 18:34:25 INFO BlockManager: Removing block rdd_168_0
15/06/03 18:34:25 INFO MemoryStore: Block rdd_168_0 of size 2976 dropped from memory (free 558565192)
15/06/03 18:34:25 INFO SparkContext: Starting job: reduce at VertexRDDImpl.scala:90
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 2 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 3 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 5 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 4 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 1 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 0 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 8 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 7 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 6 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 11 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 10 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 9 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 14 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 13 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 12 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 17 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 16 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 15 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 20 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 19 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 18 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 23 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 22 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 21 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 26 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 25 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 24 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 29 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 28 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 27 is 145 bytes
15/06/03 18:34:25 INFO DAGScheduler: Registering RDD 204 (mapPartitions at VertexRDDImpl.scala:218)
15/06/03 18:34:25 INFO DAGScheduler: Registering RDD 208 (mapPartitions at VertexRDDImpl.scala:222)
15/06/03 18:34:25 INFO DAGScheduler: Registering RDD 212 (mapPartitions at GraphImpl.scala:235)
15/06/03 18:34:25 INFO DAGScheduler: Got job 10 (reduce at VertexRDDImpl.scala:90) with 1 output partitions (allowLocal=false)
15/06/03 18:34:25 INFO DAGScheduler: Final stage: Stage 855(reduce at VertexRDDImpl.scala:90)
15/06/03 18:34:25 INFO DAGScheduler: Parents of final stage: List(Stage 840, Stage 850, Stage 844, Stage 854, Stage 836, Stage 848, Stage 833, Stage 852, Stage 834, Stage 838, Stage 842, Stage 846)
15/06/03 18:34:25 INFO DAGScheduler: Missing parents: List(Stage 854)
15/06/03 18:34:25 INFO DAGScheduler: Submitting Stage 822 (ReplicatedVertexView.updateVertices - shippedVerts true false (broadcast) MapPartitionsRDD[204] at mapPartitions at VertexRDDImpl.scala:218), which has no missing parents
15/06/03 18:34:25 INFO MemoryStore: ensureFreeSpace(10920) called with curMem=587960, maxMem=559153152
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_41 stored as values in memory (estimated size 10.7 KB, free 532.7 MB)
15/06/03 18:34:25 INFO MemoryStore: ensureFreeSpace(5937) called with curMem=598880, maxMem=559153152
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_41_piece0 stored as bytes in memory (estimated size 5.8 KB, free 532.7 MB)
15/06/03 18:34:25 INFO BlockManagerInfo: Added broadcast_41_piece0 in memory on localhost:60836 (size: 5.8 KB, free: 533.1 MB)
15/06/03 18:34:25 INFO BlockManagerMaster: Updated info of block broadcast_41_piece0
15/06/03 18:34:25 INFO SparkContext: Created broadcast 41 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:25 INFO DAGScheduler: Submitting 1 missing tasks from Stage 822 (ReplicatedVertexView.updateVertices - shippedVerts true false (broadcast) MapPartitionsRDD[204] at mapPartitions at VertexRDDImpl.scala:218)
15/06/03 18:34:25 INFO TaskSchedulerImpl: Adding task set 822.0 with 1 tasks
15/06/03 18:34:25 INFO TaskSetManager: Starting task 0.0 in stage 822.0 (TID 40, localhost, PROCESS_LOCAL, 2471 bytes)
15/06/03 18:34:25 INFO Executor: Running task 0.0 in stage 822.0 (TID 40)
15/06/03 18:34:25 INFO DAGScheduler: Submitting Stage 853 (ReplicatedVertexView.withActiveSet - shippedActives (broadcast) MapPartitionsRDD[208] at mapPartitions at VertexRDDImpl.scala:222), which has no missing parents
15/06/03 18:34:25 INFO MemoryStore: ensureFreeSpace(10576) called with curMem=604817, maxMem=559153152
15/06/03 18:34:25 INFO BlockManager: Found block rdd_181_0 locally
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_42 stored as values in memory (estimated size 10.3 KB, free 532.7 MB)
15/06/03 18:34:25 INFO CacheManager: Partition rdd_200_0 not found, computing it
15/06/03 18:34:25 INFO BlockManager: Found block rdd_181_0 locally
15/06/03 18:34:25 INFO CacheManager: Partition rdd_198_0 not found, computing it
15/06/03 18:34:25 INFO BlockManager: Found block rdd_181_0 locally
15/06/03 18:34:25 INFO BlockManager: Found block rdd_195_0 locally
15/06/03 18:34:25 INFO MemoryStore: ensureFreeSpace(5815) called with curMem=615393, maxMem=559153152
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_42_piece0 stored as bytes in memory (estimated size 5.7 KB, free 532.7 MB)
15/06/03 18:34:25 INFO BlockManagerInfo: Added broadcast_42_piece0 in memory on localhost:60836 (size: 5.7 KB, free: 533.1 MB)
15/06/03 18:34:25 INFO MemoryStore: ensureFreeSpace(1544) called with curMem=621208, maxMem=559153152
15/06/03 18:34:25 INFO BlockManagerMaster: Updated info of block broadcast_42_piece0
15/06/03 18:34:25 INFO MemoryStore: Block rdd_198_0 stored as values in memory (estimated size 1544.0 B, free 532.7 MB)
15/06/03 18:34:25 INFO BlockManagerInfo: Added rdd_198_0 in memory on localhost:60836 (size: 1544.0 B, free: 533.1 MB)
15/06/03 18:34:25 INFO SparkContext: Created broadcast 42 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:25 INFO BlockManagerMaster: Updated info of block rdd_198_0
15/06/03 18:34:25 INFO DAGScheduler: Submitting 1 missing tasks from Stage 853 (ReplicatedVertexView.withActiveSet - shippedActives (broadcast) MapPartitionsRDD[208] at mapPartitions at VertexRDDImpl.scala:222)
15/06/03 18:34:25 INFO TaskSchedulerImpl: Adding task set 853.0 with 1 tasks
15/06/03 18:34:25 INFO MemoryStore: ensureFreeSpace(1496) called with curMem=622752, maxMem=559153152
15/06/03 18:34:25 INFO MemoryStore: Block rdd_200_0 stored as values in memory (estimated size 1496.0 B, free 532.7 MB)
15/06/03 18:34:25 INFO BlockManagerInfo: Added rdd_200_0 in memory on localhost:60836 (size: 1496.0 B, free: 533.1 MB)
15/06/03 18:34:25 INFO BlockManagerMaster: Updated info of block rdd_200_0
15/06/03 18:34:25 INFO Executor: Finished task 0.0 in stage 822.0 (TID 40). 2627 bytes result sent to driver
15/06/03 18:34:25 INFO TaskSetManager: Starting task 0.0 in stage 853.0 (TID 41, localhost, PROCESS_LOCAL, 2407 bytes)
15/06/03 18:34:25 INFO Executor: Running task 0.0 in stage 853.0 (TID 41)
15/06/03 18:34:25 INFO TaskSetManager: Finished task 0.0 in stage 822.0 (TID 40) in 29 ms on localhost (1/1)
15/06/03 18:34:25 INFO TaskSchedulerImpl: Removed TaskSet 822.0, whose tasks have all completed, from pool
15/06/03 18:34:25 INFO DAGScheduler: Stage 822 (mapPartitions at VertexRDDImpl.scala:218) finished in 0.028 s
15/06/03 18:34:25 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:25 INFO DAGScheduler: running: Set(Stage 853)
15/06/03 18:34:25 INFO DAGScheduler: waiting: Set(Stage 854, Stage 855)
15/06/03 18:34:25 INFO DAGScheduler: failed: Set()
15/06/03 18:34:25 INFO BlockManager: Found block rdd_198_0 locally
15/06/03 18:34:25 INFO DAGScheduler: Missing parents for Stage 854: List(Stage 853)
15/06/03 18:34:25 INFO DAGScheduler: Missing parents for Stage 855: List(Stage 854)
15/06/03 18:34:25 INFO Executor: Finished task 0.0 in stage 853.0 (TID 41). 2003 bytes result sent to driver
15/06/03 18:34:25 INFO TaskSetManager: Finished task 0.0 in stage 853.0 (TID 41) in 14 ms on localhost (1/1)
15/06/03 18:34:25 INFO TaskSchedulerImpl: Removed TaskSet 853.0, whose tasks have all completed, from pool
15/06/03 18:34:25 INFO DAGScheduler: Stage 853 (mapPartitions at VertexRDDImpl.scala:222) finished in 0.034 s
15/06/03 18:34:25 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:25 INFO DAGScheduler: running: Set()
15/06/03 18:34:25 INFO DAGScheduler: waiting: Set(Stage 854, Stage 855)
15/06/03 18:34:25 INFO DAGScheduler: failed: Set()
15/06/03 18:34:25 INFO DAGScheduler: Missing parents for Stage 854: List()
15/06/03 18:34:25 INFO DAGScheduler: Missing parents for Stage 855: List(Stage 854)
15/06/03 18:34:25 INFO DAGScheduler: Submitting Stage 854 (GraphImpl.aggregateMessages - preAgg MapPartitionsRDD[212] at mapPartitions at GraphImpl.scala:235), which is now runnable
15/06/03 18:34:25 INFO MemoryStore: ensureFreeSpace(8904) called with curMem=624248, maxMem=559153152
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_43 stored as values in memory (estimated size 8.7 KB, free 532.6 MB)
15/06/03 18:34:25 INFO MemoryStore: ensureFreeSpace(5343) called with curMem=633152, maxMem=559153152
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_43_piece0 stored as bytes in memory (estimated size 5.2 KB, free 532.6 MB)
15/06/03 18:34:25 INFO BlockManagerInfo: Added broadcast_43_piece0 in memory on localhost:60836 (size: 5.2 KB, free: 533.1 MB)
15/06/03 18:34:25 INFO BlockManagerMaster: Updated info of block broadcast_43_piece0
15/06/03 18:34:25 INFO SparkContext: Created broadcast 43 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:25 INFO DAGScheduler: Submitting 1 missing tasks from Stage 854 (GraphImpl.aggregateMessages - preAgg MapPartitionsRDD[212] at mapPartitions at GraphImpl.scala:235)
15/06/03 18:34:25 INFO TaskSchedulerImpl: Adding task set 854.0 with 1 tasks
15/06/03 18:34:25 INFO TaskSetManager: Starting task 0.0 in stage 854.0 (TID 42, localhost, PROCESS_LOCAL, 2304 bytes)
15/06/03 18:34:25 INFO Executor: Running task 0.0 in stage 854.0 (TID 42)
15/06/03 18:34:25 INFO CacheManager: Partition rdd_206_0 not found, computing it
15/06/03 18:34:25 INFO BlockManager: Found block rdd_187_0 locally
15/06/03 18:34:25 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:25 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:25 INFO MemoryStore: ensureFreeSpace(2976) called with curMem=638495, maxMem=559153152
15/06/03 18:34:25 INFO MemoryStore: Block rdd_206_0 stored as values in memory (estimated size 2.9 KB, free 532.6 MB)
15/06/03 18:34:25 INFO BlockManagerInfo: Added rdd_206_0 in memory on localhost:60836 (size: 2.9 KB, free: 533.0 MB)
15/06/03 18:34:25 INFO BlockManagerMaster: Updated info of block rdd_206_0
15/06/03 18:34:25 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:25 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
15/06/03 18:34:25 INFO Executor: Finished task 0.0 in stage 854.0 (TID 42). 2788 bytes result sent to driver
15/06/03 18:34:25 INFO TaskSetManager: Finished task 0.0 in stage 854.0 (TID 42) in 23 ms on localhost (1/1)
15/06/03 18:34:25 INFO TaskSchedulerImpl: Removed TaskSet 854.0, whose tasks have all completed, from pool
15/06/03 18:34:25 INFO DAGScheduler: Stage 854 (mapPartitions at GraphImpl.scala:235) finished in 0.024 s
15/06/03 18:34:25 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:25 INFO DAGScheduler: running: Set()
15/06/03 18:34:25 INFO DAGScheduler: waiting: Set(Stage 855)
15/06/03 18:34:25 INFO DAGScheduler: failed: Set()
15/06/03 18:34:25 INFO DAGScheduler: Missing parents for Stage 855: List()
15/06/03 18:34:25 INFO DAGScheduler: Submitting Stage 855 (MapPartitionsRDD[216] at map at VertexRDDImpl.scala:90), which is now runnable
15/06/03 18:34:25 INFO MemoryStore: ensureFreeSpace(11208) called with curMem=641471, maxMem=559153152
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_44 stored as values in memory (estimated size 10.9 KB, free 532.6 MB)
15/06/03 18:34:25 INFO MemoryStore: ensureFreeSpace(6044) called with curMem=652679, maxMem=559153152
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_44_piece0 stored as bytes in memory (estimated size 5.9 KB, free 532.6 MB)
15/06/03 18:34:25 INFO BlockManagerInfo: Added broadcast_44_piece0 in memory on localhost:60836 (size: 5.9 KB, free: 533.0 MB)
15/06/03 18:34:25 INFO BlockManagerMaster: Updated info of block broadcast_44_piece0
15/06/03 18:34:25 INFO SparkContext: Created broadcast 44 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:25 INFO DAGScheduler: Submitting 1 missing tasks from Stage 855 (MapPartitionsRDD[216] at map at VertexRDDImpl.scala:90)
15/06/03 18:34:25 INFO TaskSchedulerImpl: Adding task set 855.0 with 1 tasks
15/06/03 18:34:25 INFO TaskSetManager: Starting task 0.0 in stage 855.0 (TID 43, localhost, PROCESS_LOCAL, 2491 bytes)
15/06/03 18:34:25 INFO Executor: Running task 0.0 in stage 855.0 (TID 43)
15/06/03 18:34:25 INFO CacheManager: Partition rdd_214_0 not found, computing it
15/06/03 18:34:25 INFO BlockManager: Found block rdd_200_0 locally
15/06/03 18:34:25 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:25 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:25 INFO MemoryStore: ensureFreeSpace(1664) called with curMem=658723, maxMem=559153152
15/06/03 18:34:25 INFO MemoryStore: Block rdd_214_0 stored as values in memory (estimated size 1664.0 B, free 532.6 MB)
15/06/03 18:34:25 INFO BlockManagerInfo: Added rdd_214_0 in memory on localhost:60836 (size: 1664.0 B, free: 533.0 MB)
15/06/03 18:34:25 INFO BlockManagerMaster: Updated info of block rdd_214_0
15/06/03 18:34:25 INFO Executor: Finished task 0.0 in stage 855.0 (TID 43). 2693 bytes result sent to driver
15/06/03 18:34:25 INFO TaskSetManager: Finished task 0.0 in stage 855.0 (TID 43) in 10 ms on localhost (1/1)
15/06/03 18:34:25 INFO TaskSchedulerImpl: Removed TaskSet 855.0, whose tasks have all completed, from pool
15/06/03 18:34:25 INFO DAGScheduler: Stage 855 (reduce at VertexRDDImpl.scala:90) finished in 0.011 s
15/06/03 18:34:25 INFO DAGScheduler: Job 10 finished: reduce at VertexRDDImpl.scala:90, took 0.160383 s
15/06/03 18:34:25 INFO Pregel: Pregel finished iteration 8
15/06/03 18:34:25 INFO ZippedPartitionsRDD2: Removing RDD 195 from persistence list
15/06/03 18:34:25 INFO BlockManager: Removing RDD 195
15/06/03 18:34:25 INFO ZippedPartitionsRDD2: Removing RDD 198 from persistence list
15/06/03 18:34:25 INFO BlockManager: Removing block rdd_195_0
15/06/03 18:34:25 INFO MemoryStore: Block rdd_195_0 of size 1664 dropped from memory (free 558494429)
15/06/03 18:34:25 INFO BlockManager: Removing RDD 198
15/06/03 18:34:25 INFO ZippedPartitionsRDD2: Removing RDD 181 from persistence list
15/06/03 18:34:25 INFO BlockManager: Removing block rdd_198_0
15/06/03 18:34:25 INFO MemoryStore: Block rdd_198_0 of size 1544 dropped from memory (free 558495973)
15/06/03 18:34:25 INFO BlockManager: Removing RDD 181
15/06/03 18:34:25 INFO BlockManager: Removing block rdd_181_0
15/06/03 18:34:25 INFO MemoryStore: Block rdd_181_0 of size 1496 dropped from memory (free 558497469)
15/06/03 18:34:25 INFO ZippedPartitionsRDD2: Removing RDD 187 from persistence list
15/06/03 18:34:25 INFO BlockManager: Removing RDD 187
15/06/03 18:34:25 INFO BlockManager: Removing block rdd_187_0
15/06/03 18:34:25 INFO MemoryStore: Block rdd_187_0 of size 2976 dropped from memory (free 558500445)
15/06/03 18:34:25 INFO SparkContext: Starting job: reduce at VertexRDDImpl.scala:90
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 2 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 3 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 5 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 4 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 1 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 0 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 8 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 7 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 6 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 11 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 10 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 9 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 14 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 13 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 12 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 17 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 16 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 15 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 20 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 19 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 18 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 23 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 22 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 21 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 26 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 25 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 24 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 29 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 28 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 27 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 32 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 31 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 30 is 145 bytes
15/06/03 18:34:25 INFO DAGScheduler: Registering RDD 223 (mapPartitions at VertexRDDImpl.scala:218)
15/06/03 18:34:25 INFO DAGScheduler: Registering RDD 227 (mapPartitions at VertexRDDImpl.scala:222)
15/06/03 18:34:25 INFO DAGScheduler: Registering RDD 231 (mapPartitions at GraphImpl.scala:235)
15/06/03 18:34:25 INFO DAGScheduler: Got job 11 (reduce at VertexRDDImpl.scala:90) with 1 output partitions (allowLocal=false)
15/06/03 18:34:25 INFO DAGScheduler: Final stage: Stage 1089(reduce at VertexRDDImpl.scala:90)
15/06/03 18:34:25 INFO DAGScheduler: Parents of final stage: List(Stage 1086, Stage 1068, Stage 1080, Stage 1065, Stage 1072, Stage 1084, Stage 1066, Stage 1076, Stage 1088, Stage 1070, Stage 1074, Stage 1078, Stage 1082)
15/06/03 18:34:25 INFO DAGScheduler: Missing parents: List(Stage 1088)
15/06/03 18:34:25 INFO DAGScheduler: Submitting Stage 1053 (ReplicatedVertexView.updateVertices - shippedVerts true false (broadcast) MapPartitionsRDD[223] at mapPartitions at VertexRDDImpl.scala:218), which has no missing parents
15/06/03 18:34:25 INFO MemoryStore: ensureFreeSpace(11448) called with curMem=652707, maxMem=559153152
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_45 stored as values in memory (estimated size 11.2 KB, free 532.6 MB)
15/06/03 18:34:25 INFO MemoryStore: ensureFreeSpace(6146) called with curMem=664155, maxMem=559153152
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_45_piece0 stored as bytes in memory (estimated size 6.0 KB, free 532.6 MB)
15/06/03 18:34:25 INFO BlockManagerInfo: Added broadcast_45_piece0 in memory on localhost:60836 (size: 6.0 KB, free: 533.0 MB)
15/06/03 18:34:25 INFO BlockManagerMaster: Updated info of block broadcast_45_piece0
15/06/03 18:34:25 INFO SparkContext: Created broadcast 45 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:25 INFO DAGScheduler: Submitting 1 missing tasks from Stage 1053 (ReplicatedVertexView.updateVertices - shippedVerts true false (broadcast) MapPartitionsRDD[223] at mapPartitions at VertexRDDImpl.scala:218)
15/06/03 18:34:25 INFO TaskSchedulerImpl: Adding task set 1053.0 with 1 tasks
15/06/03 18:34:25 INFO DAGScheduler: Submitting Stage 1087 (ReplicatedVertexView.withActiveSet - shippedActives (broadcast) MapPartitionsRDD[227] at mapPartitions at VertexRDDImpl.scala:222), which has no missing parents
15/06/03 18:34:25 INFO TaskSetManager: Starting task 0.0 in stage 1053.0 (TID 44, localhost, PROCESS_LOCAL, 2576 bytes)
15/06/03 18:34:25 INFO Executor: Running task 0.0 in stage 1053.0 (TID 44)
15/06/03 18:34:25 INFO MemoryStore: ensureFreeSpace(11104) called with curMem=670301, maxMem=559153152
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_46 stored as values in memory (estimated size 10.8 KB, free 532.6 MB)
15/06/03 18:34:25 INFO MemoryStore: ensureFreeSpace(5997) called with curMem=681405, maxMem=559153152
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_46_piece0 stored as bytes in memory (estimated size 5.9 KB, free 532.6 MB)
15/06/03 18:34:25 INFO BlockManagerInfo: Added broadcast_46_piece0 in memory on localhost:60836 (size: 5.9 KB, free: 533.0 MB)
15/06/03 18:34:25 INFO BlockManager: Found block rdd_200_0 locally
15/06/03 18:34:25 INFO BlockManagerMaster: Updated info of block broadcast_46_piece0
15/06/03 18:34:25 INFO CacheManager: Partition rdd_219_0 not found, computing it
15/06/03 18:34:25 INFO BlockManager: Found block rdd_200_0 locally
15/06/03 18:34:25 INFO SparkContext: Created broadcast 46 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:25 INFO CacheManager: Partition rdd_217_0 not found, computing it
15/06/03 18:34:25 INFO BlockManager: Found block rdd_200_0 locally
15/06/03 18:34:25 INFO BlockManager: Found block rdd_214_0 locally
15/06/03 18:34:25 INFO DAGScheduler: Submitting 1 missing tasks from Stage 1087 (ReplicatedVertexView.withActiveSet - shippedActives (broadcast) MapPartitionsRDD[227] at mapPartitions at VertexRDDImpl.scala:222)
15/06/03 18:34:25 INFO TaskSchedulerImpl: Adding task set 1087.0 with 1 tasks
15/06/03 18:34:25 INFO MemoryStore: ensureFreeSpace(1544) called with curMem=687402, maxMem=559153152
15/06/03 18:34:25 INFO MemoryStore: Block rdd_217_0 stored as values in memory (estimated size 1544.0 B, free 532.6 MB)
15/06/03 18:34:25 INFO BlockManagerInfo: Added rdd_217_0 in memory on localhost:60836 (size: 1544.0 B, free: 533.0 MB)
15/06/03 18:34:25 INFO BlockManagerMaster: Updated info of block rdd_217_0
15/06/03 18:34:25 INFO MemoryStore: ensureFreeSpace(1496) called with curMem=688946, maxMem=559153152
15/06/03 18:34:25 INFO MemoryStore: Block rdd_219_0 stored as values in memory (estimated size 1496.0 B, free 532.6 MB)
15/06/03 18:34:25 INFO BlockManagerInfo: Added rdd_219_0 in memory on localhost:60836 (size: 1496.0 B, free: 533.0 MB)
15/06/03 18:34:25 INFO BlockManagerMaster: Updated info of block rdd_219_0
15/06/03 18:34:25 INFO Executor: Finished task 0.0 in stage 1053.0 (TID 44). 2627 bytes result sent to driver
15/06/03 18:34:25 INFO TaskSetManager: Starting task 0.0 in stage 1087.0 (TID 45, localhost, PROCESS_LOCAL, 2512 bytes)
15/06/03 18:34:25 INFO Executor: Running task 0.0 in stage 1087.0 (TID 45)
15/06/03 18:34:25 INFO TaskSetManager: Finished task 0.0 in stage 1053.0 (TID 44) in 25 ms on localhost (1/1)
15/06/03 18:34:25 INFO TaskSchedulerImpl: Removed TaskSet 1053.0, whose tasks have all completed, from pool
15/06/03 18:34:25 INFO DAGScheduler: Stage 1053 (mapPartitions at VertexRDDImpl.scala:218) finished in 0.025 s
15/06/03 18:34:25 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:25 INFO DAGScheduler: running: Set(Stage 1087)
15/06/03 18:34:25 INFO DAGScheduler: waiting: Set(Stage 1088, Stage 1089)
15/06/03 18:34:25 INFO DAGScheduler: failed: Set()
15/06/03 18:34:25 INFO DAGScheduler: Missing parents for Stage 1088: List(Stage 1087)
15/06/03 18:34:25 INFO DAGScheduler: Missing parents for Stage 1089: List(Stage 1088)
15/06/03 18:34:25 INFO BlockManager: Found block rdd_217_0 locally
15/06/03 18:34:25 INFO Executor: Finished task 0.0 in stage 1087.0 (TID 45). 2003 bytes result sent to driver
15/06/03 18:34:25 INFO TaskSetManager: Finished task 0.0 in stage 1087.0 (TID 45) in 14 ms on localhost (1/1)
15/06/03 18:34:25 INFO TaskSchedulerImpl: Removed TaskSet 1087.0, whose tasks have all completed, from pool
15/06/03 18:34:25 INFO DAGScheduler: Stage 1087 (mapPartitions at VertexRDDImpl.scala:222) finished in 0.032 s
15/06/03 18:34:25 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:25 INFO DAGScheduler: running: Set()
15/06/03 18:34:25 INFO DAGScheduler: waiting: Set(Stage 1088, Stage 1089)
15/06/03 18:34:25 INFO DAGScheduler: failed: Set()
15/06/03 18:34:25 INFO DAGScheduler: Missing parents for Stage 1088: List()
15/06/03 18:34:25 INFO DAGScheduler: Missing parents for Stage 1089: List(Stage 1088)
15/06/03 18:34:25 INFO DAGScheduler: Submitting Stage 1088 (GraphImpl.aggregateMessages - preAgg MapPartitionsRDD[231] at mapPartitions at GraphImpl.scala:235), which is now runnable
15/06/03 18:34:25 INFO MemoryStore: ensureFreeSpace(9128) called with curMem=690442, maxMem=559153152
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_47 stored as values in memory (estimated size 8.9 KB, free 532.6 MB)
15/06/03 18:34:25 INFO MemoryStore: ensureFreeSpace(5401) called with curMem=699570, maxMem=559153152
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_47_piece0 stored as bytes in memory (estimated size 5.3 KB, free 532.6 MB)
15/06/03 18:34:25 INFO BlockManager: Removing broadcast 20
15/06/03 18:34:25 INFO BlockManager: Removing block broadcast_20
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_20 of size 8040 dropped from memory (free 558456221)
15/06/03 18:34:25 INFO BlockManager: Removing block broadcast_20_piece0
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_20_piece0 of size 4815 dropped from memory (free 558461036)
15/06/03 18:34:25 INFO BlockManagerInfo: Added broadcast_47_piece0 in memory on localhost:60836 (size: 5.3 KB, free: 533.0 MB)
15/06/03 18:34:25 INFO BlockManagerInfo: Removed broadcast_20_piece0 on localhost:60836 in memory (size: 4.7 KB, free: 533.0 MB)
15/06/03 18:34:25 INFO BlockManagerMaster: Updated info of block broadcast_47_piece0
15/06/03 18:34:25 INFO BlockManagerMaster: Updated info of block broadcast_20_piece0
15/06/03 18:34:25 INFO ContextCleaner: Cleaned broadcast 20
15/06/03 18:34:25 INFO SparkContext: Created broadcast 47 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:25 INFO DAGScheduler: Submitting 1 missing tasks from Stage 1088 (GraphImpl.aggregateMessages - preAgg MapPartitionsRDD[231] at mapPartitions at GraphImpl.scala:235)
15/06/03 18:34:25 INFO TaskSchedulerImpl: Adding task set 1088.0 with 1 tasks
15/06/03 18:34:25 INFO BlockManager: Removing broadcast 19
15/06/03 18:34:25 INFO BlockManager: Removing block broadcast_19
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_19 of size 7536 dropped from memory (free 558468572)
15/06/03 18:34:25 INFO BlockManager: Removing block broadcast_19_piece0
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_19_piece0 of size 4796 dropped from memory (free 558473368)
15/06/03 18:34:25 INFO TaskSetManager: Starting task 0.0 in stage 1088.0 (TID 46, localhost, PROCESS_LOCAL, 2345 bytes)
15/06/03 18:34:25 INFO Executor: Running task 0.0 in stage 1088.0 (TID 46)
15/06/03 18:34:25 INFO BlockManagerInfo: Removed broadcast_19_piece0 on localhost:60836 in memory (size: 4.7 KB, free: 533.0 MB)
15/06/03 18:34:25 INFO BlockManagerMaster: Updated info of block broadcast_19_piece0
15/06/03 18:34:25 INFO ContextCleaner: Cleaned broadcast 19
15/06/03 18:34:25 INFO BlockManager: Removing broadcast 18
15/06/03 18:34:25 INFO BlockManager: Removing block broadcast_18_piece0
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_18_piece0 of size 4548 dropped from memory (free 558477916)
15/06/03 18:34:25 INFO BlockManagerInfo: Removed broadcast_18_piece0 on localhost:60836 in memory (size: 4.4 KB, free: 533.0 MB)
15/06/03 18:34:25 INFO BlockManagerMaster: Updated info of block broadcast_18_piece0
15/06/03 18:34:25 INFO BlockManager: Removing block broadcast_18
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_18 of size 7416 dropped from memory (free 558485332)
15/06/03 18:34:25 INFO ContextCleaner: Cleaned broadcast 18
15/06/03 18:34:25 INFO BlockManager: Removing broadcast 17
15/06/03 18:34:25 INFO BlockManager: Removing block broadcast_17_piece0
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_17_piece0 of size 4673 dropped from memory (free 558490005)
15/06/03 18:34:25 INFO BlockManagerInfo: Removed broadcast_17_piece0 on localhost:60836 in memory (size: 4.6 KB, free: 533.0 MB)
15/06/03 18:34:25 INFO BlockManagerMaster: Updated info of block broadcast_17_piece0
15/06/03 18:34:25 INFO BlockManager: Removing block broadcast_17
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_17 of size 7760 dropped from memory (free 558497765)
15/06/03 18:34:25 INFO ContextCleaner: Cleaned broadcast 17
15/06/03 18:34:25 INFO CacheManager: Partition rdd_225_0 not found, computing it
15/06/03 18:34:25 INFO BlockManager: Found block rdd_206_0 locally
15/06/03 18:34:25 INFO BlockManager: Removing broadcast 16
15/06/03 18:34:25 INFO BlockManager: Removing block broadcast_16_piece0
15/06/03 18:34:25 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_16_piece0 of size 4584 dropped from memory (free 558502349)
15/06/03 18:34:25 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
15/06/03 18:34:25 INFO BlockManagerInfo: Removed broadcast_16_piece0 on localhost:60836 in memory (size: 4.5 KB, free: 533.1 MB)
15/06/03 18:34:25 INFO BlockManagerMaster: Updated info of block broadcast_16_piece0
15/06/03 18:34:25 INFO BlockManager: Removing block broadcast_16
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_16 of size 7520 dropped from memory (free 558509869)
15/06/03 18:34:25 INFO ContextCleaner: Cleaned broadcast 16
15/06/03 18:34:25 INFO BlockManager: Removing broadcast 15
15/06/03 18:34:25 INFO BlockManager: Removing block broadcast_15_piece0
15/06/03 18:34:25 INFO MemoryStore: ensureFreeSpace(2976) called with curMem=643283, maxMem=559153152
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_15_piece0 of size 4628 dropped from memory (free 558514497)
15/06/03 18:34:25 INFO MemoryStore: Block rdd_225_0 stored as values in memory (estimated size 2.9 KB, free 532.6 MB)
15/06/03 18:34:25 INFO BlockManagerInfo: Removed broadcast_15_piece0 on localhost:60836 in memory (size: 4.5 KB, free: 533.1 MB)
15/06/03 18:34:25 INFO BlockManagerMaster: Updated info of block broadcast_15_piece0
15/06/03 18:34:25 INFO BlockManager: Removing block broadcast_15
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_15 of size 7304 dropped from memory (free 558518825)
15/06/03 18:34:25 INFO BlockManagerInfo: Added rdd_225_0 in memory on localhost:60836 (size: 2.9 KB, free: 533.1 MB)
15/06/03 18:34:25 INFO BlockManagerMaster: Updated info of block rdd_225_0
15/06/03 18:34:25 INFO ContextCleaner: Cleaned broadcast 15
15/06/03 18:34:25 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:25 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:25 INFO BlockManager: Removing broadcast 14
15/06/03 18:34:25 INFO BlockManager: Removing block broadcast_14_piece0
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_14_piece0 of size 4286 dropped from memory (free 558523111)
15/06/03 18:34:25 INFO BlockManagerInfo: Removed broadcast_14_piece0 on localhost:60836 in memory (size: 4.2 KB, free: 533.1 MB)
15/06/03 18:34:25 INFO BlockManagerMaster: Updated info of block broadcast_14_piece0
15/06/03 18:34:25 INFO BlockManager: Removing block broadcast_14
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_14 of size 6888 dropped from memory (free 558529999)
15/06/03 18:34:25 INFO ContextCleaner: Cleaned broadcast 14
15/06/03 18:34:25 INFO BlockManager: Removing broadcast 13
15/06/03 18:34:25 INFO BlockManager: Removing block broadcast_13_piece0
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_13_piece0 of size 4438 dropped from memory (free 558534437)
15/06/03 18:34:25 INFO BlockManagerInfo: Removed broadcast_13_piece0 on localhost:60836 in memory (size: 4.3 KB, free: 533.1 MB)
15/06/03 18:34:25 INFO BlockManagerMaster: Updated info of block broadcast_13_piece0
15/06/03 18:34:25 INFO BlockManager: Removing block broadcast_13
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_13 of size 7232 dropped from memory (free 558541669)
15/06/03 18:34:25 INFO ContextCleaner: Cleaned broadcast 13
15/06/03 18:34:25 INFO BlockManager: Removing broadcast 12
15/06/03 18:34:25 INFO BlockManager: Removing block broadcast_12
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_12 of size 6992 dropped from memory (free 558548661)
15/06/03 18:34:25 INFO BlockManager: Removing block broadcast_12_piece0
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_12_piece0 of size 4344 dropped from memory (free 558553005)
15/06/03 18:34:25 INFO BlockManagerInfo: Removed broadcast_12_piece0 on localhost:60836 in memory (size: 4.2 KB, free: 533.1 MB)
15/06/03 18:34:25 INFO BlockManagerMaster: Updated info of block broadcast_12_piece0
15/06/03 18:34:25 INFO ContextCleaner: Cleaned broadcast 12
15/06/03 18:34:25 INFO BlockManager: Removing broadcast 11
15/06/03 18:34:25 INFO BlockManager: Removing block broadcast_11
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_11 of size 7080 dropped from memory (free 558560085)
15/06/03 18:34:25 INFO BlockManager: Removing block broadcast_11_piece0
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_11_piece0 of size 4505 dropped from memory (free 558564590)
15/06/03 18:34:25 INFO BlockManagerInfo: Removed broadcast_11_piece0 on localhost:60836 in memory (size: 4.4 KB, free: 533.1 MB)
15/06/03 18:34:25 INFO BlockManagerMaster: Updated info of block broadcast_11_piece0
15/06/03 18:34:25 INFO ContextCleaner: Cleaned broadcast 11
15/06/03 18:34:25 INFO BlockManager: Removing broadcast 45
15/06/03 18:34:25 INFO BlockManager: Removing block broadcast_45
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_45 of size 11448 dropped from memory (free 558576038)
15/06/03 18:34:25 INFO BlockManager: Removing block broadcast_45_piece0
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_45_piece0 of size 6146 dropped from memory (free 558582184)
15/06/03 18:34:25 INFO BlockManagerInfo: Removed broadcast_45_piece0 on localhost:60836 in memory (size: 6.0 KB, free: 533.1 MB)
15/06/03 18:34:25 INFO BlockManagerMaster: Updated info of block broadcast_45_piece0
15/06/03 18:34:25 INFO Executor: Finished task 0.0 in stage 1088.0 (TID 46). 2788 bytes result sent to driver
15/06/03 18:34:25 INFO ContextCleaner: Cleaned broadcast 45
15/06/03 18:34:25 INFO BlockManager: Removing broadcast 44
15/06/03 18:34:25 INFO BlockManager: Removing block broadcast_44_piece0
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_44_piece0 of size 6044 dropped from memory (free 558588228)
15/06/03 18:34:25 INFO TaskSetManager: Finished task 0.0 in stage 1088.0 (TID 46) in 18 ms on localhost (1/1)
15/06/03 18:34:25 INFO TaskSchedulerImpl: Removed TaskSet 1088.0, whose tasks have all completed, from pool
15/06/03 18:34:25 INFO DAGScheduler: Stage 1088 (mapPartitions at GraphImpl.scala:235) finished in 0.018 s
15/06/03 18:34:25 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:25 INFO DAGScheduler: running: Set()
15/06/03 18:34:25 INFO DAGScheduler: waiting: Set(Stage 1089)
15/06/03 18:34:25 INFO DAGScheduler: failed: Set()
15/06/03 18:34:25 INFO BlockManagerInfo: Removed broadcast_44_piece0 on localhost:60836 in memory (size: 5.9 KB, free: 533.1 MB)
15/06/03 18:34:25 INFO BlockManagerMaster: Updated info of block broadcast_44_piece0
15/06/03 18:34:25 INFO BlockManager: Removing block broadcast_44
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_44 of size 11208 dropped from memory (free 558599436)
15/06/03 18:34:25 INFO ContextCleaner: Cleaned broadcast 44
15/06/03 18:34:25 INFO BlockManager: Removing broadcast 43
15/06/03 18:34:25 INFO BlockManager: Removing block broadcast_43
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_43 of size 8904 dropped from memory (free 558608340)
15/06/03 18:34:25 INFO BlockManager: Removing block broadcast_43_piece0
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_43_piece0 of size 5343 dropped from memory (free 558613683)
15/06/03 18:34:25 INFO DAGScheduler: Missing parents for Stage 1089: List()
15/06/03 18:34:25 INFO BlockManagerInfo: Removed broadcast_43_piece0 on localhost:60836 in memory (size: 5.2 KB, free: 533.1 MB)
15/06/03 18:34:25 INFO BlockManagerMaster: Updated info of block broadcast_43_piece0
15/06/03 18:34:25 INFO ContextCleaner: Cleaned broadcast 43
15/06/03 18:34:25 INFO DAGScheduler: Submitting Stage 1089 (MapPartitionsRDD[235] at map at VertexRDDImpl.scala:90), which is now runnable
15/06/03 18:34:25 INFO BlockManager: Removing broadcast 42
15/06/03 18:34:25 INFO BlockManager: Removing block broadcast_42
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_42 of size 10576 dropped from memory (free 558624259)
15/06/03 18:34:25 INFO BlockManager: Removing block broadcast_42_piece0
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_42_piece0 of size 5815 dropped from memory (free 558630074)
15/06/03 18:34:25 INFO BlockManagerInfo: Removed broadcast_42_piece0 on localhost:60836 in memory (size: 5.7 KB, free: 533.1 MB)
15/06/03 18:34:25 INFO MemoryStore: ensureFreeSpace(11736) called with curMem=523078, maxMem=559153152
15/06/03 18:34:25 INFO BlockManagerMaster: Updated info of block broadcast_42_piece0
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_48 stored as values in memory (estimated size 11.5 KB, free 532.7 MB)
15/06/03 18:34:25 INFO ContextCleaner: Cleaned broadcast 42
15/06/03 18:34:25 INFO MemoryStore: ensureFreeSpace(6275) called with curMem=534814, maxMem=559153152
15/06/03 18:34:25 INFO BlockManager: Removing broadcast 41
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_48_piece0 stored as bytes in memory (estimated size 6.1 KB, free 532.7 MB)
15/06/03 18:34:25 INFO BlockManager: Removing block broadcast_41_piece0
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_41_piece0 of size 5937 dropped from memory (free 558618000)
15/06/03 18:34:25 INFO BlockManagerInfo: Added broadcast_48_piece0 in memory on localhost:60836 (size: 6.1 KB, free: 533.1 MB)
15/06/03 18:34:25 INFO BlockManagerMaster: Updated info of block broadcast_48_piece0
15/06/03 18:34:25 INFO BlockManagerInfo: Removed broadcast_41_piece0 on localhost:60836 in memory (size: 5.8 KB, free: 533.1 MB)
15/06/03 18:34:25 INFO BlockManagerMaster: Updated info of block broadcast_41_piece0
15/06/03 18:34:25 INFO BlockManager: Removing block broadcast_41
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_41 of size 10920 dropped from memory (free 558628920)
15/06/03 18:34:25 INFO ContextCleaner: Cleaned broadcast 41
15/06/03 18:34:25 INFO SparkContext: Created broadcast 48 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:25 INFO DAGScheduler: Submitting 1 missing tasks from Stage 1089 (MapPartitionsRDD[235] at map at VertexRDDImpl.scala:90)
15/06/03 18:34:25 INFO BlockManager: Removing broadcast 40
15/06/03 18:34:25 INFO TaskSchedulerImpl: Adding task set 1089.0 with 1 tasks
15/06/03 18:34:25 INFO BlockManager: Removing block broadcast_40_piece0
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_40_piece0 of size 5827 dropped from memory (free 558634747)
15/06/03 18:34:25 INFO BlockManagerInfo: Removed broadcast_40_piece0 on localhost:60836 in memory (size: 5.7 KB, free: 533.1 MB)
15/06/03 18:34:25 INFO BlockManagerMaster: Updated info of block broadcast_40_piece0
15/06/03 18:34:25 INFO BlockManager: Removing block broadcast_40
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_40 of size 10680 dropped from memory (free 558645427)
15/06/03 18:34:25 INFO ContextCleaner: Cleaned broadcast 40
15/06/03 18:34:25 INFO BlockManager: Removing broadcast 39
15/06/03 18:34:25 INFO TaskSetManager: Starting task 0.0 in stage 1089.0 (TID 47, localhost, PROCESS_LOCAL, 2596 bytes)
15/06/03 18:34:25 INFO BlockManager: Removing block broadcast_39_piece0
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_39_piece0 of size 5273 dropped from memory (free 558650700)
15/06/03 18:34:25 INFO Executor: Running task 0.0 in stage 1089.0 (TID 47)
15/06/03 18:34:25 INFO BlockManagerInfo: Removed broadcast_39_piece0 on localhost:60836 in memory (size: 5.1 KB, free: 533.1 MB)
15/06/03 18:34:25 INFO BlockManagerMaster: Updated info of block broadcast_39_piece0
15/06/03 18:34:25 INFO BlockManager: Removing block broadcast_39
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_39 of size 8672 dropped from memory (free 558659372)
15/06/03 18:34:25 INFO ContextCleaner: Cleaned broadcast 39
15/06/03 18:34:25 INFO BlockManager: Removing broadcast 38
15/06/03 18:34:25 INFO BlockManager: Removing block broadcast_38_piece0
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_38_piece0 of size 5563 dropped from memory (free 558664935)
15/06/03 18:34:25 INFO BlockManagerInfo: Removed broadcast_38_piece0 on localhost:60836 in memory (size: 5.4 KB, free: 533.1 MB)
15/06/03 18:34:25 INFO BlockManagerMaster: Updated info of block broadcast_38_piece0
15/06/03 18:34:25 INFO BlockManager: Removing block broadcast_38
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_38 of size 10048 dropped from memory (free 558674983)
15/06/03 18:34:25 INFO ContextCleaner: Cleaned broadcast 38
15/06/03 18:34:25 INFO BlockManager: Removing broadcast 37
15/06/03 18:34:25 INFO BlockManager: Removing block broadcast_37_piece0
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_37_piece0 of size 5692 dropped from memory (free 558680675)
15/06/03 18:34:25 INFO BlockManagerInfo: Removed broadcast_37_piece0 on localhost:60836 in memory (size: 5.6 KB, free: 533.1 MB)
15/06/03 18:34:25 INFO BlockManagerMaster: Updated info of block broadcast_37_piece0
15/06/03 18:34:25 INFO BlockManager: Removing block broadcast_37
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_37 of size 10392 dropped from memory (free 558691067)
15/06/03 18:34:25 INFO ContextCleaner: Cleaned broadcast 37
15/06/03 18:34:25 INFO BlockManager: Removing broadcast 25
15/06/03 18:34:25 INFO CacheManager: Partition rdd_233_0 not found, computing it
15/06/03 18:34:25 INFO BlockManager: Removing block broadcast_25_piece0
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_25_piece0 of size 5103 dropped from memory (free 558696170)
15/06/03 18:34:25 INFO BlockManager: Found block rdd_219_0 locally
15/06/03 18:34:25 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:25 INFO BlockManagerInfo: Removed broadcast_25_piece0 on localhost:60836 in memory (size: 5.0 KB, free: 533.1 MB)
15/06/03 18:34:25 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:25 INFO BlockManagerMaster: Updated info of block broadcast_25_piece0
15/06/03 18:34:25 INFO BlockManager: Removing block broadcast_25
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_25 of size 8808 dropped from memory (free 558704978)
15/06/03 18:34:25 INFO ContextCleaner: Cleaned broadcast 25
15/06/03 18:34:25 INFO BlockManager: Removing broadcast 24
15/06/03 18:34:25 INFO BlockManager: Removing block broadcast_24_piece0
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_24_piece0 of size 5006 dropped from memory (free 558709984)
15/06/03 18:34:25 INFO BlockManagerInfo: Removed broadcast_24_piece0 on localhost:60836 in memory (size: 4.9 KB, free: 533.1 MB)
15/06/03 18:34:25 INFO BlockManagerMaster: Updated info of block broadcast_24_piece0
15/06/03 18:34:25 INFO BlockManager: Removing block broadcast_24
15/06/03 18:34:25 INFO MemoryStore: ensureFreeSpace(1664) called with curMem=443168, maxMem=559153152
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_24 of size 8568 dropped from memory (free 558718552)
15/06/03 18:34:25 INFO MemoryStore: Block rdd_233_0 stored as values in memory (estimated size 1664.0 B, free 532.8 MB)
15/06/03 18:34:25 INFO ContextCleaner: Cleaned broadcast 24
15/06/03 18:34:25 INFO BlockManagerInfo: Added rdd_233_0 in memory on localhost:60836 (size: 1664.0 B, free: 533.1 MB)
15/06/03 18:34:25 INFO BlockManagerMaster: Updated info of block rdd_233_0
15/06/03 18:34:25 INFO BlockManager: Removing broadcast 23
15/06/03 18:34:25 INFO BlockManager: Removing block broadcast_23_piece0
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_23_piece0 of size 4856 dropped from memory (free 558721744)
15/06/03 18:34:25 INFO Executor: Finished task 0.0 in stage 1089.0 (TID 47). 2693 bytes result sent to driver
15/06/03 18:34:25 INFO BlockManagerInfo: Removed broadcast_23_piece0 on localhost:60836 in memory (size: 4.7 KB, free: 533.1 MB)
15/06/03 18:34:25 INFO BlockManagerMaster: Updated info of block broadcast_23_piece0
15/06/03 18:34:25 INFO BlockManager: Removing block broadcast_23
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_23 of size 7760 dropped from memory (free 558729504)
15/06/03 18:34:25 INFO ContextCleaner: Cleaned broadcast 23
15/06/03 18:34:25 INFO TaskSetManager: Finished task 0.0 in stage 1089.0 (TID 47) in 10 ms on localhost (1/1)
15/06/03 18:34:25 INFO TaskSchedulerImpl: Removed TaskSet 1089.0, whose tasks have all completed, from pool
15/06/03 18:34:25 INFO BlockManager: Removing broadcast 22
15/06/03 18:34:25 INFO BlockManager: Removing block broadcast_22_piece0
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_22_piece0 of size 4778 dropped from memory (free 558734282)
15/06/03 18:34:25 INFO DAGScheduler: Stage 1089 (reduce at VertexRDDImpl.scala:90) finished in 0.010 s
15/06/03 18:34:25 INFO BlockManagerInfo: Removed broadcast_22_piece0 on localhost:60836 in memory (size: 4.7 KB, free: 533.1 MB)
15/06/03 18:34:25 INFO BlockManagerMaster: Updated info of block broadcast_22_piece0
15/06/03 18:34:25 INFO BlockManager: Removing block broadcast_22
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_22 of size 7944 dropped from memory (free 558742226)
15/06/03 18:34:25 INFO ContextCleaner: Cleaned broadcast 22
15/06/03 18:34:25 INFO BlockManager: Removing broadcast 21
15/06/03 18:34:25 INFO BlockManager: Removing block broadcast_21_piece0
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_21_piece0 of size 4909 dropped from memory (free 558747135)
15/06/03 18:34:25 INFO BlockManagerInfo: Removed broadcast_21_piece0 on localhost:60836 in memory (size: 4.8 KB, free: 533.1 MB)
15/06/03 18:34:25 INFO BlockManagerMaster: Updated info of block broadcast_21_piece0
15/06/03 18:34:25 INFO BlockManager: Removing block broadcast_21
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_21 of size 8280 dropped from memory (free 558755415)
15/06/03 18:34:25 INFO ContextCleaner: Cleaned broadcast 21
15/06/03 18:34:25 INFO BlockManager: Removing broadcast 36
15/06/03 18:34:25 INFO BlockManager: Removing block broadcast_36_piece0
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_36_piece0 of size 5614 dropped from memory (free 558761029)
15/06/03 18:34:25 INFO BlockManagerInfo: Removed broadcast_36_piece0 on localhost:60836 in memory (size: 5.5 KB, free: 533.1 MB)
15/06/03 18:34:25 INFO BlockManagerMaster: Updated info of block broadcast_36_piece0
15/06/03 18:34:25 INFO BlockManager: Removing block broadcast_36
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_36 of size 10152 dropped from memory (free 558771181)
15/06/03 18:34:25 INFO DAGScheduler: Job 11 finished: reduce at VertexRDDImpl.scala:90, took 0.150274 s
15/06/03 18:34:25 INFO ContextCleaner: Cleaned broadcast 36
15/06/03 18:34:25 INFO Pregel: Pregel finished iteration 9
15/06/03 18:34:25 INFO ZippedPartitionsRDD2: Removing RDD 214 from persistence list
15/06/03 18:34:25 INFO BlockManager: Removing broadcast 35
15/06/03 18:34:25 INFO BlockManager: Removing block broadcast_35
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_35 of size 8448 dropped from memory (free 558779629)
15/06/03 18:34:25 INFO BlockManager: Removing block broadcast_35_piece0
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_35_piece0 of size 5095 dropped from memory (free 558784724)
15/06/03 18:34:25 INFO BlockManager: Removing RDD 214
15/06/03 18:34:25 INFO BlockManager: Removing block rdd_214_0
15/06/03 18:34:25 INFO MemoryStore: Block rdd_214_0 of size 1664 dropped from memory (free 558786388)
15/06/03 18:34:25 INFO BlockManagerInfo: Removed broadcast_35_piece0 on localhost:60836 in memory (size: 5.0 KB, free: 533.1 MB)
15/06/03 18:34:25 INFO BlockManagerMaster: Updated info of block broadcast_35_piece0
15/06/03 18:34:25 INFO ContextCleaner: Cleaned broadcast 35
15/06/03 18:34:25 INFO ZippedPartitionsRDD2: Removing RDD 217 from persistence list
15/06/03 18:34:25 INFO BlockManager: Removing broadcast 34
15/06/03 18:34:25 INFO BlockManager: Removing block broadcast_34_piece0
15/06/03 18:34:25 INFO BlockManager: Removing RDD 217
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_34_piece0 of size 5357 dropped from memory (free 558791745)
15/06/03 18:34:25 INFO BlockManager: Removing block rdd_217_0
15/06/03 18:34:25 INFO MemoryStore: Block rdd_217_0 of size 1544 dropped from memory (free 558793289)
15/06/03 18:34:25 INFO ZippedPartitionsRDD2: Removing RDD 200 from persistence list
15/06/03 18:34:25 INFO BlockManagerInfo: Removed broadcast_34_piece0 on localhost:60836 in memory (size: 5.2 KB, free: 533.2 MB)
15/06/03 18:34:25 INFO BlockManagerMaster: Updated info of block broadcast_34_piece0
15/06/03 18:34:25 INFO BlockManager: Removing block broadcast_34
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_34 of size 9520 dropped from memory (free 558802809)
15/06/03 18:34:25 INFO BlockManager: Removing RDD 200
15/06/03 18:34:25 INFO ContextCleaner: Cleaned broadcast 34
15/06/03 18:34:25 INFO ZippedPartitionsRDD2: Removing RDD 206 from persistence list
15/06/03 18:34:25 INFO BlockManager: Removing block rdd_200_0
15/06/03 18:34:25 INFO MemoryStore: Block rdd_200_0 of size 1496 dropped from memory (free 558804305)
15/06/03 18:34:25 INFO BlockManager: Removing RDD 206
15/06/03 18:34:25 INFO BlockManager: Removing block rdd_206_0
15/06/03 18:34:25 INFO MemoryStore: Block rdd_206_0 of size 2976 dropped from memory (free 558807281)
15/06/03 18:34:25 INFO BlockManager: Removing broadcast 33
15/06/03 18:34:25 INFO BlockManager: Removing block broadcast_33_piece0
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_33_piece0 of size 5498 dropped from memory (free 558812779)
15/06/03 18:34:25 INFO BlockManagerInfo: Removed broadcast_33_piece0 on localhost:60836 in memory (size: 5.4 KB, free: 533.2 MB)
15/06/03 18:34:25 INFO BlockManagerMaster: Updated info of block broadcast_33_piece0
15/06/03 18:34:25 INFO BlockManager: Removing block broadcast_33
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_33 of size 9864 dropped from memory (free 558822643)
15/06/03 18:34:25 INFO ContextCleaner: Cleaned broadcast 33
15/06/03 18:34:25 INFO BlockManager: Removing broadcast 32
15/06/03 18:34:25 INFO BlockManager: Removing block broadcast_32
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_32 of size 9624 dropped from memory (free 558832267)
15/06/03 18:34:25 INFO BlockManager: Removing block broadcast_32_piece0
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_32_piece0 of size 5377 dropped from memory (free 558837644)
15/06/03 18:34:25 INFO BlockManagerInfo: Removed broadcast_32_piece0 on localhost:60836 in memory (size: 5.3 KB, free: 533.2 MB)
15/06/03 18:34:25 INFO BlockManagerMaster: Updated info of block broadcast_32_piece0
15/06/03 18:34:25 INFO ContextCleaner: Cleaned broadcast 32
15/06/03 18:34:25 INFO BlockManager: Removing broadcast 31
15/06/03 18:34:25 INFO BlockManager: Removing block broadcast_31
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_31 of size 8216 dropped from memory (free 558845860)
15/06/03 18:34:25 INFO BlockManager: Removing block broadcast_31_piece0
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_31_piece0 of size 4954 dropped from memory (free 558850814)
15/06/03 18:34:25 INFO BlockManagerInfo: Removed broadcast_31_piece0 on localhost:60836 in memory (size: 4.8 KB, free: 533.2 MB)
15/06/03 18:34:25 INFO BlockManagerMaster: Updated info of block broadcast_31_piece0
15/06/03 18:34:25 INFO ContextCleaner: Cleaned broadcast 31
15/06/03 18:34:25 INFO BlockManager: Removing broadcast 30
15/06/03 18:34:25 INFO BlockManager: Removing block broadcast_30_piece0
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_30_piece0 of size 5165 dropped from memory (free 558855979)
15/06/03 18:34:25 INFO BlockManagerInfo: Removed broadcast_30_piece0 on localhost:60836 in memory (size: 5.0 KB, free: 533.2 MB)
15/06/03 18:34:25 INFO BlockManagerMaster: Updated info of block broadcast_30_piece0
15/06/03 18:34:25 INFO BlockManager: Removing block broadcast_30
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_30 of size 8992 dropped from memory (free 558864971)
15/06/03 18:34:25 INFO ContextCleaner: Cleaned broadcast 30
15/06/03 18:34:25 INFO BlockManager: Removing broadcast 29
15/06/03 18:34:25 INFO BlockManager: Removing block broadcast_29_piece0
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_29_piece0 of size 5298 dropped from memory (free 558870269)
15/06/03 18:34:25 INFO BlockManagerInfo: Removed broadcast_29_piece0 on localhost:60836 in memory (size: 5.2 KB, free: 533.2 MB)
15/06/03 18:34:25 INFO BlockManagerMaster: Updated info of block broadcast_29_piece0
15/06/03 18:34:25 INFO BlockManager: Removing block broadcast_29
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_29 of size 9336 dropped from memory (free 558879605)
15/06/03 18:34:25 INFO ContextCleaner: Cleaned broadcast 29
15/06/03 18:34:25 INFO BlockManager: Removing broadcast 28
15/06/03 18:34:25 INFO BlockManager: Removing block broadcast_28
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_28 of size 9096 dropped from memory (free 558888701)
15/06/03 18:34:25 INFO BlockManager: Removing block broadcast_28_piece0
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_28_piece0 of size 5212 dropped from memory (free 558893913)
15/06/03 18:34:25 INFO BlockManagerInfo: Removed broadcast_28_piece0 on localhost:60836 in memory (size: 5.1 KB, free: 533.2 MB)
15/06/03 18:34:25 INFO BlockManagerMaster: Updated info of block broadcast_28_piece0
15/06/03 18:34:25 INFO ContextCleaner: Cleaned broadcast 28
15/06/03 18:34:25 INFO BlockManager: Removing broadcast 27
15/06/03 18:34:25 INFO BlockManager: Removing block broadcast_27
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_27 of size 7992 dropped from memory (free 558901905)
15/06/03 18:34:25 INFO BlockManager: Removing block broadcast_27_piece0
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_27_piece0 of size 4921 dropped from memory (free 558906826)
15/06/03 18:34:25 INFO BlockManagerInfo: Removed broadcast_27_piece0 on localhost:60836 in memory (size: 4.8 KB, free: 533.2 MB)
15/06/03 18:34:25 INFO BlockManagerMaster: Updated info of block broadcast_27_piece0
15/06/03 18:34:25 INFO ContextCleaner: Cleaned broadcast 27
15/06/03 18:34:25 INFO BlockManager: Removing broadcast 26
15/06/03 18:34:25 INFO BlockManager: Removing block broadcast_26
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_26 of size 8464 dropped from memory (free 558915290)
15/06/03 18:34:25 INFO BlockManager: Removing block broadcast_26_piece0
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_26_piece0 of size 4955 dropped from memory (free 558920245)
15/06/03 18:34:25 INFO BlockManagerInfo: Removed broadcast_26_piece0 on localhost:60836 in memory (size: 4.8 KB, free: 533.2 MB)
15/06/03 18:34:25 INFO BlockManagerMaster: Updated info of block broadcast_26_piece0
15/06/03 18:34:25 INFO ContextCleaner: Cleaned broadcast 26
15/06/03 18:34:25 INFO SparkContext: Starting job: reduce at VertexRDDImpl.scala:90
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 2 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 3 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 5 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 4 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 1 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 0 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 8 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 7 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 6 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 11 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 10 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 9 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 14 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 13 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 12 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 17 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 16 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 15 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 20 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 19 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 18 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 23 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 22 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 21 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 26 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 25 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 24 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 29 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 28 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 27 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 32 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 31 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 30 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 35 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 34 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 33 is 145 bytes
15/06/03 18:34:25 INFO DAGScheduler: Registering RDD 242 (mapPartitions at VertexRDDImpl.scala:218)
15/06/03 18:34:25 INFO DAGScheduler: Registering RDD 246 (mapPartitions at VertexRDDImpl.scala:222)
15/06/03 18:34:25 INFO DAGScheduler: Registering RDD 250 (mapPartitions at GraphImpl.scala:235)
15/06/03 18:34:25 INFO DAGScheduler: Got job 12 (reduce at VertexRDDImpl.scala:90) with 1 output partitions (allowLocal=false)
15/06/03 18:34:25 INFO DAGScheduler: Final stage: Stage 1362(reduce at VertexRDDImpl.scala:90)
15/06/03 18:34:25 INFO DAGScheduler: Parents of final stage: List(Stage 1337, Stage 1359, Stage 1345, Stage 1361, Stage 1353, Stage 1347, Stage 1339, Stage 1355, Stage 1341, Stage 1357, Stage 1349, Stage 1343, Stage 1336, Stage 1351)
15/06/03 18:34:25 INFO DAGScheduler: Missing parents: List(Stage 1361)
15/06/03 18:34:25 INFO DAGScheduler: Submitting Stage 1323 (ReplicatedVertexView.updateVertices - shippedVerts true false (broadcast) MapPartitionsRDD[242] at mapPartitions at VertexRDDImpl.scala:218), which has no missing parents
15/06/03 18:34:25 INFO MemoryStore: ensureFreeSpace(11976) called with curMem=232907, maxMem=559153152
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_49 stored as values in memory (estimated size 11.7 KB, free 533.0 MB)
15/06/03 18:34:25 INFO MemoryStore: ensureFreeSpace(6391) called with curMem=244883, maxMem=559153152
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_49_piece0 stored as bytes in memory (estimated size 6.2 KB, free 533.0 MB)
15/06/03 18:34:25 INFO BlockManagerInfo: Added broadcast_49_piece0 in memory on localhost:60836 (size: 6.2 KB, free: 533.2 MB)
15/06/03 18:34:25 INFO BlockManagerMaster: Updated info of block broadcast_49_piece0
15/06/03 18:34:25 INFO SparkContext: Created broadcast 49 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:25 INFO DAGScheduler: Submitting 1 missing tasks from Stage 1323 (ReplicatedVertexView.updateVertices - shippedVerts true false (broadcast) MapPartitionsRDD[242] at mapPartitions at VertexRDDImpl.scala:218)
15/06/03 18:34:25 INFO TaskSchedulerImpl: Adding task set 1323.0 with 1 tasks
15/06/03 18:34:25 INFO DAGScheduler: Submitting Stage 1360 (ReplicatedVertexView.withActiveSet - shippedActives (broadcast) MapPartitionsRDD[246] at mapPartitions at VertexRDDImpl.scala:222), which has no missing parents
15/06/03 18:34:25 INFO TaskSetManager: Starting task 0.0 in stage 1323.0 (TID 48, localhost, PROCESS_LOCAL, 2681 bytes)
15/06/03 18:34:25 INFO Executor: Running task 0.0 in stage 1323.0 (TID 48)
15/06/03 18:34:25 INFO MemoryStore: ensureFreeSpace(11632) called with curMem=251274, maxMem=559153152
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_50 stored as values in memory (estimated size 11.4 KB, free 533.0 MB)
15/06/03 18:34:25 INFO MemoryStore: ensureFreeSpace(6261) called with curMem=262906, maxMem=559153152
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_50_piece0 stored as bytes in memory (estimated size 6.1 KB, free 533.0 MB)
15/06/03 18:34:25 INFO BlockManagerInfo: Added broadcast_50_piece0 in memory on localhost:60836 (size: 6.1 KB, free: 533.2 MB)
15/06/03 18:34:25 INFO BlockManagerMaster: Updated info of block broadcast_50_piece0
15/06/03 18:34:25 INFO BlockManager: Found block rdd_219_0 locally
15/06/03 18:34:25 INFO SparkContext: Created broadcast 50 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:25 INFO CacheManager: Partition rdd_238_0 not found, computing it
15/06/03 18:34:25 INFO DAGScheduler: Submitting 1 missing tasks from Stage 1360 (ReplicatedVertexView.withActiveSet - shippedActives (broadcast) MapPartitionsRDD[246] at mapPartitions at VertexRDDImpl.scala:222)
15/06/03 18:34:25 INFO BlockManager: Found block rdd_219_0 locally
15/06/03 18:34:25 INFO TaskSchedulerImpl: Adding task set 1360.0 with 1 tasks
15/06/03 18:34:25 INFO CacheManager: Partition rdd_236_0 not found, computing it
15/06/03 18:34:25 INFO BlockManager: Found block rdd_219_0 locally
15/06/03 18:34:25 INFO BlockManager: Found block rdd_233_0 locally
15/06/03 18:34:25 INFO MemoryStore: ensureFreeSpace(1544) called with curMem=269167, maxMem=559153152
15/06/03 18:34:25 INFO MemoryStore: Block rdd_236_0 stored as values in memory (estimated size 1544.0 B, free 533.0 MB)
15/06/03 18:34:25 INFO BlockManagerInfo: Added rdd_236_0 in memory on localhost:60836 (size: 1544.0 B, free: 533.2 MB)
15/06/03 18:34:25 INFO BlockManagerMaster: Updated info of block rdd_236_0
15/06/03 18:34:25 INFO MemoryStore: ensureFreeSpace(1496) called with curMem=270711, maxMem=559153152
15/06/03 18:34:25 INFO MemoryStore: Block rdd_238_0 stored as values in memory (estimated size 1496.0 B, free 533.0 MB)
15/06/03 18:34:25 INFO BlockManagerInfo: Added rdd_238_0 in memory on localhost:60836 (size: 1496.0 B, free: 533.2 MB)
15/06/03 18:34:25 INFO BlockManagerMaster: Updated info of block rdd_238_0
15/06/03 18:34:25 INFO Executor: Finished task 0.0 in stage 1323.0 (TID 48). 2627 bytes result sent to driver
15/06/03 18:34:25 INFO TaskSetManager: Starting task 0.0 in stage 1360.0 (TID 49, localhost, PROCESS_LOCAL, 2617 bytes)
15/06/03 18:34:25 INFO Executor: Running task 0.0 in stage 1360.0 (TID 49)
15/06/03 18:34:25 INFO TaskSetManager: Finished task 0.0 in stage 1323.0 (TID 48) in 24 ms on localhost (1/1)
15/06/03 18:34:25 INFO TaskSchedulerImpl: Removed TaskSet 1323.0, whose tasks have all completed, from pool
15/06/03 18:34:25 INFO DAGScheduler: Stage 1323 (mapPartitions at VertexRDDImpl.scala:218) finished in 0.024 s
15/06/03 18:34:25 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:25 INFO DAGScheduler: running: Set(Stage 1360)
15/06/03 18:34:25 INFO DAGScheduler: waiting: Set(Stage 1362, Stage 1361)
15/06/03 18:34:25 INFO DAGScheduler: failed: Set()
15/06/03 18:34:25 INFO DAGScheduler: Missing parents for Stage 1362: List(Stage 1361)
15/06/03 18:34:25 INFO DAGScheduler: Missing parents for Stage 1361: List(Stage 1360)
15/06/03 18:34:25 INFO BlockManager: Found block rdd_236_0 locally
15/06/03 18:34:25 INFO Executor: Finished task 0.0 in stage 1360.0 (TID 49). 2003 bytes result sent to driver
15/06/03 18:34:25 INFO TaskSetManager: Finished task 0.0 in stage 1360.0 (TID 49) in 13 ms on localhost (1/1)
15/06/03 18:34:25 INFO TaskSchedulerImpl: Removed TaskSet 1360.0, whose tasks have all completed, from pool
15/06/03 18:34:25 INFO DAGScheduler: Stage 1360 (mapPartitions at VertexRDDImpl.scala:222) finished in 0.031 s
15/06/03 18:34:25 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:25 INFO DAGScheduler: running: Set()
15/06/03 18:34:25 INFO DAGScheduler: waiting: Set(Stage 1362, Stage 1361)
15/06/03 18:34:25 INFO DAGScheduler: failed: Set()
15/06/03 18:34:25 INFO DAGScheduler: Missing parents for Stage 1362: List(Stage 1361)
15/06/03 18:34:25 INFO DAGScheduler: Missing parents for Stage 1361: List()
15/06/03 18:34:25 INFO DAGScheduler: Submitting Stage 1361 (GraphImpl.aggregateMessages - preAgg MapPartitionsRDD[250] at mapPartitions at GraphImpl.scala:235), which is now runnable
15/06/03 18:34:25 INFO MemoryStore: ensureFreeSpace(9360) called with curMem=272207, maxMem=559153152
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_51 stored as values in memory (estimated size 9.1 KB, free 533.0 MB)
15/06/03 18:34:25 INFO MemoryStore: ensureFreeSpace(5412) called with curMem=281567, maxMem=559153152
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_51_piece0 stored as bytes in memory (estimated size 5.3 KB, free 533.0 MB)
15/06/03 18:34:25 INFO BlockManagerInfo: Added broadcast_51_piece0 in memory on localhost:60836 (size: 5.3 KB, free: 533.2 MB)
15/06/03 18:34:25 INFO BlockManagerMaster: Updated info of block broadcast_51_piece0
15/06/03 18:34:25 INFO SparkContext: Created broadcast 51 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:25 INFO DAGScheduler: Submitting 1 missing tasks from Stage 1361 (GraphImpl.aggregateMessages - preAgg MapPartitionsRDD[250] at mapPartitions at GraphImpl.scala:235)
15/06/03 18:34:25 INFO TaskSchedulerImpl: Adding task set 1361.0 with 1 tasks
15/06/03 18:34:25 INFO TaskSetManager: Starting task 0.0 in stage 1361.0 (TID 50, localhost, PROCESS_LOCAL, 2386 bytes)
15/06/03 18:34:25 INFO Executor: Running task 0.0 in stage 1361.0 (TID 50)
15/06/03 18:34:25 INFO CacheManager: Partition rdd_244_0 not found, computing it
15/06/03 18:34:25 INFO BlockManager: Found block rdd_225_0 locally
15/06/03 18:34:25 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:25 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:25 INFO MemoryStore: ensureFreeSpace(2976) called with curMem=286979, maxMem=559153152
15/06/03 18:34:25 INFO MemoryStore: Block rdd_244_0 stored as values in memory (estimated size 2.9 KB, free 533.0 MB)
15/06/03 18:34:25 INFO BlockManagerInfo: Added rdd_244_0 in memory on localhost:60836 (size: 2.9 KB, free: 533.2 MB)
15/06/03 18:34:25 INFO BlockManagerMaster: Updated info of block rdd_244_0
15/06/03 18:34:25 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:25 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:25 INFO Executor: Finished task 0.0 in stage 1361.0 (TID 50). 2788 bytes result sent to driver
15/06/03 18:34:25 INFO TaskSetManager: Finished task 0.0 in stage 1361.0 (TID 50) in 16 ms on localhost (1/1)
15/06/03 18:34:25 INFO TaskSchedulerImpl: Removed TaskSet 1361.0, whose tasks have all completed, from pool
15/06/03 18:34:25 INFO DAGScheduler: Stage 1361 (mapPartitions at GraphImpl.scala:235) finished in 0.016 s
15/06/03 18:34:25 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:25 INFO DAGScheduler: running: Set()
15/06/03 18:34:25 INFO DAGScheduler: waiting: Set(Stage 1362)
15/06/03 18:34:25 INFO DAGScheduler: failed: Set()
15/06/03 18:34:25 INFO DAGScheduler: Missing parents for Stage 1362: List()
15/06/03 18:34:25 INFO DAGScheduler: Submitting Stage 1362 (MapPartitionsRDD[254] at map at VertexRDDImpl.scala:90), which is now runnable
15/06/03 18:34:25 INFO MemoryStore: ensureFreeSpace(12256) called with curMem=289955, maxMem=559153152
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_52 stored as values in memory (estimated size 12.0 KB, free 533.0 MB)
15/06/03 18:34:25 INFO MemoryStore: ensureFreeSpace(6474) called with curMem=302211, maxMem=559153152
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_52_piece0 stored as bytes in memory (estimated size 6.3 KB, free 533.0 MB)
15/06/03 18:34:25 INFO BlockManagerInfo: Added broadcast_52_piece0 in memory on localhost:60836 (size: 6.3 KB, free: 533.2 MB)
15/06/03 18:34:25 INFO BlockManagerMaster: Updated info of block broadcast_52_piece0
15/06/03 18:34:25 INFO SparkContext: Created broadcast 52 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:25 INFO DAGScheduler: Submitting 1 missing tasks from Stage 1362 (MapPartitionsRDD[254] at map at VertexRDDImpl.scala:90)
15/06/03 18:34:25 INFO TaskSchedulerImpl: Adding task set 1362.0 with 1 tasks
15/06/03 18:34:25 INFO TaskSetManager: Starting task 0.0 in stage 1362.0 (TID 51, localhost, PROCESS_LOCAL, 2701 bytes)
15/06/03 18:34:25 INFO Executor: Running task 0.0 in stage 1362.0 (TID 51)
15/06/03 18:34:25 INFO CacheManager: Partition rdd_252_0 not found, computing it
15/06/03 18:34:25 INFO BlockManager: Found block rdd_238_0 locally
15/06/03 18:34:25 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:25 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:25 INFO MemoryStore: ensureFreeSpace(1664) called with curMem=308685, maxMem=559153152
15/06/03 18:34:25 INFO MemoryStore: Block rdd_252_0 stored as values in memory (estimated size 1664.0 B, free 533.0 MB)
15/06/03 18:34:25 INFO BlockManagerInfo: Added rdd_252_0 in memory on localhost:60836 (size: 1664.0 B, free: 533.2 MB)
15/06/03 18:34:25 INFO BlockManagerMaster: Updated info of block rdd_252_0
15/06/03 18:34:25 INFO Executor: Finished task 0.0 in stage 1362.0 (TID 51). 2693 bytes result sent to driver
15/06/03 18:34:25 INFO TaskSetManager: Finished task 0.0 in stage 1362.0 (TID 51) in 9 ms on localhost (1/1)
15/06/03 18:34:25 INFO TaskSchedulerImpl: Removed TaskSet 1362.0, whose tasks have all completed, from pool
15/06/03 18:34:25 INFO DAGScheduler: Stage 1362 (reduce at VertexRDDImpl.scala:90) finished in 0.009 s
15/06/03 18:34:25 INFO DAGScheduler: Job 12 finished: reduce at VertexRDDImpl.scala:90, took 0.123837 s
15/06/03 18:34:25 INFO Pregel: Pregel finished iteration 10
15/06/03 18:34:25 INFO ZippedPartitionsRDD2: Removing RDD 233 from persistence list
15/06/03 18:34:25 INFO BlockManager: Removing RDD 233
15/06/03 18:34:25 INFO BlockManager: Removing block rdd_233_0
15/06/03 18:34:25 INFO ZippedPartitionsRDD2: Removing RDD 236 from persistence list
15/06/03 18:34:25 INFO MemoryStore: Block rdd_233_0 of size 1664 dropped from memory (free 558844467)
15/06/03 18:34:25 INFO BlockManager: Removing RDD 236
15/06/03 18:34:25 INFO ZippedPartitionsRDD2: Removing RDD 219 from persistence list
15/06/03 18:34:25 INFO BlockManager: Removing block rdd_236_0
15/06/03 18:34:25 INFO MemoryStore: Block rdd_236_0 of size 1544 dropped from memory (free 558846011)
15/06/03 18:34:25 INFO BlockManager: Removing RDD 219
15/06/03 18:34:25 INFO ZippedPartitionsRDD2: Removing RDD 225 from persistence list
15/06/03 18:34:25 INFO BlockManager: Removing block rdd_219_0
15/06/03 18:34:25 INFO MemoryStore: Block rdd_219_0 of size 1496 dropped from memory (free 558847507)
15/06/03 18:34:25 INFO BlockManager: Removing RDD 225
15/06/03 18:34:25 INFO BlockManager: Removing block rdd_225_0
15/06/03 18:34:25 INFO MemoryStore: Block rdd_225_0 of size 2976 dropped from memory (free 558850483)
15/06/03 18:34:25 INFO SparkContext: Starting job: reduce at VertexRDDImpl.scala:90
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 2 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 3 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 5 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 4 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 1 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 0 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 8 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 7 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 6 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 11 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 10 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 9 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 14 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 13 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 12 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 17 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 16 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 15 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 20 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 19 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 18 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 23 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 22 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 21 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 26 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 25 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 24 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 29 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 28 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 27 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 32 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 31 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 30 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 35 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 34 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 33 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 38 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 37 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 36 is 145 bytes
15/06/03 18:34:25 INFO DAGScheduler: Registering RDD 261 (mapPartitions at VertexRDDImpl.scala:218)
15/06/03 18:34:25 INFO DAGScheduler: Registering RDD 265 (mapPartitions at VertexRDDImpl.scala:222)
15/06/03 18:34:25 INFO DAGScheduler: Registering RDD 269 (mapPartitions at GraphImpl.scala:235)
15/06/03 18:34:25 INFO DAGScheduler: Got job 13 (reduce at VertexRDDImpl.scala:90) with 1 output partitions (allowLocal=false)
15/06/03 18:34:25 INFO DAGScheduler: Final stage: Stage 1677(reduce at VertexRDDImpl.scala:90)
15/06/03 18:34:25 INFO DAGScheduler: Parents of final stage: List(Stage 1649, Stage 1664, Stage 1650, Stage 1672, Stage 1666, Stage 1658, Stage 1652, Stage 1674, Stage 1660, Stage 1668, Stage 1654, Stage 1676, Stage 1670, Stage 1662, Stage 1656)
15/06/03 18:34:25 INFO DAGScheduler: Missing parents: List(Stage 1676)
15/06/03 18:34:25 INFO DAGScheduler: Submitting Stage 1635 (ReplicatedVertexView.updateVertices - shippedVerts true false (broadcast) MapPartitionsRDD[261] at mapPartitions at VertexRDDImpl.scala:218), which has no missing parents
15/06/03 18:34:25 INFO MemoryStore: ensureFreeSpace(12496) called with curMem=302669, maxMem=559153152
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_53 stored as values in memory (estimated size 12.2 KB, free 532.9 MB)
15/06/03 18:34:25 INFO MemoryStore: ensureFreeSpace(6613) called with curMem=315165, maxMem=559153152
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_53_piece0 stored as bytes in memory (estimated size 6.5 KB, free 532.9 MB)
15/06/03 18:34:25 INFO BlockManagerInfo: Added broadcast_53_piece0 in memory on localhost:60836 (size: 6.5 KB, free: 533.2 MB)
15/06/03 18:34:25 INFO BlockManagerMaster: Updated info of block broadcast_53_piece0
15/06/03 18:34:25 INFO SparkContext: Created broadcast 53 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:25 INFO DAGScheduler: Submitting 1 missing tasks from Stage 1635 (ReplicatedVertexView.updateVertices - shippedVerts true false (broadcast) MapPartitionsRDD[261] at mapPartitions at VertexRDDImpl.scala:218)
15/06/03 18:34:25 INFO TaskSchedulerImpl: Adding task set 1635.0 with 1 tasks
15/06/03 18:34:25 INFO DAGScheduler: Submitting Stage 1675 (ReplicatedVertexView.withActiveSet - shippedActives (broadcast) MapPartitionsRDD[265] at mapPartitions at VertexRDDImpl.scala:222), which has no missing parents
15/06/03 18:34:25 INFO TaskSetManager: Starting task 0.0 in stage 1635.0 (TID 52, localhost, PROCESS_LOCAL, 2786 bytes)
15/06/03 18:34:25 INFO Executor: Running task 0.0 in stage 1635.0 (TID 52)
15/06/03 18:34:25 INFO MemoryStore: ensureFreeSpace(12160) called with curMem=321778, maxMem=559153152
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_54 stored as values in memory (estimated size 11.9 KB, free 532.9 MB)
15/06/03 18:34:25 INFO MemoryStore: ensureFreeSpace(6430) called with curMem=333938, maxMem=559153152
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_54_piece0 stored as bytes in memory (estimated size 6.3 KB, free 532.9 MB)
15/06/03 18:34:25 INFO BlockManager: Found block rdd_238_0 locally
15/06/03 18:34:25 INFO BlockManagerInfo: Added broadcast_54_piece0 in memory on localhost:60836 (size: 6.3 KB, free: 533.2 MB)
15/06/03 18:34:25 INFO BlockManagerMaster: Updated info of block broadcast_54_piece0
15/06/03 18:34:25 INFO CacheManager: Partition rdd_257_0 not found, computing it
15/06/03 18:34:25 INFO BlockManager: Found block rdd_238_0 locally
15/06/03 18:34:25 INFO CacheManager: Partition rdd_255_0 not found, computing it
15/06/03 18:34:25 INFO SparkContext: Created broadcast 54 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:25 INFO BlockManager: Found block rdd_238_0 locally
15/06/03 18:34:25 INFO BlockManager: Found block rdd_252_0 locally
15/06/03 18:34:25 INFO DAGScheduler: Submitting 1 missing tasks from Stage 1675 (ReplicatedVertexView.withActiveSet - shippedActives (broadcast) MapPartitionsRDD[265] at mapPartitions at VertexRDDImpl.scala:222)
15/06/03 18:34:25 INFO TaskSchedulerImpl: Adding task set 1675.0 with 1 tasks
15/06/03 18:34:25 INFO MemoryStore: ensureFreeSpace(1544) called with curMem=340368, maxMem=559153152
15/06/03 18:34:25 INFO MemoryStore: Block rdd_255_0 stored as values in memory (estimated size 1544.0 B, free 532.9 MB)
15/06/03 18:34:25 INFO BlockManagerInfo: Added rdd_255_0 in memory on localhost:60836 (size: 1544.0 B, free: 533.2 MB)
15/06/03 18:34:25 INFO BlockManagerMaster: Updated info of block rdd_255_0
15/06/03 18:34:25 INFO MemoryStore: ensureFreeSpace(1496) called with curMem=341912, maxMem=559153152
15/06/03 18:34:25 INFO MemoryStore: Block rdd_257_0 stored as values in memory (estimated size 1496.0 B, free 532.9 MB)
15/06/03 18:34:25 INFO BlockManagerInfo: Added rdd_257_0 in memory on localhost:60836 (size: 1496.0 B, free: 533.2 MB)
15/06/03 18:34:25 INFO BlockManagerMaster: Updated info of block rdd_257_0
15/06/03 18:34:25 INFO Executor: Finished task 0.0 in stage 1635.0 (TID 52). 2627 bytes result sent to driver
15/06/03 18:34:25 INFO TaskSetManager: Starting task 0.0 in stage 1675.0 (TID 53, localhost, PROCESS_LOCAL, 2722 bytes)
15/06/03 18:34:25 INFO Executor: Running task 0.0 in stage 1675.0 (TID 53)
15/06/03 18:34:25 INFO TaskSetManager: Finished task 0.0 in stage 1635.0 (TID 52) in 15 ms on localhost (1/1)
15/06/03 18:34:25 INFO TaskSchedulerImpl: Removed TaskSet 1635.0, whose tasks have all completed, from pool
15/06/03 18:34:25 INFO DAGScheduler: Stage 1635 (mapPartitions at VertexRDDImpl.scala:218) finished in 0.017 s
15/06/03 18:34:25 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:25 INFO DAGScheduler: running: Set(Stage 1675)
15/06/03 18:34:25 INFO DAGScheduler: waiting: Set(Stage 1676, Stage 1677)
15/06/03 18:34:25 INFO DAGScheduler: failed: Set()
15/06/03 18:34:25 INFO DAGScheduler: Missing parents for Stage 1676: List(Stage 1675)
15/06/03 18:34:25 INFO BlockManager: Found block rdd_255_0 locally
15/06/03 18:34:25 INFO DAGScheduler: Missing parents for Stage 1677: List(Stage 1676)
15/06/03 18:34:25 INFO Executor: Finished task 0.0 in stage 1675.0 (TID 53). 2003 bytes result sent to driver
15/06/03 18:34:25 INFO TaskSetManager: Finished task 0.0 in stage 1675.0 (TID 53) in 13 ms on localhost (1/1)
15/06/03 18:34:25 INFO TaskSchedulerImpl: Removed TaskSet 1675.0, whose tasks have all completed, from pool
15/06/03 18:34:25 INFO DAGScheduler: Stage 1675 (mapPartitions at VertexRDDImpl.scala:222) finished in 0.022 s
15/06/03 18:34:25 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:25 INFO DAGScheduler: running: Set()
15/06/03 18:34:25 INFO DAGScheduler: waiting: Set(Stage 1676, Stage 1677)
15/06/03 18:34:25 INFO DAGScheduler: failed: Set()
15/06/03 18:34:25 INFO DAGScheduler: Missing parents for Stage 1676: List()
15/06/03 18:34:25 INFO DAGScheduler: Missing parents for Stage 1677: List(Stage 1676)
15/06/03 18:34:25 INFO DAGScheduler: Submitting Stage 1676 (GraphImpl.aggregateMessages - preAgg MapPartitionsRDD[269] at mapPartitions at GraphImpl.scala:235), which is now runnable
15/06/03 18:34:25 INFO MemoryStore: ensureFreeSpace(9584) called with curMem=343408, maxMem=559153152
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_55 stored as values in memory (estimated size 9.4 KB, free 532.9 MB)
15/06/03 18:34:25 INFO MemoryStore: ensureFreeSpace(5619) called with curMem=352992, maxMem=559153152
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_55_piece0 stored as bytes in memory (estimated size 5.5 KB, free 532.9 MB)
15/06/03 18:34:25 INFO BlockManagerInfo: Added broadcast_55_piece0 in memory on localhost:60836 (size: 5.5 KB, free: 533.2 MB)
15/06/03 18:34:25 INFO BlockManagerMaster: Updated info of block broadcast_55_piece0
15/06/03 18:34:25 INFO SparkContext: Created broadcast 55 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:25 INFO DAGScheduler: Submitting 1 missing tasks from Stage 1676 (GraphImpl.aggregateMessages - preAgg MapPartitionsRDD[269] at mapPartitions at GraphImpl.scala:235)
15/06/03 18:34:25 INFO TaskSchedulerImpl: Adding task set 1676.0 with 1 tasks
15/06/03 18:34:25 INFO TaskSetManager: Starting task 0.0 in stage 1676.0 (TID 54, localhost, PROCESS_LOCAL, 2427 bytes)
15/06/03 18:34:25 INFO Executor: Running task 0.0 in stage 1676.0 (TID 54)
15/06/03 18:34:25 INFO CacheManager: Partition rdd_263_0 not found, computing it
15/06/03 18:34:25 INFO BlockManager: Found block rdd_244_0 locally
15/06/03 18:34:25 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:25 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:25 INFO MemoryStore: ensureFreeSpace(2976) called with curMem=358611, maxMem=559153152
15/06/03 18:34:25 INFO MemoryStore: Block rdd_263_0 stored as values in memory (estimated size 2.9 KB, free 532.9 MB)
15/06/03 18:34:25 INFO BlockManagerInfo: Added rdd_263_0 in memory on localhost:60836 (size: 2.9 KB, free: 533.1 MB)
15/06/03 18:34:25 INFO BlockManagerMaster: Updated info of block rdd_263_0
15/06/03 18:34:25 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:25 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:25 INFO Executor: Finished task 0.0 in stage 1676.0 (TID 54). 2788 bytes result sent to driver
15/06/03 18:34:25 INFO TaskSetManager: Finished task 0.0 in stage 1676.0 (TID 54) in 17 ms on localhost (1/1)
15/06/03 18:34:25 INFO TaskSchedulerImpl: Removed TaskSet 1676.0, whose tasks have all completed, from pool
15/06/03 18:34:25 INFO DAGScheduler: Stage 1676 (mapPartitions at GraphImpl.scala:235) finished in 0.018 s
15/06/03 18:34:25 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:25 INFO DAGScheduler: running: Set()
15/06/03 18:34:25 INFO DAGScheduler: waiting: Set(Stage 1677)
15/06/03 18:34:25 INFO DAGScheduler: failed: Set()
15/06/03 18:34:25 INFO DAGScheduler: Missing parents for Stage 1677: List()
15/06/03 18:34:25 INFO DAGScheduler: Submitting Stage 1677 (MapPartitionsRDD[273] at map at VertexRDDImpl.scala:90), which is now runnable
15/06/03 18:34:25 INFO MemoryStore: ensureFreeSpace(12784) called with curMem=361587, maxMem=559153152
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_56 stored as values in memory (estimated size 12.5 KB, free 532.9 MB)
15/06/03 18:34:25 INFO MemoryStore: ensureFreeSpace(6685) called with curMem=374371, maxMem=559153152
15/06/03 18:34:25 INFO MemoryStore: Block broadcast_56_piece0 stored as bytes in memory (estimated size 6.5 KB, free 532.9 MB)
15/06/03 18:34:25 INFO BlockManagerInfo: Added broadcast_56_piece0 in memory on localhost:60836 (size: 6.5 KB, free: 533.1 MB)
15/06/03 18:34:25 INFO BlockManagerMaster: Updated info of block broadcast_56_piece0
15/06/03 18:34:25 INFO SparkContext: Created broadcast 56 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:25 INFO DAGScheduler: Submitting 1 missing tasks from Stage 1677 (MapPartitionsRDD[273] at map at VertexRDDImpl.scala:90)
15/06/03 18:34:25 INFO TaskSchedulerImpl: Adding task set 1677.0 with 1 tasks
15/06/03 18:34:25 INFO TaskSetManager: Starting task 0.0 in stage 1677.0 (TID 55, localhost, PROCESS_LOCAL, 2806 bytes)
15/06/03 18:34:25 INFO Executor: Running task 0.0 in stage 1677.0 (TID 55)
15/06/03 18:34:25 INFO CacheManager: Partition rdd_271_0 not found, computing it
15/06/03 18:34:25 INFO BlockManager: Found block rdd_257_0 locally
15/06/03 18:34:25 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:25 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:25 INFO MemoryStore: ensureFreeSpace(1664) called with curMem=381056, maxMem=559153152
15/06/03 18:34:25 INFO MemoryStore: Block rdd_271_0 stored as values in memory (estimated size 1664.0 B, free 532.9 MB)
15/06/03 18:34:25 INFO BlockManagerInfo: Added rdd_271_0 in memory on localhost:60836 (size: 1664.0 B, free: 533.1 MB)
15/06/03 18:34:25 INFO BlockManagerMaster: Updated info of block rdd_271_0
15/06/03 18:34:25 INFO Executor: Finished task 0.0 in stage 1677.0 (TID 55). 2693 bytes result sent to driver
15/06/03 18:34:25 INFO TaskSetManager: Finished task 0.0 in stage 1677.0 (TID 55) in 9 ms on localhost (1/1)
15/06/03 18:34:25 INFO TaskSchedulerImpl: Removed TaskSet 1677.0, whose tasks have all completed, from pool
15/06/03 18:34:25 INFO DAGScheduler: Stage 1677 (reduce at VertexRDDImpl.scala:90) finished in 0.014 s
15/06/03 18:34:25 INFO DAGScheduler: Job 13 finished: reduce at VertexRDDImpl.scala:90, took 0.177960 s
15/06/03 18:34:25 INFO Pregel: Pregel finished iteration 11
15/06/03 18:34:25 INFO ZippedPartitionsRDD2: Removing RDD 252 from persistence list
15/06/03 18:34:25 INFO BlockManager: Removing RDD 252
15/06/03 18:34:25 INFO ZippedPartitionsRDD2: Removing RDD 255 from persistence list
15/06/03 18:34:25 INFO BlockManager: Removing block rdd_252_0
15/06/03 18:34:25 INFO MemoryStore: Block rdd_252_0 of size 1664 dropped from memory (free 558772096)
15/06/03 18:34:25 INFO BlockManager: Removing RDD 255
15/06/03 18:34:25 INFO BlockManager: Removing block rdd_255_0
15/06/03 18:34:25 INFO MemoryStore: Block rdd_255_0 of size 1544 dropped from memory (free 558773640)
15/06/03 18:34:25 INFO ZippedPartitionsRDD2: Removing RDD 238 from persistence list
15/06/03 18:34:25 INFO BlockManager: Removing RDD 238
15/06/03 18:34:25 INFO BlockManager: Removing block rdd_238_0
15/06/03 18:34:25 INFO ZippedPartitionsRDD2: Removing RDD 244 from persistence list
15/06/03 18:34:25 INFO MemoryStore: Block rdd_238_0 of size 1496 dropped from memory (free 558775136)
15/06/03 18:34:25 INFO BlockManager: Removing RDD 244
15/06/03 18:34:25 INFO BlockManager: Removing block rdd_244_0
15/06/03 18:34:25 INFO MemoryStore: Block rdd_244_0 of size 2976 dropped from memory (free 558778112)
15/06/03 18:34:25 INFO SparkContext: Starting job: reduce at VertexRDDImpl.scala:90
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 2 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 3 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 5 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 4 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 1 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 0 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 8 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 7 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 6 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 11 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 10 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 9 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 14 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 13 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 12 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 17 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 16 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 15 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 20 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 19 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 18 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 23 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 22 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 21 is 145 bytes
15/06/03 18:34:25 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 26 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 25 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 24 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 29 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 28 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 27 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 32 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 31 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 30 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 35 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 34 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 33 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 38 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 37 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 36 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 41 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 40 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 39 is 145 bytes
15/06/03 18:34:26 INFO DAGScheduler: Registering RDD 280 (mapPartitions at VertexRDDImpl.scala:218)
15/06/03 18:34:26 INFO DAGScheduler: Registering RDD 284 (mapPartitions at VertexRDDImpl.scala:222)
15/06/03 18:34:26 INFO DAGScheduler: Registering RDD 288 (mapPartitions at GraphImpl.scala:235)
15/06/03 18:34:26 INFO DAGScheduler: Got job 14 (reduce at VertexRDDImpl.scala:90) with 1 output partitions (allowLocal=false)
15/06/03 18:34:26 INFO DAGScheduler: Final stage: Stage 2037(reduce at VertexRDDImpl.scala:90)
15/06/03 18:34:26 INFO DAGScheduler: Parents of final stage: List(Stage 2034, Stage 2020, Stage 2028, Stage 2007, Stage 2014, Stage 2036, Stage 2030, Stage 2022, Stage 2016, Stage 2008, Stage 2024, Stage 2010, Stage 2032, Stage 2026, Stage 2018, Stage 2012)
15/06/03 18:34:26 INFO DAGScheduler: Missing parents: List(Stage 2036)
15/06/03 18:34:26 INFO DAGScheduler: Submitting Stage 1992 (ReplicatedVertexView.updateVertices - shippedVerts true false (broadcast) MapPartitionsRDD[280] at mapPartitions at VertexRDDImpl.scala:218), which has no missing parents
15/06/03 18:34:26 INFO MemoryStore: ensureFreeSpace(13024) called with curMem=375040, maxMem=559153152
15/06/03 18:34:26 INFO MemoryStore: Block broadcast_57 stored as values in memory (estimated size 12.7 KB, free 532.9 MB)
15/06/03 18:34:26 INFO MemoryStore: ensureFreeSpace(6784) called with curMem=388064, maxMem=559153152
15/06/03 18:34:26 INFO MemoryStore: Block broadcast_57_piece0 stored as bytes in memory (estimated size 6.6 KB, free 532.9 MB)
15/06/03 18:34:26 INFO BlockManagerInfo: Added broadcast_57_piece0 in memory on localhost:60836 (size: 6.6 KB, free: 533.1 MB)
15/06/03 18:34:26 INFO BlockManagerMaster: Updated info of block broadcast_57_piece0
15/06/03 18:34:26 INFO SparkContext: Created broadcast 57 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:26 INFO DAGScheduler: Submitting 1 missing tasks from Stage 1992 (ReplicatedVertexView.updateVertices - shippedVerts true false (broadcast) MapPartitionsRDD[280] at mapPartitions at VertexRDDImpl.scala:218)
15/06/03 18:34:26 INFO TaskSchedulerImpl: Adding task set 1992.0 with 1 tasks
15/06/03 18:34:26 INFO DAGScheduler: Submitting Stage 2035 (ReplicatedVertexView.withActiveSet - shippedActives (broadcast) MapPartitionsRDD[284] at mapPartitions at VertexRDDImpl.scala:222), which has no missing parents
15/06/03 18:34:26 INFO TaskSetManager: Starting task 0.0 in stage 1992.0 (TID 56, localhost, PROCESS_LOCAL, 2891 bytes)
15/06/03 18:34:26 INFO Executor: Running task 0.0 in stage 1992.0 (TID 56)
15/06/03 18:34:26 INFO MemoryStore: ensureFreeSpace(12680) called with curMem=394848, maxMem=559153152
15/06/03 18:34:26 INFO MemoryStore: Block broadcast_58 stored as values in memory (estimated size 12.4 KB, free 532.9 MB)
15/06/03 18:34:26 INFO MemoryStore: ensureFreeSpace(6645) called with curMem=407528, maxMem=559153152
15/06/03 18:34:26 INFO BlockManager: Found block rdd_257_0 locally
15/06/03 18:34:26 INFO MemoryStore: Block broadcast_58_piece0 stored as bytes in memory (estimated size 6.5 KB, free 532.9 MB)
15/06/03 18:34:26 INFO CacheManager: Partition rdd_276_0 not found, computing it
15/06/03 18:34:26 INFO BlockManager: Found block rdd_257_0 locally
15/06/03 18:34:26 INFO CacheManager: Partition rdd_274_0 not found, computing it
15/06/03 18:34:26 INFO BlockManager: Found block rdd_257_0 locally
15/06/03 18:34:26 INFO BlockManagerInfo: Added broadcast_58_piece0 in memory on localhost:60836 (size: 6.5 KB, free: 533.1 MB)
15/06/03 18:34:26 INFO BlockManager: Found block rdd_271_0 locally
15/06/03 18:34:26 INFO BlockManagerMaster: Updated info of block broadcast_58_piece0
15/06/03 18:34:26 INFO SparkContext: Created broadcast 58 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:26 INFO DAGScheduler: Submitting 1 missing tasks from Stage 2035 (ReplicatedVertexView.withActiveSet - shippedActives (broadcast) MapPartitionsRDD[284] at mapPartitions at VertexRDDImpl.scala:222)
15/06/03 18:34:26 INFO TaskSchedulerImpl: Adding task set 2035.0 with 1 tasks
15/06/03 18:34:26 INFO MemoryStore: ensureFreeSpace(1544) called with curMem=414173, maxMem=559153152
15/06/03 18:34:26 INFO MemoryStore: Block rdd_274_0 stored as values in memory (estimated size 1544.0 B, free 532.9 MB)
15/06/03 18:34:26 INFO BlockManagerInfo: Added rdd_274_0 in memory on localhost:60836 (size: 1544.0 B, free: 533.1 MB)
15/06/03 18:34:26 INFO BlockManagerMaster: Updated info of block rdd_274_0
15/06/03 18:34:26 INFO MemoryStore: ensureFreeSpace(1496) called with curMem=415717, maxMem=559153152
15/06/03 18:34:26 INFO MemoryStore: Block rdd_276_0 stored as values in memory (estimated size 1496.0 B, free 532.9 MB)
15/06/03 18:34:26 INFO BlockManagerInfo: Added rdd_276_0 in memory on localhost:60836 (size: 1496.0 B, free: 533.1 MB)
15/06/03 18:34:26 INFO BlockManagerMaster: Updated info of block rdd_276_0
15/06/03 18:34:26 INFO Executor: Finished task 0.0 in stage 1992.0 (TID 56). 2627 bytes result sent to driver
15/06/03 18:34:26 INFO TaskSetManager: Starting task 0.0 in stage 2035.0 (TID 57, localhost, PROCESS_LOCAL, 2827 bytes)
15/06/03 18:34:26 INFO TaskSetManager: Finished task 0.0 in stage 1992.0 (TID 56) in 29 ms on localhost (1/1)
15/06/03 18:34:26 INFO TaskSchedulerImpl: Removed TaskSet 1992.0, whose tasks have all completed, from pool
15/06/03 18:34:26 INFO Executor: Running task 0.0 in stage 2035.0 (TID 57)
15/06/03 18:34:26 INFO DAGScheduler: Stage 1992 (mapPartitions at VertexRDDImpl.scala:218) finished in 0.029 s
15/06/03 18:34:26 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:26 INFO DAGScheduler: running: Set(Stage 2035)
15/06/03 18:34:26 INFO DAGScheduler: waiting: Set(Stage 2037, Stage 2036)
15/06/03 18:34:26 INFO DAGScheduler: failed: Set()
15/06/03 18:34:26 INFO DAGScheduler: Missing parents for Stage 2037: List(Stage 2036)
15/06/03 18:34:26 INFO DAGScheduler: Missing parents for Stage 2036: List(Stage 2035)
15/06/03 18:34:26 INFO BlockManager: Found block rdd_274_0 locally
15/06/03 18:34:26 INFO Executor: Finished task 0.0 in stage 2035.0 (TID 57). 2003 bytes result sent to driver
15/06/03 18:34:26 INFO TaskSetManager: Finished task 0.0 in stage 2035.0 (TID 57) in 16 ms on localhost (1/1)
15/06/03 18:34:26 INFO TaskSchedulerImpl: Removed TaskSet 2035.0, whose tasks have all completed, from pool
15/06/03 18:34:26 INFO DAGScheduler: Stage 2035 (mapPartitions at VertexRDDImpl.scala:222) finished in 0.036 s
15/06/03 18:34:26 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:26 INFO DAGScheduler: running: Set()
15/06/03 18:34:26 INFO DAGScheduler: waiting: Set(Stage 2037, Stage 2036)
15/06/03 18:34:26 INFO DAGScheduler: failed: Set()
15/06/03 18:34:26 INFO DAGScheduler: Missing parents for Stage 2037: List(Stage 2036)
15/06/03 18:34:26 INFO DAGScheduler: Missing parents for Stage 2036: List()
15/06/03 18:34:26 INFO DAGScheduler: Submitting Stage 2036 (GraphImpl.aggregateMessages - preAgg MapPartitionsRDD[288] at mapPartitions at GraphImpl.scala:235), which is now runnable
15/06/03 18:34:26 INFO MemoryStore: ensureFreeSpace(9816) called with curMem=417213, maxMem=559153152
15/06/03 18:34:26 INFO MemoryStore: Block broadcast_59 stored as values in memory (estimated size 9.6 KB, free 532.8 MB)
15/06/03 18:34:26 INFO MemoryStore: ensureFreeSpace(5675) called with curMem=427029, maxMem=559153152
15/06/03 18:34:26 INFO MemoryStore: Block broadcast_59_piece0 stored as bytes in memory (estimated size 5.5 KB, free 532.8 MB)
15/06/03 18:34:26 INFO BlockManagerInfo: Added broadcast_59_piece0 in memory on localhost:60836 (size: 5.5 KB, free: 533.1 MB)
15/06/03 18:34:26 INFO BlockManagerMaster: Updated info of block broadcast_59_piece0
15/06/03 18:34:26 INFO SparkContext: Created broadcast 59 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:26 INFO DAGScheduler: Submitting 1 missing tasks from Stage 2036 (GraphImpl.aggregateMessages - preAgg MapPartitionsRDD[288] at mapPartitions at GraphImpl.scala:235)
15/06/03 18:34:26 INFO TaskSchedulerImpl: Adding task set 2036.0 with 1 tasks
15/06/03 18:34:26 INFO TaskSetManager: Starting task 0.0 in stage 2036.0 (TID 58, localhost, PROCESS_LOCAL, 2468 bytes)
15/06/03 18:34:26 INFO Executor: Running task 0.0 in stage 2036.0 (TID 58)
15/06/03 18:34:26 INFO CacheManager: Partition rdd_282_0 not found, computing it
15/06/03 18:34:26 INFO BlockManager: Found block rdd_263_0 locally
15/06/03 18:34:26 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:26 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:26 INFO MemoryStore: ensureFreeSpace(2976) called with curMem=432704, maxMem=559153152
15/06/03 18:34:26 INFO MemoryStore: Block rdd_282_0 stored as values in memory (estimated size 2.9 KB, free 532.8 MB)
15/06/03 18:34:26 INFO BlockManagerInfo: Added rdd_282_0 in memory on localhost:60836 (size: 2.9 KB, free: 533.1 MB)
15/06/03 18:34:26 INFO BlockManagerMaster: Updated info of block rdd_282_0
15/06/03 18:34:26 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:26 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:26 INFO Executor: Finished task 0.0 in stage 2036.0 (TID 58). 2788 bytes result sent to driver
15/06/03 18:34:26 INFO TaskSetManager: Finished task 0.0 in stage 2036.0 (TID 58) in 14 ms on localhost (1/1)
15/06/03 18:34:26 INFO TaskSchedulerImpl: Removed TaskSet 2036.0, whose tasks have all completed, from pool
15/06/03 18:34:26 INFO DAGScheduler: Stage 2036 (mapPartitions at GraphImpl.scala:235) finished in 0.014 s
15/06/03 18:34:26 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:26 INFO DAGScheduler: running: Set()
15/06/03 18:34:26 INFO DAGScheduler: waiting: Set(Stage 2037)
15/06/03 18:34:26 INFO DAGScheduler: failed: Set()
15/06/03 18:34:26 INFO DAGScheduler: Missing parents for Stage 2037: List()
15/06/03 18:34:26 INFO DAGScheduler: Submitting Stage 2037 (MapPartitionsRDD[292] at map at VertexRDDImpl.scala:90), which is now runnable
15/06/03 18:34:26 INFO MemoryStore: ensureFreeSpace(13312) called with curMem=435680, maxMem=559153152
15/06/03 18:34:26 INFO MemoryStore: Block broadcast_60 stored as values in memory (estimated size 13.0 KB, free 532.8 MB)
15/06/03 18:34:26 INFO MemoryStore: ensureFreeSpace(6881) called with curMem=448992, maxMem=559153152
15/06/03 18:34:26 INFO MemoryStore: Block broadcast_60_piece0 stored as bytes in memory (estimated size 6.7 KB, free 532.8 MB)
15/06/03 18:34:26 INFO BlockManagerInfo: Added broadcast_60_piece0 in memory on localhost:60836 (size: 6.7 KB, free: 533.1 MB)
15/06/03 18:34:26 INFO BlockManagerMaster: Updated info of block broadcast_60_piece0
15/06/03 18:34:26 INFO SparkContext: Created broadcast 60 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:26 INFO DAGScheduler: Submitting 1 missing tasks from Stage 2037 (MapPartitionsRDD[292] at map at VertexRDDImpl.scala:90)
15/06/03 18:34:26 INFO TaskSchedulerImpl: Adding task set 2037.0 with 1 tasks
15/06/03 18:34:26 INFO TaskSetManager: Starting task 0.0 in stage 2037.0 (TID 59, localhost, PROCESS_LOCAL, 2911 bytes)
15/06/03 18:34:26 INFO Executor: Running task 0.0 in stage 2037.0 (TID 59)
15/06/03 18:34:26 INFO CacheManager: Partition rdd_290_0 not found, computing it
15/06/03 18:34:26 INFO BlockManager: Found block rdd_276_0 locally
15/06/03 18:34:26 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:26 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:26 INFO MemoryStore: ensureFreeSpace(1664) called with curMem=455873, maxMem=559153152
15/06/03 18:34:26 INFO MemoryStore: Block rdd_290_0 stored as values in memory (estimated size 1664.0 B, free 532.8 MB)
15/06/03 18:34:26 INFO BlockManagerInfo: Added rdd_290_0 in memory on localhost:60836 (size: 1664.0 B, free: 533.1 MB)
15/06/03 18:34:26 INFO BlockManagerMaster: Updated info of block rdd_290_0
15/06/03 18:34:26 INFO Executor: Finished task 0.0 in stage 2037.0 (TID 59). 2693 bytes result sent to driver
15/06/03 18:34:26 INFO TaskSetManager: Finished task 0.0 in stage 2037.0 (TID 59) in 7 ms on localhost (1/1)
15/06/03 18:34:26 INFO TaskSchedulerImpl: Removed TaskSet 2037.0, whose tasks have all completed, from pool
15/06/03 18:34:26 INFO DAGScheduler: Stage 2037 (reduce at VertexRDDImpl.scala:90) finished in 0.008 s
15/06/03 18:34:26 INFO DAGScheduler: Job 14 finished: reduce at VertexRDDImpl.scala:90, took 0.166727 s
15/06/03 18:34:26 INFO Pregel: Pregel finished iteration 12
15/06/03 18:34:26 INFO ZippedPartitionsRDD2: Removing RDD 271 from persistence list
15/06/03 18:34:26 INFO BlockManager: Removing RDD 271
15/06/03 18:34:26 INFO ZippedPartitionsRDD2: Removing RDD 274 from persistence list
15/06/03 18:34:26 INFO BlockManager: Removing block rdd_271_0
15/06/03 18:34:26 INFO MemoryStore: Block rdd_271_0 of size 1664 dropped from memory (free 558697279)
15/06/03 18:34:26 INFO BlockManager: Removing RDD 274
15/06/03 18:34:26 INFO BlockManager: Removing block rdd_274_0
15/06/03 18:34:26 INFO ZippedPartitionsRDD2: Removing RDD 257 from persistence list
15/06/03 18:34:26 INFO MemoryStore: Block rdd_274_0 of size 1544 dropped from memory (free 558698823)
15/06/03 18:34:26 INFO BlockManager: Removing RDD 257
15/06/03 18:34:26 INFO ZippedPartitionsRDD2: Removing RDD 263 from persistence list
15/06/03 18:34:26 INFO BlockManager: Removing block rdd_257_0
15/06/03 18:34:26 INFO MemoryStore: Block rdd_257_0 of size 1496 dropped from memory (free 558700319)
15/06/03 18:34:26 INFO BlockManager: Removing RDD 263
15/06/03 18:34:26 INFO BlockManager: Removing block rdd_263_0
15/06/03 18:34:26 INFO MemoryStore: Block rdd_263_0 of size 2976 dropped from memory (free 558703295)
15/06/03 18:34:26 INFO SparkContext: Starting job: reduce at VertexRDDImpl.scala:90
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 2 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 3 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 5 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 4 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 1 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 0 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 8 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 7 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 6 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 11 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 10 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 9 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 14 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 13 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 12 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 17 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 16 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 15 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 20 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 19 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 18 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 23 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 22 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 21 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 26 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 25 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 24 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 29 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 28 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 27 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 32 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 31 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 30 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 35 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 34 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 33 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 38 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 37 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 36 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 41 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 40 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 39 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 44 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 43 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 42 is 145 bytes
15/06/03 18:34:26 INFO DAGScheduler: Registering RDD 299 (mapPartitions at VertexRDDImpl.scala:218)
15/06/03 18:34:26 INFO DAGScheduler: Registering RDD 303 (mapPartitions at VertexRDDImpl.scala:222)
15/06/03 18:34:26 INFO DAGScheduler: Registering RDD 307 (mapPartitions at GraphImpl.scala:235)
15/06/03 18:34:26 INFO DAGScheduler: Got job 15 (reduce at VertexRDDImpl.scala:90) with 1 output partitions (allowLocal=false)
15/06/03 18:34:26 INFO DAGScheduler: Final stage: Stage 2445(reduce at VertexRDDImpl.scala:90)
15/06/03 18:34:26 INFO DAGScheduler: Parents of final stage: List(Stage 2440, Stage 2426, Stage 2434, Stage 2413, Stage 2420, Stage 2414, Stage 2442, Stage 2436, Stage 2428, Stage 2422, Stage 2444, Stage 2430, Stage 2416, Stage 2438, Stage 2432, Stage 2424, Stage 2418)
15/06/03 18:34:26 INFO DAGScheduler: Missing parents: List(Stage 2444)
15/06/03 18:34:26 INFO DAGScheduler: Submitting Stage 2397 (ReplicatedVertexView.updateVertices - shippedVerts true false (broadcast) MapPartitionsRDD[299] at mapPartitions at VertexRDDImpl.scala:218), which has no missing parents
15/06/03 18:34:26 INFO MemoryStore: ensureFreeSpace(13552) called with curMem=449857, maxMem=559153152
15/06/03 18:34:26 INFO MemoryStore: Block broadcast_61 stored as values in memory (estimated size 13.2 KB, free 532.8 MB)
15/06/03 18:34:26 INFO MemoryStore: ensureFreeSpace(7039) called with curMem=463409, maxMem=559153152
15/06/03 18:34:26 INFO MemoryStore: Block broadcast_61_piece0 stored as bytes in memory (estimated size 6.9 KB, free 532.8 MB)
15/06/03 18:34:26 INFO BlockManagerInfo: Added broadcast_61_piece0 in memory on localhost:60836 (size: 6.9 KB, free: 533.1 MB)
15/06/03 18:34:26 INFO BlockManagerMaster: Updated info of block broadcast_61_piece0
15/06/03 18:34:26 INFO SparkContext: Created broadcast 61 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:26 INFO DAGScheduler: Submitting 1 missing tasks from Stage 2397 (ReplicatedVertexView.updateVertices - shippedVerts true false (broadcast) MapPartitionsRDD[299] at mapPartitions at VertexRDDImpl.scala:218)
15/06/03 18:34:26 INFO TaskSchedulerImpl: Adding task set 2397.0 with 1 tasks
15/06/03 18:34:26 INFO DAGScheduler: Submitting Stage 2443 (ReplicatedVertexView.withActiveSet - shippedActives (broadcast) MapPartitionsRDD[303] at mapPartitions at VertexRDDImpl.scala:222), which has no missing parents
15/06/03 18:34:26 INFO TaskSetManager: Starting task 0.0 in stage 2397.0 (TID 60, localhost, PROCESS_LOCAL, 2996 bytes)
15/06/03 18:34:26 INFO Executor: Running task 0.0 in stage 2397.0 (TID 60)
15/06/03 18:34:26 INFO MemoryStore: ensureFreeSpace(13208) called with curMem=470448, maxMem=559153152
15/06/03 18:34:26 INFO MemoryStore: Block broadcast_62 stored as values in memory (estimated size 12.9 KB, free 532.8 MB)
15/06/03 18:34:26 INFO MemoryStore: ensureFreeSpace(6841) called with curMem=483656, maxMem=559153152
15/06/03 18:34:26 INFO MemoryStore: Block broadcast_62_piece0 stored as bytes in memory (estimated size 6.7 KB, free 532.8 MB)
15/06/03 18:34:26 INFO BlockManager: Found block rdd_276_0 locally
15/06/03 18:34:26 INFO CacheManager: Partition rdd_295_0 not found, computing it
15/06/03 18:34:26 INFO BlockManagerInfo: Added broadcast_62_piece0 in memory on localhost:60836 (size: 6.7 KB, free: 533.1 MB)
15/06/03 18:34:26 INFO BlockManager: Found block rdd_276_0 locally
15/06/03 18:34:26 INFO BlockManagerMaster: Updated info of block broadcast_62_piece0
15/06/03 18:34:26 INFO CacheManager: Partition rdd_293_0 not found, computing it
15/06/03 18:34:26 INFO BlockManager: Found block rdd_276_0 locally
15/06/03 18:34:26 INFO BlockManager: Found block rdd_290_0 locally
15/06/03 18:34:26 INFO SparkContext: Created broadcast 62 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:26 INFO DAGScheduler: Submitting 1 missing tasks from Stage 2443 (ReplicatedVertexView.withActiveSet - shippedActives (broadcast) MapPartitionsRDD[303] at mapPartitions at VertexRDDImpl.scala:222)
15/06/03 18:34:26 INFO TaskSchedulerImpl: Adding task set 2443.0 with 1 tasks
15/06/03 18:34:26 INFO MemoryStore: ensureFreeSpace(1544) called with curMem=490497, maxMem=559153152
15/06/03 18:34:26 INFO MemoryStore: Block rdd_293_0 stored as values in memory (estimated size 1544.0 B, free 532.8 MB)
15/06/03 18:34:26 INFO BlockManagerInfo: Added rdd_293_0 in memory on localhost:60836 (size: 1544.0 B, free: 533.1 MB)
15/06/03 18:34:26 INFO BlockManagerMaster: Updated info of block rdd_293_0
15/06/03 18:34:26 INFO MemoryStore: ensureFreeSpace(1496) called with curMem=492041, maxMem=559153152
15/06/03 18:34:26 INFO MemoryStore: Block rdd_295_0 stored as values in memory (estimated size 1496.0 B, free 532.8 MB)
15/06/03 18:34:26 INFO BlockManagerInfo: Added rdd_295_0 in memory on localhost:60836 (size: 1496.0 B, free: 533.1 MB)
15/06/03 18:34:26 INFO BlockManagerMaster: Updated info of block rdd_295_0
15/06/03 18:34:26 INFO Executor: Finished task 0.0 in stage 2397.0 (TID 60). 2627 bytes result sent to driver
15/06/03 18:34:26 INFO TaskSetManager: Starting task 0.0 in stage 2443.0 (TID 61, localhost, PROCESS_LOCAL, 2932 bytes)
15/06/03 18:34:26 INFO Executor: Running task 0.0 in stage 2443.0 (TID 61)
15/06/03 18:34:26 INFO TaskSetManager: Finished task 0.0 in stage 2397.0 (TID 60) in 24 ms on localhost (1/1)
15/06/03 18:34:26 INFO TaskSchedulerImpl: Removed TaskSet 2397.0, whose tasks have all completed, from pool
15/06/03 18:34:26 INFO DAGScheduler: Stage 2397 (mapPartitions at VertexRDDImpl.scala:218) finished in 0.025 s
15/06/03 18:34:26 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:26 INFO DAGScheduler: running: Set(Stage 2443)
15/06/03 18:34:26 INFO DAGScheduler: waiting: Set(Stage 2444, Stage 2445)
15/06/03 18:34:26 INFO DAGScheduler: failed: Set()
15/06/03 18:34:26 INFO DAGScheduler: Missing parents for Stage 2444: List(Stage 2443)
15/06/03 18:34:26 INFO DAGScheduler: Missing parents for Stage 2445: List(Stage 2444)
15/06/03 18:34:26 INFO BlockManager: Found block rdd_293_0 locally
15/06/03 18:34:26 INFO Executor: Finished task 0.0 in stage 2443.0 (TID 61). 2003 bytes result sent to driver
15/06/03 18:34:26 INFO TaskSetManager: Finished task 0.0 in stage 2443.0 (TID 61) in 13 ms on localhost (1/1)
15/06/03 18:34:26 INFO TaskSchedulerImpl: Removed TaskSet 2443.0, whose tasks have all completed, from pool
15/06/03 18:34:26 INFO DAGScheduler: Stage 2443 (mapPartitions at VertexRDDImpl.scala:222) finished in 0.032 s
15/06/03 18:34:26 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:26 INFO DAGScheduler: running: Set()
15/06/03 18:34:26 INFO DAGScheduler: waiting: Set(Stage 2444, Stage 2445)
15/06/03 18:34:26 INFO DAGScheduler: failed: Set()
15/06/03 18:34:26 INFO DAGScheduler: Missing parents for Stage 2444: List()
15/06/03 18:34:26 INFO DAGScheduler: Missing parents for Stage 2445: List(Stage 2444)
15/06/03 18:34:26 INFO DAGScheduler: Submitting Stage 2444 (GraphImpl.aggregateMessages - preAgg MapPartitionsRDD[307] at mapPartitions at GraphImpl.scala:235), which is now runnable
15/06/03 18:34:26 INFO MemoryStore: ensureFreeSpace(10040) called with curMem=493537, maxMem=559153152
15/06/03 18:34:26 INFO MemoryStore: Block broadcast_63 stored as values in memory (estimated size 9.8 KB, free 532.8 MB)
15/06/03 18:34:26 INFO MemoryStore: ensureFreeSpace(5780) called with curMem=503577, maxMem=559153152
15/06/03 18:34:26 INFO MemoryStore: Block broadcast_63_piece0 stored as bytes in memory (estimated size 5.6 KB, free 532.8 MB)
15/06/03 18:34:26 INFO BlockManagerInfo: Added broadcast_63_piece0 in memory on localhost:60836 (size: 5.6 KB, free: 533.1 MB)
15/06/03 18:34:26 INFO BlockManagerMaster: Updated info of block broadcast_63_piece0
15/06/03 18:34:26 INFO SparkContext: Created broadcast 63 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:26 INFO DAGScheduler: Submitting 1 missing tasks from Stage 2444 (GraphImpl.aggregateMessages - preAgg MapPartitionsRDD[307] at mapPartitions at GraphImpl.scala:235)
15/06/03 18:34:26 INFO TaskSchedulerImpl: Adding task set 2444.0 with 1 tasks
15/06/03 18:34:26 INFO TaskSetManager: Starting task 0.0 in stage 2444.0 (TID 62, localhost, PROCESS_LOCAL, 2509 bytes)
15/06/03 18:34:26 INFO Executor: Running task 0.0 in stage 2444.0 (TID 62)
15/06/03 18:34:26 INFO CacheManager: Partition rdd_301_0 not found, computing it
15/06/03 18:34:26 INFO BlockManager: Found block rdd_282_0 locally
15/06/03 18:34:26 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:26 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:26 INFO MemoryStore: ensureFreeSpace(2976) called with curMem=509357, maxMem=559153152
15/06/03 18:34:26 INFO MemoryStore: Block rdd_301_0 stored as values in memory (estimated size 2.9 KB, free 532.8 MB)
15/06/03 18:34:26 INFO BlockManagerInfo: Added rdd_301_0 in memory on localhost:60836 (size: 2.9 KB, free: 533.1 MB)
15/06/03 18:34:26 INFO BlockManagerMaster: Updated info of block rdd_301_0
15/06/03 18:34:26 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:26 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:26 INFO Executor: Finished task 0.0 in stage 2444.0 (TID 62). 2788 bytes result sent to driver
15/06/03 18:34:26 INFO TaskSetManager: Finished task 0.0 in stage 2444.0 (TID 62) in 17 ms on localhost (1/1)
15/06/03 18:34:26 INFO TaskSchedulerImpl: Removed TaskSet 2444.0, whose tasks have all completed, from pool
15/06/03 18:34:26 INFO DAGScheduler: Stage 2444 (mapPartitions at GraphImpl.scala:235) finished in 0.017 s
15/06/03 18:34:26 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:26 INFO DAGScheduler: running: Set()
15/06/03 18:34:26 INFO DAGScheduler: waiting: Set(Stage 2445)
15/06/03 18:34:26 INFO DAGScheduler: failed: Set()
15/06/03 18:34:26 INFO DAGScheduler: Missing parents for Stage 2445: List()
15/06/03 18:34:26 INFO DAGScheduler: Submitting Stage 2445 (MapPartitionsRDD[311] at map at VertexRDDImpl.scala:90), which is now runnable
15/06/03 18:34:26 INFO MemoryStore: ensureFreeSpace(13840) called with curMem=512333, maxMem=559153152
15/06/03 18:34:26 INFO MemoryStore: Block broadcast_64 stored as values in memory (estimated size 13.5 KB, free 532.7 MB)
15/06/03 18:34:26 INFO MemoryStore: ensureFreeSpace(7080) called with curMem=526173, maxMem=559153152
15/06/03 18:34:26 INFO MemoryStore: Block broadcast_64_piece0 stored as bytes in memory (estimated size 6.9 KB, free 532.7 MB)
15/06/03 18:34:26 INFO BlockManagerInfo: Added broadcast_64_piece0 in memory on localhost:60836 (size: 6.9 KB, free: 533.1 MB)
15/06/03 18:34:26 INFO BlockManagerMaster: Updated info of block broadcast_64_piece0
15/06/03 18:34:26 INFO SparkContext: Created broadcast 64 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:26 INFO DAGScheduler: Submitting 1 missing tasks from Stage 2445 (MapPartitionsRDD[311] at map at VertexRDDImpl.scala:90)
15/06/03 18:34:26 INFO TaskSchedulerImpl: Adding task set 2445.0 with 1 tasks
15/06/03 18:34:26 INFO TaskSetManager: Starting task 0.0 in stage 2445.0 (TID 63, localhost, PROCESS_LOCAL, 3016 bytes)
15/06/03 18:34:26 INFO Executor: Running task 0.0 in stage 2445.0 (TID 63)
15/06/03 18:34:26 INFO CacheManager: Partition rdd_309_0 not found, computing it
15/06/03 18:34:26 INFO BlockManager: Found block rdd_295_0 locally
15/06/03 18:34:26 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:26 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:26 INFO MemoryStore: ensureFreeSpace(1664) called with curMem=533253, maxMem=559153152
15/06/03 18:34:26 INFO MemoryStore: Block rdd_309_0 stored as values in memory (estimated size 1664.0 B, free 532.7 MB)
15/06/03 18:34:26 INFO BlockManagerInfo: Added rdd_309_0 in memory on localhost:60836 (size: 1664.0 B, free: 533.1 MB)
15/06/03 18:34:26 INFO BlockManagerMaster: Updated info of block rdd_309_0
15/06/03 18:34:26 INFO Executor: Finished task 0.0 in stage 2445.0 (TID 63). 2693 bytes result sent to driver
15/06/03 18:34:26 INFO TaskSetManager: Finished task 0.0 in stage 2445.0 (TID 63) in 10 ms on localhost (1/1)
15/06/03 18:34:26 INFO TaskSchedulerImpl: Removed TaskSet 2445.0, whose tasks have all completed, from pool
15/06/03 18:34:26 INFO DAGScheduler: Stage 2445 (reduce at VertexRDDImpl.scala:90) finished in 0.010 s
15/06/03 18:34:26 INFO DAGScheduler: Job 15 finished: reduce at VertexRDDImpl.scala:90, took 0.145516 s
15/06/03 18:34:26 INFO Pregel: Pregel finished iteration 13
15/06/03 18:34:26 INFO ZippedPartitionsRDD2: Removing RDD 290 from persistence list
15/06/03 18:34:26 INFO BlockManager: Removing RDD 290
15/06/03 18:34:26 INFO ZippedPartitionsRDD2: Removing RDD 293 from persistence list
15/06/03 18:34:26 INFO BlockManager: Removing block rdd_290_0
15/06/03 18:34:26 INFO MemoryStore: Block rdd_290_0 of size 1664 dropped from memory (free 558619899)
15/06/03 18:34:26 INFO BlockManager: Removing RDD 293
15/06/03 18:34:26 INFO ZippedPartitionsRDD2: Removing RDD 276 from persistence list
15/06/03 18:34:26 INFO BlockManager: Removing block rdd_293_0
15/06/03 18:34:26 INFO MemoryStore: Block rdd_293_0 of size 1544 dropped from memory (free 558621443)
15/06/03 18:34:26 INFO BlockManager: Removing RDD 276
15/06/03 18:34:26 INFO BlockManager: Removing block rdd_276_0
15/06/03 18:34:26 INFO MemoryStore: Block rdd_276_0 of size 1496 dropped from memory (free 558622939)
15/06/03 18:34:26 INFO ZippedPartitionsRDD2: Removing RDD 282 from persistence list
15/06/03 18:34:26 INFO BlockManager: Removing RDD 282
15/06/03 18:34:26 INFO BlockManager: Removing block rdd_282_0
15/06/03 18:34:26 INFO MemoryStore: Block rdd_282_0 of size 2976 dropped from memory (free 558625915)
15/06/03 18:34:26 INFO SparkContext: Starting job: reduce at VertexRDDImpl.scala:90
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 2 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 3 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 5 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 4 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 1 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 0 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 8 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 7 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 6 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 11 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 10 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 9 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 14 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 13 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 12 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 17 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 16 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 15 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 20 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 19 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 18 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 23 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 22 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 21 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 26 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 25 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 24 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 29 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 28 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 27 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 32 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 31 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 30 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 35 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 34 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 33 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 38 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 37 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 36 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 41 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 40 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 39 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 44 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 43 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 42 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 47 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 46 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 45 is 145 bytes
15/06/03 18:34:26 INFO DAGScheduler: Registering RDD 318 (mapPartitions at VertexRDDImpl.scala:218)
15/06/03 18:34:26 INFO DAGScheduler: Registering RDD 322 (mapPartitions at VertexRDDImpl.scala:222)
15/06/03 18:34:26 INFO DAGScheduler: Registering RDD 326 (mapPartitions at GraphImpl.scala:235)
15/06/03 18:34:26 INFO DAGScheduler: Got job 16 (reduce at VertexRDDImpl.scala:90) with 1 output partitions (allowLocal=false)
15/06/03 18:34:26 INFO DAGScheduler: Final stage: Stage 2904(reduce at VertexRDDImpl.scala:90)
15/06/03 18:34:26 INFO DAGScheduler: Parents of final stage: List(Stage 2883, Stage 2891, Stage 2870, Stage 2877, Stage 2871, Stage 2899, Stage 2893, Stage 2885, Stage 2879, Stage 2901, Stage 2887, Stage 2873, Stage 2903, Stage 2895, Stage 2889, Stage 2881, Stage 2875, Stage 2897)
15/06/03 18:34:26 INFO DAGScheduler: Missing parents: List(Stage 2903)
15/06/03 18:34:26 INFO DAGScheduler: Submitting Stage 2853 (ReplicatedVertexView.updateVertices - shippedVerts true false (broadcast) MapPartitionsRDD[318] at mapPartitions at VertexRDDImpl.scala:218), which has no missing parents
15/06/03 18:34:26 INFO MemoryStore: ensureFreeSpace(14080) called with curMem=527237, maxMem=559153152
15/06/03 18:34:26 INFO MemoryStore: Block broadcast_65 stored as values in memory (estimated size 13.8 KB, free 532.7 MB)
15/06/03 18:34:26 INFO MemoryStore: ensureFreeSpace(7192) called with curMem=541317, maxMem=559153152
15/06/03 18:34:26 INFO MemoryStore: Block broadcast_65_piece0 stored as bytes in memory (estimated size 7.0 KB, free 532.7 MB)
15/06/03 18:34:26 INFO BlockManagerInfo: Added broadcast_65_piece0 in memory on localhost:60836 (size: 7.0 KB, free: 533.1 MB)
15/06/03 18:34:26 INFO BlockManagerMaster: Updated info of block broadcast_65_piece0
15/06/03 18:34:26 INFO SparkContext: Created broadcast 65 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:26 INFO DAGScheduler: Submitting 1 missing tasks from Stage 2853 (ReplicatedVertexView.updateVertices - shippedVerts true false (broadcast) MapPartitionsRDD[318] at mapPartitions at VertexRDDImpl.scala:218)
15/06/03 18:34:26 INFO TaskSchedulerImpl: Adding task set 2853.0 with 1 tasks
15/06/03 18:34:26 INFO DAGScheduler: Submitting Stage 2902 (ReplicatedVertexView.withActiveSet - shippedActives (broadcast) MapPartitionsRDD[322] at mapPartitions at VertexRDDImpl.scala:222), which has no missing parents
15/06/03 18:34:26 INFO TaskSetManager: Starting task 0.0 in stage 2853.0 (TID 64, localhost, PROCESS_LOCAL, 3101 bytes)
15/06/03 18:34:26 INFO Executor: Running task 0.0 in stage 2853.0 (TID 64)
15/06/03 18:34:26 INFO MemoryStore: ensureFreeSpace(13736) called with curMem=548509, maxMem=559153152
15/06/03 18:34:26 INFO MemoryStore: Block broadcast_66 stored as values in memory (estimated size 13.4 KB, free 532.7 MB)
15/06/03 18:34:26 INFO MemoryStore: ensureFreeSpace(7047) called with curMem=562245, maxMem=559153152
15/06/03 18:34:26 INFO BlockManager: Found block rdd_295_0 locally
15/06/03 18:34:26 INFO MemoryStore: Block broadcast_66_piece0 stored as bytes in memory (estimated size 6.9 KB, free 532.7 MB)
15/06/03 18:34:26 INFO CacheManager: Partition rdd_314_0 not found, computing it
15/06/03 18:34:26 INFO BlockManager: Found block rdd_295_0 locally
15/06/03 18:34:26 INFO BlockManagerInfo: Added broadcast_66_piece0 in memory on localhost:60836 (size: 6.9 KB, free: 533.1 MB)
15/06/03 18:34:26 INFO BlockManagerMaster: Updated info of block broadcast_66_piece0
15/06/03 18:34:26 INFO CacheManager: Partition rdd_312_0 not found, computing it
15/06/03 18:34:26 INFO BlockManager: Found block rdd_295_0 locally
15/06/03 18:34:26 INFO SparkContext: Created broadcast 66 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:26 INFO BlockManager: Found block rdd_309_0 locally
15/06/03 18:34:26 INFO DAGScheduler: Submitting 1 missing tasks from Stage 2902 (ReplicatedVertexView.withActiveSet - shippedActives (broadcast) MapPartitionsRDD[322] at mapPartitions at VertexRDDImpl.scala:222)
15/06/03 18:34:26 INFO TaskSchedulerImpl: Adding task set 2902.0 with 1 tasks
15/06/03 18:34:26 INFO MemoryStore: ensureFreeSpace(1544) called with curMem=569292, maxMem=559153152
15/06/03 18:34:26 INFO MemoryStore: Block rdd_312_0 stored as values in memory (estimated size 1544.0 B, free 532.7 MB)
15/06/03 18:34:26 INFO BlockManagerInfo: Added rdd_312_0 in memory on localhost:60836 (size: 1544.0 B, free: 533.1 MB)
15/06/03 18:34:26 INFO BlockManagerMaster: Updated info of block rdd_312_0
15/06/03 18:34:26 INFO MemoryStore: ensureFreeSpace(1496) called with curMem=570836, maxMem=559153152
15/06/03 18:34:26 INFO MemoryStore: Block rdd_314_0 stored as values in memory (estimated size 1496.0 B, free 532.7 MB)
15/06/03 18:34:26 INFO BlockManagerInfo: Added rdd_314_0 in memory on localhost:60836 (size: 1496.0 B, free: 533.1 MB)
15/06/03 18:34:26 INFO BlockManagerMaster: Updated info of block rdd_314_0
15/06/03 18:34:26 INFO Executor: Finished task 0.0 in stage 2853.0 (TID 64). 2627 bytes result sent to driver
15/06/03 18:34:26 INFO TaskSetManager: Starting task 0.0 in stage 2902.0 (TID 65, localhost, PROCESS_LOCAL, 3037 bytes)
15/06/03 18:34:26 INFO Executor: Running task 0.0 in stage 2902.0 (TID 65)
15/06/03 18:34:26 INFO TaskSetManager: Finished task 0.0 in stage 2853.0 (TID 64) in 19 ms on localhost (1/1)
15/06/03 18:34:26 INFO TaskSchedulerImpl: Removed TaskSet 2853.0, whose tasks have all completed, from pool
15/06/03 18:34:26 INFO DAGScheduler: Stage 2853 (mapPartitions at VertexRDDImpl.scala:218) finished in 0.021 s
15/06/03 18:34:26 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:26 INFO DAGScheduler: running: Set(Stage 2902)
15/06/03 18:34:26 INFO DAGScheduler: waiting: Set(Stage 2904, Stage 2903)
15/06/03 18:34:26 INFO DAGScheduler: failed: Set()
15/06/03 18:34:26 INFO DAGScheduler: Missing parents for Stage 2904: List(Stage 2903)
15/06/03 18:34:26 INFO BlockManager: Found block rdd_312_0 locally
15/06/03 18:34:26 INFO DAGScheduler: Missing parents for Stage 2903: List(Stage 2902)
15/06/03 18:34:26 INFO Executor: Finished task 0.0 in stage 2902.0 (TID 65). 2003 bytes result sent to driver
15/06/03 18:34:26 INFO TaskSetManager: Finished task 0.0 in stage 2902.0 (TID 65) in 15 ms on localhost (1/1)
15/06/03 18:34:26 INFO TaskSchedulerImpl: Removed TaskSet 2902.0, whose tasks have all completed, from pool
15/06/03 18:34:26 INFO DAGScheduler: Stage 2902 (mapPartitions at VertexRDDImpl.scala:222) finished in 0.025 s
15/06/03 18:34:26 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:26 INFO DAGScheduler: running: Set()
15/06/03 18:34:26 INFO DAGScheduler: waiting: Set(Stage 2904, Stage 2903)
15/06/03 18:34:26 INFO DAGScheduler: failed: Set()
15/06/03 18:34:26 INFO DAGScheduler: Missing parents for Stage 2904: List(Stage 2903)
15/06/03 18:34:26 INFO DAGScheduler: Missing parents for Stage 2903: List()
15/06/03 18:34:26 INFO DAGScheduler: Submitting Stage 2903 (GraphImpl.aggregateMessages - preAgg MapPartitionsRDD[326] at mapPartitions at GraphImpl.scala:235), which is now runnable
15/06/03 18:34:26 INFO MemoryStore: ensureFreeSpace(10272) called with curMem=572332, maxMem=559153152
15/06/03 18:34:26 INFO MemoryStore: Block broadcast_67 stored as values in memory (estimated size 10.0 KB, free 532.7 MB)
15/06/03 18:34:26 INFO MemoryStore: ensureFreeSpace(5866) called with curMem=582604, maxMem=559153152
15/06/03 18:34:26 INFO MemoryStore: Block broadcast_67_piece0 stored as bytes in memory (estimated size 5.7 KB, free 532.7 MB)
15/06/03 18:34:26 INFO BlockManagerInfo: Added broadcast_67_piece0 in memory on localhost:60836 (size: 5.7 KB, free: 533.1 MB)
15/06/03 18:34:26 INFO BlockManagerMaster: Updated info of block broadcast_67_piece0
15/06/03 18:34:26 INFO SparkContext: Created broadcast 67 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:26 INFO DAGScheduler: Submitting 1 missing tasks from Stage 2903 (GraphImpl.aggregateMessages - preAgg MapPartitionsRDD[326] at mapPartitions at GraphImpl.scala:235)
15/06/03 18:34:26 INFO TaskSchedulerImpl: Adding task set 2903.0 with 1 tasks
15/06/03 18:34:26 INFO TaskSetManager: Starting task 0.0 in stage 2903.0 (TID 66, localhost, PROCESS_LOCAL, 2550 bytes)
15/06/03 18:34:26 INFO Executor: Running task 0.0 in stage 2903.0 (TID 66)
15/06/03 18:34:26 INFO CacheManager: Partition rdd_320_0 not found, computing it
15/06/03 18:34:26 INFO BlockManager: Found block rdd_301_0 locally
15/06/03 18:34:26 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:26 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:26 INFO MemoryStore: ensureFreeSpace(2976) called with curMem=588470, maxMem=559153152
15/06/03 18:34:26 INFO MemoryStore: Block rdd_320_0 stored as values in memory (estimated size 2.9 KB, free 532.7 MB)
15/06/03 18:34:26 INFO BlockManagerInfo: Added rdd_320_0 in memory on localhost:60836 (size: 2.9 KB, free: 533.1 MB)
15/06/03 18:34:26 INFO BlockManagerMaster: Updated info of block rdd_320_0
15/06/03 18:34:26 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:26 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:26 INFO Executor: Finished task 0.0 in stage 2903.0 (TID 66). 2788 bytes result sent to driver
15/06/03 18:34:26 INFO TaskSetManager: Finished task 0.0 in stage 2903.0 (TID 66) in 16 ms on localhost (1/1)
15/06/03 18:34:26 INFO TaskSchedulerImpl: Removed TaskSet 2903.0, whose tasks have all completed, from pool
15/06/03 18:34:26 INFO DAGScheduler: Stage 2903 (mapPartitions at GraphImpl.scala:235) finished in 0.016 s
15/06/03 18:34:26 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:26 INFO DAGScheduler: running: Set()
15/06/03 18:34:26 INFO DAGScheduler: waiting: Set(Stage 2904)
15/06/03 18:34:26 INFO DAGScheduler: failed: Set()
15/06/03 18:34:26 INFO DAGScheduler: Missing parents for Stage 2904: List()
15/06/03 18:34:26 INFO DAGScheduler: Submitting Stage 2904 (MapPartitionsRDD[330] at map at VertexRDDImpl.scala:90), which is now runnable
15/06/03 18:34:26 INFO MemoryStore: ensureFreeSpace(14368) called with curMem=591446, maxMem=559153152
15/06/03 18:34:26 INFO MemoryStore: Block broadcast_68 stored as values in memory (estimated size 14.0 KB, free 532.7 MB)
15/06/03 18:34:26 INFO MemoryStore: ensureFreeSpace(7243) called with curMem=605814, maxMem=559153152
15/06/03 18:34:26 INFO MemoryStore: Block broadcast_68_piece0 stored as bytes in memory (estimated size 7.1 KB, free 532.7 MB)
15/06/03 18:34:26 INFO BlockManagerInfo: Added broadcast_68_piece0 in memory on localhost:60836 (size: 7.1 KB, free: 533.1 MB)
15/06/03 18:34:26 INFO BlockManagerMaster: Updated info of block broadcast_68_piece0
15/06/03 18:34:26 INFO SparkContext: Created broadcast 68 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:26 INFO DAGScheduler: Submitting 1 missing tasks from Stage 2904 (MapPartitionsRDD[330] at map at VertexRDDImpl.scala:90)
15/06/03 18:34:26 INFO TaskSchedulerImpl: Adding task set 2904.0 with 1 tasks
15/06/03 18:34:26 INFO TaskSetManager: Starting task 0.0 in stage 2904.0 (TID 67, localhost, PROCESS_LOCAL, 3121 bytes)
15/06/03 18:34:26 INFO Executor: Running task 0.0 in stage 2904.0 (TID 67)
15/06/03 18:34:26 INFO CacheManager: Partition rdd_328_0 not found, computing it
15/06/03 18:34:26 INFO BlockManager: Found block rdd_314_0 locally
15/06/03 18:34:26 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:26 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:26 INFO MemoryStore: ensureFreeSpace(1664) called with curMem=613057, maxMem=559153152
15/06/03 18:34:26 INFO MemoryStore: Block rdd_328_0 stored as values in memory (estimated size 1664.0 B, free 532.7 MB)
15/06/03 18:34:26 INFO BlockManagerInfo: Added rdd_328_0 in memory on localhost:60836 (size: 1664.0 B, free: 533.1 MB)
15/06/03 18:34:26 INFO BlockManagerMaster: Updated info of block rdd_328_0
15/06/03 18:34:26 INFO Executor: Finished task 0.0 in stage 2904.0 (TID 67). 2693 bytes result sent to driver
15/06/03 18:34:26 INFO TaskSetManager: Finished task 0.0 in stage 2904.0 (TID 67) in 7 ms on localhost (1/1)
15/06/03 18:34:26 INFO TaskSchedulerImpl: Removed TaskSet 2904.0, whose tasks have all completed, from pool
15/06/03 18:34:26 INFO DAGScheduler: Stage 2904 (reduce at VertexRDDImpl.scala:90) finished in 0.007 s
15/06/03 18:34:26 INFO DAGScheduler: Job 16 finished: reduce at VertexRDDImpl.scala:90, took 0.149059 s
15/06/03 18:34:26 INFO Pregel: Pregel finished iteration 14
15/06/03 18:34:26 INFO ZippedPartitionsRDD2: Removing RDD 309 from persistence list
15/06/03 18:34:26 INFO BlockManager: Removing RDD 309
15/06/03 18:34:26 INFO BlockManager: Removing block rdd_309_0
15/06/03 18:34:26 INFO MemoryStore: Block rdd_309_0 of size 1664 dropped from memory (free 558540095)
15/06/03 18:34:26 INFO ZippedPartitionsRDD2: Removing RDD 312 from persistence list
15/06/03 18:34:26 INFO BlockManager: Removing RDD 312
15/06/03 18:34:26 INFO BlockManager: Removing block rdd_312_0
15/06/03 18:34:26 INFO MemoryStore: Block rdd_312_0 of size 1544 dropped from memory (free 558541639)
15/06/03 18:34:26 INFO ZippedPartitionsRDD2: Removing RDD 295 from persistence list
15/06/03 18:34:26 INFO BlockManager: Removing RDD 295
15/06/03 18:34:26 INFO BlockManager: Removing block rdd_295_0
15/06/03 18:34:26 INFO MemoryStore: Block rdd_295_0 of size 1496 dropped from memory (free 558543135)
15/06/03 18:34:26 INFO ZippedPartitionsRDD2: Removing RDD 301 from persistence list
15/06/03 18:34:26 INFO BlockManager: Removing RDD 301
15/06/03 18:34:26 INFO BlockManager: Removing block rdd_301_0
15/06/03 18:34:26 INFO MemoryStore: Block rdd_301_0 of size 2976 dropped from memory (free 558546111)
15/06/03 18:34:26 INFO SparkContext: Starting job: reduce at VertexRDDImpl.scala:90
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 2 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 3 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 5 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 4 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 1 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 0 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 8 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 7 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 6 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 11 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 10 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 9 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 14 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 13 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 12 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 17 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 16 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 15 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 20 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 19 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 18 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 23 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 22 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 21 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 26 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 25 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 24 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 29 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 28 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 27 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 32 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 31 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 30 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 35 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 34 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 33 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 38 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 37 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 36 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 41 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 40 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 39 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 44 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 43 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 42 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 47 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 46 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 45 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 50 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 49 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 48 is 145 bytes
15/06/03 18:34:26 INFO DAGScheduler: Registering RDD 337 (mapPartitions at VertexRDDImpl.scala:218)
15/06/03 18:34:26 INFO DAGScheduler: Registering RDD 341 (mapPartitions at VertexRDDImpl.scala:222)
15/06/03 18:34:26 INFO DAGScheduler: Registering RDD 345 (mapPartitions at GraphImpl.scala:235)
15/06/03 18:34:26 INFO DAGScheduler: Got job 17 (reduce at VertexRDDImpl.scala:90) with 1 output partitions (allowLocal=false)
15/06/03 18:34:26 INFO DAGScheduler: Final stage: Stage 3417(reduce at VertexRDDImpl.scala:90)
15/06/03 18:34:26 INFO DAGScheduler: Parents of final stage: List(Stage 3406, Stage 3398, Stage 3392, Stage 3384, Stage 3414, Stage 3400, Stage 3386, Stage 3408, Stage 3394, Stage 3388, Stage 3416, Stage 3410, Stage 3402, Stage 3381, Stage 3396, Stage 3382, Stage 3404, Stage 3390, Stage 3412)
15/06/03 18:34:26 INFO DAGScheduler: Missing parents: List(Stage 3416)
15/06/03 18:34:26 INFO DAGScheduler: Submitting Stage 3363 (ReplicatedVertexView.updateVertices - shippedVerts true false (broadcast) MapPartitionsRDD[337] at mapPartitions at VertexRDDImpl.scala:218), which has no missing parents
15/06/03 18:34:26 INFO MemoryStore: ensureFreeSpace(14608) called with curMem=607041, maxMem=559153152
15/06/03 18:34:26 INFO MemoryStore: Block broadcast_69 stored as values in memory (estimated size 14.3 KB, free 532.7 MB)
15/06/03 18:34:26 INFO MemoryStore: ensureFreeSpace(7400) called with curMem=621649, maxMem=559153152
15/06/03 18:34:26 INFO MemoryStore: Block broadcast_69_piece0 stored as bytes in memory (estimated size 7.2 KB, free 532.7 MB)
15/06/03 18:34:26 INFO BlockManagerInfo: Added broadcast_69_piece0 in memory on localhost:60836 (size: 7.2 KB, free: 533.1 MB)
15/06/03 18:34:26 INFO BlockManagerMaster: Updated info of block broadcast_69_piece0
15/06/03 18:34:26 INFO SparkContext: Created broadcast 69 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:26 INFO DAGScheduler: Submitting 1 missing tasks from Stage 3363 (ReplicatedVertexView.updateVertices - shippedVerts true false (broadcast) MapPartitionsRDD[337] at mapPartitions at VertexRDDImpl.scala:218)
15/06/03 18:34:26 INFO TaskSchedulerImpl: Adding task set 3363.0 with 1 tasks
15/06/03 18:34:26 INFO DAGScheduler: Submitting Stage 3415 (ReplicatedVertexView.withActiveSet - shippedActives (broadcast) MapPartitionsRDD[341] at mapPartitions at VertexRDDImpl.scala:222), which has no missing parents
15/06/03 18:34:26 INFO TaskSetManager: Starting task 0.0 in stage 3363.0 (TID 68, localhost, PROCESS_LOCAL, 3206 bytes)
15/06/03 18:34:26 INFO Executor: Running task 0.0 in stage 3363.0 (TID 68)
15/06/03 18:34:26 INFO MemoryStore: ensureFreeSpace(14264) called with curMem=629049, maxMem=559153152
15/06/03 18:34:26 INFO MemoryStore: Block broadcast_70 stored as values in memory (estimated size 13.9 KB, free 532.6 MB)
15/06/03 18:34:26 INFO MemoryStore: ensureFreeSpace(7203) called with curMem=643313, maxMem=559153152
15/06/03 18:34:26 INFO MemoryStore: Block broadcast_70_piece0 stored as bytes in memory (estimated size 7.0 KB, free 532.6 MB)
15/06/03 18:34:26 INFO BlockManagerInfo: Added broadcast_70_piece0 in memory on localhost:60836 (size: 7.0 KB, free: 533.1 MB)
15/06/03 18:34:26 INFO BlockManagerMaster: Updated info of block broadcast_70_piece0
15/06/03 18:34:26 INFO SparkContext: Created broadcast 70 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:26 INFO BlockManager: Found block rdd_314_0 locally
15/06/03 18:34:26 INFO DAGScheduler: Submitting 1 missing tasks from Stage 3415 (ReplicatedVertexView.withActiveSet - shippedActives (broadcast) MapPartitionsRDD[341] at mapPartitions at VertexRDDImpl.scala:222)
15/06/03 18:34:26 INFO TaskSchedulerImpl: Adding task set 3415.0 with 1 tasks
15/06/03 18:34:26 INFO CacheManager: Partition rdd_333_0 not found, computing it
15/06/03 18:34:26 INFO BlockManager: Found block rdd_314_0 locally
15/06/03 18:34:26 INFO CacheManager: Partition rdd_331_0 not found, computing it
15/06/03 18:34:26 INFO BlockManager: Found block rdd_314_0 locally
15/06/03 18:34:26 INFO BlockManager: Found block rdd_328_0 locally
15/06/03 18:34:26 INFO MemoryStore: ensureFreeSpace(1544) called with curMem=650516, maxMem=559153152
15/06/03 18:34:26 INFO MemoryStore: Block rdd_331_0 stored as values in memory (estimated size 1544.0 B, free 532.6 MB)
15/06/03 18:34:26 INFO BlockManagerInfo: Added rdd_331_0 in memory on localhost:60836 (size: 1544.0 B, free: 533.1 MB)
15/06/03 18:34:26 INFO BlockManagerMaster: Updated info of block rdd_331_0
15/06/03 18:34:26 INFO MemoryStore: ensureFreeSpace(1496) called with curMem=652060, maxMem=559153152
15/06/03 18:34:26 INFO MemoryStore: Block rdd_333_0 stored as values in memory (estimated size 1496.0 B, free 532.6 MB)
15/06/03 18:34:26 INFO BlockManagerInfo: Added rdd_333_0 in memory on localhost:60836 (size: 1496.0 B, free: 533.1 MB)
15/06/03 18:34:26 INFO BlockManagerMaster: Updated info of block rdd_333_0
15/06/03 18:34:26 INFO Executor: Finished task 0.0 in stage 3363.0 (TID 68). 2627 bytes result sent to driver
15/06/03 18:34:26 INFO TaskSetManager: Starting task 0.0 in stage 3415.0 (TID 69, localhost, PROCESS_LOCAL, 3142 bytes)
15/06/03 18:34:26 INFO TaskSetManager: Finished task 0.0 in stage 3363.0 (TID 68) in 21 ms on localhost (1/1)
15/06/03 18:34:26 INFO Executor: Running task 0.0 in stage 3415.0 (TID 69)
15/06/03 18:34:26 INFO TaskSchedulerImpl: Removed TaskSet 3363.0, whose tasks have all completed, from pool
15/06/03 18:34:26 INFO DAGScheduler: Stage 3363 (mapPartitions at VertexRDDImpl.scala:218) finished in 0.022 s
15/06/03 18:34:26 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:26 INFO DAGScheduler: running: Set(Stage 3415)
15/06/03 18:34:26 INFO DAGScheduler: waiting: Set(Stage 3416, Stage 3417)
15/06/03 18:34:26 INFO DAGScheduler: failed: Set()
15/06/03 18:34:26 INFO BlockManager: Found block rdd_331_0 locally
15/06/03 18:34:26 INFO DAGScheduler: Missing parents for Stage 3416: List(Stage 3415)
15/06/03 18:34:26 INFO DAGScheduler: Missing parents for Stage 3417: List(Stage 3416)
15/06/03 18:34:26 INFO Executor: Finished task 0.0 in stage 3415.0 (TID 69). 2003 bytes result sent to driver
15/06/03 18:34:26 INFO TaskSetManager: Finished task 0.0 in stage 3415.0 (TID 69) in 14 ms on localhost (1/1)
15/06/03 18:34:26 INFO TaskSchedulerImpl: Removed TaskSet 3415.0, whose tasks have all completed, from pool
15/06/03 18:34:26 INFO DAGScheduler: Stage 3415 (mapPartitions at VertexRDDImpl.scala:222) finished in 0.027 s
15/06/03 18:34:26 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:26 INFO DAGScheduler: running: Set()
15/06/03 18:34:26 INFO DAGScheduler: waiting: Set(Stage 3416, Stage 3417)
15/06/03 18:34:26 INFO DAGScheduler: failed: Set()
15/06/03 18:34:26 INFO DAGScheduler: Missing parents for Stage 3416: List()
15/06/03 18:34:26 INFO DAGScheduler: Missing parents for Stage 3417: List(Stage 3416)
15/06/03 18:34:26 INFO DAGScheduler: Submitting Stage 3416 (GraphImpl.aggregateMessages - preAgg MapPartitionsRDD[345] at mapPartitions at GraphImpl.scala:235), which is now runnable
15/06/03 18:34:26 INFO MemoryStore: ensureFreeSpace(10496) called with curMem=653556, maxMem=559153152
15/06/03 18:34:26 INFO MemoryStore: Block broadcast_71 stored as values in memory (estimated size 10.3 KB, free 532.6 MB)
15/06/03 18:34:26 INFO MemoryStore: ensureFreeSpace(5974) called with curMem=664052, maxMem=559153152
15/06/03 18:34:26 INFO MemoryStore: Block broadcast_71_piece0 stored as bytes in memory (estimated size 5.8 KB, free 532.6 MB)
15/06/03 18:34:26 INFO BlockManagerInfo: Added broadcast_71_piece0 in memory on localhost:60836 (size: 5.8 KB, free: 533.1 MB)
15/06/03 18:34:26 INFO BlockManagerMaster: Updated info of block broadcast_71_piece0
15/06/03 18:34:26 INFO SparkContext: Created broadcast 71 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:26 INFO DAGScheduler: Submitting 1 missing tasks from Stage 3416 (GraphImpl.aggregateMessages - preAgg MapPartitionsRDD[345] at mapPartitions at GraphImpl.scala:235)
15/06/03 18:34:26 INFO TaskSchedulerImpl: Adding task set 3416.0 with 1 tasks
15/06/03 18:34:26 INFO TaskSetManager: Starting task 0.0 in stage 3416.0 (TID 70, localhost, PROCESS_LOCAL, 2591 bytes)
15/06/03 18:34:26 INFO Executor: Running task 0.0 in stage 3416.0 (TID 70)
15/06/03 18:34:26 INFO CacheManager: Partition rdd_339_0 not found, computing it
15/06/03 18:34:26 INFO BlockManager: Found block rdd_320_0 locally
15/06/03 18:34:26 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:26 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:26 INFO MemoryStore: ensureFreeSpace(2976) called with curMem=670026, maxMem=559153152
15/06/03 18:34:26 INFO MemoryStore: Block rdd_339_0 stored as values in memory (estimated size 2.9 KB, free 532.6 MB)
15/06/03 18:34:26 INFO BlockManagerInfo: Added rdd_339_0 in memory on localhost:60836 (size: 2.9 KB, free: 533.0 MB)
15/06/03 18:34:26 INFO BlockManagerMaster: Updated info of block rdd_339_0
15/06/03 18:34:26 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:26 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:26 INFO Executor: Finished task 0.0 in stage 3416.0 (TID 70). 2788 bytes result sent to driver
15/06/03 18:34:26 INFO TaskSetManager: Finished task 0.0 in stage 3416.0 (TID 70) in 15 ms on localhost (1/1)
15/06/03 18:34:26 INFO TaskSchedulerImpl: Removed TaskSet 3416.0, whose tasks have all completed, from pool
15/06/03 18:34:26 INFO DAGScheduler: Stage 3416 (mapPartitions at GraphImpl.scala:235) finished in 0.015 s
15/06/03 18:34:26 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:26 INFO DAGScheduler: running: Set()
15/06/03 18:34:26 INFO DAGScheduler: waiting: Set(Stage 3417)
15/06/03 18:34:26 INFO DAGScheduler: failed: Set()
15/06/03 18:34:26 INFO DAGScheduler: Missing parents for Stage 3417: List()
15/06/03 18:34:26 INFO DAGScheduler: Submitting Stage 3417 (MapPartitionsRDD[349] at map at VertexRDDImpl.scala:90), which is now runnable
15/06/03 18:34:26 INFO MemoryStore: ensureFreeSpace(14896) called with curMem=673002, maxMem=559153152
15/06/03 18:34:26 INFO MemoryStore: Block broadcast_72 stored as values in memory (estimated size 14.5 KB, free 532.6 MB)
15/06/03 18:34:26 INFO MemoryStore: ensureFreeSpace(7452) called with curMem=687898, maxMem=559153152
15/06/03 18:34:26 INFO MemoryStore: Block broadcast_72_piece0 stored as bytes in memory (estimated size 7.3 KB, free 532.6 MB)
15/06/03 18:34:26 INFO BlockManagerInfo: Added broadcast_72_piece0 in memory on localhost:60836 (size: 7.3 KB, free: 533.0 MB)
15/06/03 18:34:26 INFO BlockManagerMaster: Updated info of block broadcast_72_piece0
15/06/03 18:34:26 INFO SparkContext: Created broadcast 72 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:26 INFO DAGScheduler: Submitting 1 missing tasks from Stage 3417 (MapPartitionsRDD[349] at map at VertexRDDImpl.scala:90)
15/06/03 18:34:26 INFO TaskSchedulerImpl: Adding task set 3417.0 with 1 tasks
15/06/03 18:34:26 INFO TaskSetManager: Starting task 0.0 in stage 3417.0 (TID 71, localhost, PROCESS_LOCAL, 3226 bytes)
15/06/03 18:34:26 INFO Executor: Running task 0.0 in stage 3417.0 (TID 71)
15/06/03 18:34:26 INFO CacheManager: Partition rdd_347_0 not found, computing it
15/06/03 18:34:26 INFO BlockManager: Found block rdd_333_0 locally
15/06/03 18:34:26 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:26 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:26 INFO MemoryStore: ensureFreeSpace(1664) called with curMem=695350, maxMem=559153152
15/06/03 18:34:26 INFO MemoryStore: Block rdd_347_0 stored as values in memory (estimated size 1664.0 B, free 532.6 MB)
15/06/03 18:34:26 INFO BlockManagerInfo: Added rdd_347_0 in memory on localhost:60836 (size: 1664.0 B, free: 533.0 MB)
15/06/03 18:34:26 INFO BlockManagerMaster: Updated info of block rdd_347_0
15/06/03 18:34:26 INFO Executor: Finished task 0.0 in stage 3417.0 (TID 71). 2693 bytes result sent to driver
15/06/03 18:34:26 INFO TaskSetManager: Finished task 0.0 in stage 3417.0 (TID 71) in 15 ms on localhost (1/1)
15/06/03 18:34:26 INFO TaskSchedulerImpl: Removed TaskSet 3417.0, whose tasks have all completed, from pool
15/06/03 18:34:26 INFO DAGScheduler: Stage 3417 (reduce at VertexRDDImpl.scala:90) finished in 0.015 s
15/06/03 18:34:26 INFO DAGScheduler: Job 17 finished: reduce at VertexRDDImpl.scala:90, took 0.201433 s
15/06/03 18:34:26 INFO Pregel: Pregel finished iteration 15
15/06/03 18:34:26 INFO ZippedPartitionsRDD2: Removing RDD 328 from persistence list
15/06/03 18:34:26 INFO BlockManager: Removing RDD 328
15/06/03 18:34:26 INFO BlockManager: Removing block rdd_328_0
15/06/03 18:34:26 INFO MemoryStore: Block rdd_328_0 of size 1664 dropped from memory (free 558457802)
15/06/03 18:34:26 INFO ZippedPartitionsRDD2: Removing RDD 331 from persistence list
15/06/03 18:34:26 INFO ZippedPartitionsRDD2: Removing RDD 314 from persistence list
15/06/03 18:34:26 INFO ZippedPartitionsRDD2: Removing RDD 320 from persistence list
15/06/03 18:34:26 INFO BlockManager: Removing RDD 314
15/06/03 18:34:26 INFO BlockManager: Removing RDD 320
15/06/03 18:34:26 INFO BlockManager: Removing block rdd_314_0
15/06/03 18:34:26 INFO MemoryStore: Block rdd_314_0 of size 1496 dropped from memory (free 558459298)
15/06/03 18:34:26 INFO BlockManager: Removing block rdd_320_0
15/06/03 18:34:26 INFO MemoryStore: Block rdd_320_0 of size 2976 dropped from memory (free 558462274)
15/06/03 18:34:26 INFO BlockManager: Removing RDD 331
15/06/03 18:34:26 INFO BlockManager: Removing block rdd_331_0
15/06/03 18:34:26 INFO MemoryStore: Block rdd_331_0 of size 1544 dropped from memory (free 558463818)
15/06/03 18:34:26 INFO SparkContext: Starting job: reduce at VertexRDDImpl.scala:90
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 2 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 3 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 5 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 4 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 1 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 0 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 8 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 7 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 6 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 11 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 10 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 9 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 14 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 13 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 12 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 17 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 16 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 15 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 20 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 19 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 18 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 23 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 22 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 21 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 26 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 25 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 24 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 29 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 28 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 27 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 32 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 31 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 30 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 35 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 34 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 33 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 38 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 37 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 36 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 41 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 40 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 39 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 44 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 43 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 42 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 47 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 46 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 45 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 50 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 49 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 48 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 53 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 52 is 145 bytes
15/06/03 18:34:26 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 51 is 145 bytes
15/06/03 18:34:26 INFO DAGScheduler: Registering RDD 356 (mapPartitions at VertexRDDImpl.scala:218)
15/06/03 18:34:26 INFO DAGScheduler: Registering RDD 360 (mapPartitions at VertexRDDImpl.scala:222)
15/06/03 18:34:26 INFO DAGScheduler: Registering RDD 364 (mapPartitions at GraphImpl.scala:235)
15/06/03 18:34:26 INFO DAGScheduler: Got job 18 (reduce at VertexRDDImpl.scala:90) with 1 output partitions (allowLocal=false)
15/06/03 18:34:26 INFO DAGScheduler: Final stage: Stage 3987(reduce at VertexRDDImpl.scala:90)
15/06/03 18:34:26 INFO DAGScheduler: Parents of final stage: List(Stage 3950, Stage 3972, Stage 3958, Stage 3980, Stage 3974, Stage 3966, Stage 3960, Stage 3952, Stage 3982, Stage 3968, Stage 3954, Stage 3976, Stage 3962, Stage 3984, Stage 3978, Stage 3970, Stage 3949, Stage 3964, Stage 3956, Stage 3986)
15/06/03 18:34:26 INFO DAGScheduler: Missing parents: List(Stage 3986)
15/06/03 18:34:26 INFO DAGScheduler: Submitting Stage 3930 (ReplicatedVertexView.updateVertices - shippedVerts true false (broadcast) MapPartitionsRDD[356] at mapPartitions at VertexRDDImpl.scala:218), which has no missing parents
15/06/03 18:34:26 INFO MemoryStore: ensureFreeSpace(15136) called with curMem=689334, maxMem=559153152
15/06/03 18:34:26 INFO MemoryStore: Block broadcast_73 stored as values in memory (estimated size 14.8 KB, free 532.6 MB)
15/06/03 18:34:26 INFO MemoryStore: ensureFreeSpace(7544) called with curMem=704470, maxMem=559153152
15/06/03 18:34:26 INFO MemoryStore: Block broadcast_73_piece0 stored as bytes in memory (estimated size 7.4 KB, free 532.6 MB)
15/06/03 18:34:26 INFO BlockManagerInfo: Added broadcast_73_piece0 in memory on localhost:60836 (size: 7.4 KB, free: 533.0 MB)
15/06/03 18:34:26 INFO BlockManagerMaster: Updated info of block broadcast_73_piece0
15/06/03 18:34:26 INFO SparkContext: Created broadcast 73 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:26 INFO DAGScheduler: Submitting 1 missing tasks from Stage 3930 (ReplicatedVertexView.updateVertices - shippedVerts true false (broadcast) MapPartitionsRDD[356] at mapPartitions at VertexRDDImpl.scala:218)
15/06/03 18:34:26 INFO TaskSchedulerImpl: Adding task set 3930.0 with 1 tasks
15/06/03 18:34:26 INFO DAGScheduler: Submitting Stage 3985 (ReplicatedVertexView.withActiveSet - shippedActives (broadcast) MapPartitionsRDD[360] at mapPartitions at VertexRDDImpl.scala:222), which has no missing parents
15/06/03 18:34:26 INFO TaskSetManager: Starting task 0.0 in stage 3930.0 (TID 72, localhost, PROCESS_LOCAL, 3311 bytes)
15/06/03 18:34:26 INFO Executor: Running task 0.0 in stage 3930.0 (TID 72)
15/06/03 18:34:26 INFO MemoryStore: ensureFreeSpace(14792) called with curMem=712014, maxMem=559153152
15/06/03 18:34:26 INFO MemoryStore: Block broadcast_74 stored as values in memory (estimated size 14.4 KB, free 532.6 MB)
15/06/03 18:34:26 INFO MemoryStore: ensureFreeSpace(7428) called with curMem=726806, maxMem=559153152
15/06/03 18:34:26 INFO MemoryStore: Block broadcast_74_piece0 stored as bytes in memory (estimated size 7.3 KB, free 532.5 MB)
15/06/03 18:34:26 INFO BlockManagerInfo: Added broadcast_74_piece0 in memory on localhost:60836 (size: 7.3 KB, free: 533.0 MB)
15/06/03 18:34:26 INFO BlockManager: Removing broadcast 72
15/06/03 18:34:26 INFO BlockManager: Removing block broadcast_72
15/06/03 18:34:26 INFO MemoryStore: Block broadcast_72 of size 14896 dropped from memory (free 558433814)
15/06/03 18:34:26 INFO BlockManager: Removing block broadcast_72_piece0
15/06/03 18:34:26 INFO MemoryStore: Block broadcast_72_piece0 of size 7452 dropped from memory (free 558441266)
15/06/03 18:34:26 INFO BlockManagerInfo: Removed broadcast_72_piece0 on localhost:60836 in memory (size: 7.3 KB, free: 533.0 MB)
15/06/03 18:34:26 INFO BlockManagerMaster: Updated info of block broadcast_72_piece0
15/06/03 18:34:26 INFO ContextCleaner: Cleaned broadcast 72
15/06/03 18:34:26 INFO BlockManager: Removing broadcast 71
15/06/03 18:34:26 INFO BlockManager: Removing block broadcast_71_piece0
15/06/03 18:34:26 INFO MemoryStore: Block broadcast_71_piece0 of size 5974 dropped from memory (free 558447240)
15/06/03 18:34:26 INFO BlockManagerInfo: Removed broadcast_71_piece0 on localhost:60836 in memory (size: 5.8 KB, free: 533.0 MB)
15/06/03 18:34:26 INFO BlockManagerMaster: Updated info of block broadcast_71_piece0
15/06/03 18:34:26 INFO BlockManager: Removing block broadcast_71
15/06/03 18:34:26 INFO MemoryStore: Block broadcast_71 of size 10496 dropped from memory (free 558457736)
15/06/03 18:34:26 INFO ContextCleaner: Cleaned broadcast 71
15/06/03 18:34:26 INFO BlockManager: Removing broadcast 70
15/06/03 18:34:26 INFO BlockManager: Removing block broadcast_70_piece0
15/06/03 18:34:26 INFO MemoryStore: Block broadcast_70_piece0 of size 7203 dropped from memory (free 558464939)
15/06/03 18:34:26 INFO BlockManagerMaster: Updated info of block broadcast_74_piece0
15/06/03 18:34:26 INFO BlockManagerInfo: Removed broadcast_70_piece0 on localhost:60836 in memory (size: 7.0 KB, free: 533.1 MB)
15/06/03 18:34:26 INFO BlockManagerMaster: Updated info of block broadcast_70_piece0
15/06/03 18:34:26 INFO BlockManager: Removing block broadcast_70
15/06/03 18:34:26 INFO MemoryStore: Block broadcast_70 of size 14264 dropped from memory (free 558479203)
15/06/03 18:34:26 INFO BlockManager: Found block rdd_333_0 locally
15/06/03 18:34:26 INFO ContextCleaner: Cleaned broadcast 70
15/06/03 18:34:26 INFO CacheManager: Partition rdd_352_0 not found, computing it
15/06/03 18:34:26 INFO BlockManager: Removing broadcast 69
15/06/03 18:34:26 INFO BlockManager: Removing block broadcast_69
15/06/03 18:34:26 INFO BlockManager: Found block rdd_333_0 locally
15/06/03 18:34:26 INFO MemoryStore: Block broadcast_69 of size 14608 dropped from memory (free 558493811)
15/06/03 18:34:26 INFO BlockManager: Removing block broadcast_69_piece0
15/06/03 18:34:26 INFO MemoryStore: Block broadcast_69_piece0 of size 7400 dropped from memory (free 558501211)
15/06/03 18:34:26 INFO SparkContext: Created broadcast 74 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:26 INFO BlockManagerInfo: Removed broadcast_69_piece0 on localhost:60836 in memory (size: 7.2 KB, free: 533.1 MB)
15/06/03 18:34:26 INFO BlockManagerMaster: Updated info of block broadcast_69_piece0
15/06/03 18:34:26 INFO ContextCleaner: Cleaned broadcast 69
15/06/03 18:34:26 INFO CacheManager: Partition rdd_350_0 not found, computing it
15/06/03 18:34:26 INFO BlockManager: Found block rdd_333_0 locally
15/06/03 18:34:26 INFO BlockManager: Found block rdd_347_0 locally
15/06/03 18:34:26 INFO BlockManager: Removing broadcast 68
15/06/03 18:34:26 INFO MemoryStore: ensureFreeSpace(1544) called with curMem=651941, maxMem=559153152
15/06/03 18:34:26 INFO MemoryStore: Block rdd_350_0 stored as values in memory (estimated size 1544.0 B, free 532.6 MB)
15/06/03 18:34:26 INFO BlockManager: Removing block broadcast_68
15/06/03 18:34:26 INFO MemoryStore: Block broadcast_68 of size 14368 dropped from memory (free 558514035)
15/06/03 18:34:26 INFO BlockManager: Removing block broadcast_68_piece0
15/06/03 18:34:26 INFO BlockManagerInfo: Added rdd_350_0 in memory on localhost:60836 (size: 1544.0 B, free: 533.1 MB)
15/06/03 18:34:26 INFO MemoryStore: Block broadcast_68_piece0 of size 7243 dropped from memory (free 558521278)
15/06/03 18:34:26 INFO BlockManagerMaster: Updated info of block rdd_350_0
15/06/03 18:34:26 INFO DAGScheduler: Submitting 1 missing tasks from Stage 3985 (ReplicatedVertexView.withActiveSet - shippedActives (broadcast) MapPartitionsRDD[360] at mapPartitions at VertexRDDImpl.scala:222)
15/06/03 18:34:26 INFO BlockManagerInfo: Removed broadcast_68_piece0 on localhost:60836 in memory (size: 7.1 KB, free: 533.1 MB)
15/06/03 18:34:26 INFO BlockManagerMaster: Updated info of block broadcast_68_piece0
15/06/03 18:34:26 INFO ContextCleaner: Cleaned broadcast 68
15/06/03 18:34:26 INFO MemoryStore: ensureFreeSpace(1496) called with curMem=631874, maxMem=559153152
15/06/03 18:34:26 INFO MemoryStore: Block rdd_352_0 stored as values in memory (estimated size 1496.0 B, free 532.6 MB)
15/06/03 18:34:26 INFO TaskSchedulerImpl: Adding task set 3985.0 with 1 tasks
15/06/03 18:34:26 INFO BlockManagerInfo: Added rdd_352_0 in memory on localhost:60836 (size: 1496.0 B, free: 533.1 MB)
15/06/03 18:34:26 INFO BlockManager: Removing broadcast 67
15/06/03 18:34:26 INFO BlockManager: Removing block broadcast_67
15/06/03 18:34:26 INFO MemoryStore: Block broadcast_67 of size 10272 dropped from memory (free 558530054)
15/06/03 18:34:26 INFO BlockManager: Removing block broadcast_67_piece0
15/06/03 18:34:26 INFO MemoryStore: Block broadcast_67_piece0 of size 5866 dropped from memory (free 558535920)
15/06/03 18:34:26 INFO BlockManagerMaster: Updated info of block rdd_352_0
15/06/03 18:34:26 INFO BlockManagerInfo: Removed broadcast_67_piece0 on localhost:60836 in memory (size: 5.7 KB, free: 533.1 MB)
15/06/03 18:34:26 INFO BlockManagerMaster: Updated info of block broadcast_67_piece0
15/06/03 18:34:26 INFO ContextCleaner: Cleaned broadcast 67
15/06/03 18:34:26 INFO BlockManager: Removing broadcast 66
15/06/03 18:34:26 INFO BlockManager: Removing block broadcast_66_piece0
15/06/03 18:34:26 INFO MemoryStore: Block broadcast_66_piece0 of size 7047 dropped from memory (free 558542967)
15/06/03 18:34:26 INFO BlockManagerInfo: Removed broadcast_66_piece0 on localhost:60836 in memory (size: 6.9 KB, free: 533.1 MB)
15/06/03 18:34:26 INFO BlockManagerMaster: Updated info of block broadcast_66_piece0
15/06/03 18:34:26 INFO BlockManager: Removing block broadcast_66
15/06/03 18:34:26 INFO MemoryStore: Block broadcast_66 of size 13736 dropped from memory (free 558556703)
15/06/03 18:34:26 INFO ContextCleaner: Cleaned broadcast 66
15/06/03 18:34:26 INFO BlockManager: Removing broadcast 65
15/06/03 18:34:26 INFO BlockManager: Removing block broadcast_65_piece0
15/06/03 18:34:26 INFO MemoryStore: Block broadcast_65_piece0 of size 7192 dropped from memory (free 558563895)
15/06/03 18:34:26 INFO BlockManagerInfo: Removed broadcast_65_piece0 on localhost:60836 in memory (size: 7.0 KB, free: 533.1 MB)
15/06/03 18:34:26 INFO BlockManagerMaster: Updated info of block broadcast_65_piece0
15/06/03 18:34:26 INFO BlockManager: Removing block broadcast_65
15/06/03 18:34:26 INFO MemoryStore: Block broadcast_65 of size 14080 dropped from memory (free 558577975)
15/06/03 18:34:26 INFO ContextCleaner: Cleaned broadcast 65
15/06/03 18:34:26 INFO BlockManager: Removing broadcast 52
15/06/03 18:34:26 INFO BlockManager: Removing block broadcast_52
15/06/03 18:34:26 INFO MemoryStore: Block broadcast_52 of size 12256 dropped from memory (free 558590231)
15/06/03 18:34:26 INFO BlockManager: Removing block broadcast_52_piece0
15/06/03 18:34:26 INFO MemoryStore: Block broadcast_52_piece0 of size 6474 dropped from memory (free 558596705)
15/06/03 18:34:26 INFO BlockManagerInfo: Removed broadcast_52_piece0 on localhost:60836 in memory (size: 6.3 KB, free: 533.1 MB)
15/06/03 18:34:26 INFO BlockManagerMaster: Updated info of block broadcast_52_piece0
15/06/03 18:34:26 INFO ContextCleaner: Cleaned broadcast 52
15/06/03 18:34:26 INFO BlockManager: Removing broadcast 51
15/06/03 18:34:26 INFO BlockManager: Removing block broadcast_51
15/06/03 18:34:26 INFO MemoryStore: Block broadcast_51 of size 9360 dropped from memory (free 558606065)
15/06/03 18:34:26 INFO BlockManager: Removing block broadcast_51_piece0
15/06/03 18:34:26 INFO MemoryStore: Block broadcast_51_piece0 of size 5412 dropped from memory (free 558611477)
15/06/03 18:34:26 INFO BlockManagerInfo: Removed broadcast_51_piece0 on localhost:60836 in memory (size: 5.3 KB, free: 533.1 MB)
15/06/03 18:34:26 INFO BlockManagerMaster: Updated info of block broadcast_51_piece0
15/06/03 18:34:26 INFO ContextCleaner: Cleaned broadcast 51
15/06/03 18:34:26 INFO BlockManager: Removing broadcast 50
15/06/03 18:34:26 INFO BlockManager: Removing block broadcast_50_piece0
15/06/03 18:34:26 INFO MemoryStore: Block broadcast_50_piece0 of size 6261 dropped from memory (free 558617738)
15/06/03 18:34:26 INFO BlockManagerInfo: Removed broadcast_50_piece0 on localhost:60836 in memory (size: 6.1 KB, free: 533.1 MB)
15/06/03 18:34:26 INFO BlockManagerMaster: Updated info of block broadcast_50_piece0
15/06/03 18:34:26 INFO BlockManager: Removing block broadcast_50
15/06/03 18:34:26 INFO MemoryStore: Block broadcast_50 of size 11632 dropped from memory (free 558629370)
15/06/03 18:34:26 INFO ContextCleaner: Cleaned broadcast 50
15/06/03 18:34:26 INFO BlockManager: Removing broadcast 49
15/06/03 18:34:26 INFO BlockManager: Removing block broadcast_49
15/06/03 18:34:26 INFO MemoryStore: Block broadcast_49 of size 11976 dropped from memory (free 558641346)
15/06/03 18:34:26 INFO BlockManager: Removing block broadcast_49_piece0
15/06/03 18:34:26 INFO MemoryStore: Block broadcast_49_piece0 of size 6391 dropped from memory (free 558647737)
15/06/03 18:34:26 INFO BlockManagerInfo: Removed broadcast_49_piece0 on localhost:60836 in memory (size: 6.2 KB, free: 533.1 MB)
15/06/03 18:34:26 INFO BlockManagerMaster: Updated info of block broadcast_49_piece0
15/06/03 18:34:26 INFO ContextCleaner: Cleaned broadcast 49
15/06/03 18:34:26 INFO BlockManager: Removing broadcast 48
15/06/03 18:34:26 INFO BlockManager: Removing block broadcast_48_piece0
15/06/03 18:34:26 INFO MemoryStore: Block broadcast_48_piece0 of size 6275 dropped from memory (free 558654012)
15/06/03 18:34:26 INFO BlockManagerInfo: Removed broadcast_48_piece0 on localhost:60836 in memory (size: 6.1 KB, free: 533.1 MB)
15/06/03 18:34:26 INFO BlockManagerMaster: Updated info of block broadcast_48_piece0
15/06/03 18:34:26 INFO BlockManager: Removing block broadcast_48
15/06/03 18:34:26 INFO MemoryStore: Block broadcast_48 of size 11736 dropped from memory (free 558665748)
15/06/03 18:34:26 INFO ContextCleaner: Cleaned broadcast 48
15/06/03 18:34:26 INFO BlockManager: Removing broadcast 47
15/06/03 18:34:26 INFO BlockManager: Removing block broadcast_47
15/06/03 18:34:26 INFO MemoryStore: Block broadcast_47 of size 9128 dropped from memory (free 558674876)
15/06/03 18:34:26 INFO BlockManager: Removing block broadcast_47_piece0
15/06/03 18:34:26 INFO MemoryStore: Block broadcast_47_piece0 of size 5401 dropped from memory (free 558680277)
15/06/03 18:34:26 INFO BlockManagerInfo: Removed broadcast_47_piece0 on localhost:60836 in memory (size: 5.3 KB, free: 533.1 MB)
15/06/03 18:34:26 INFO BlockManagerMaster: Updated info of block broadcast_47_piece0
15/06/03 18:34:26 INFO ContextCleaner: Cleaned broadcast 47
15/06/03 18:34:26 INFO BlockManager: Removing broadcast 60
15/06/03 18:34:26 INFO BlockManager: Removing block broadcast_60
15/06/03 18:34:26 INFO MemoryStore: Block broadcast_60 of size 13312 dropped from memory (free 558693589)
15/06/03 18:34:26 INFO BlockManager: Removing block broadcast_60_piece0
15/06/03 18:34:26 INFO MemoryStore: Block broadcast_60_piece0 of size 6881 dropped from memory (free 558700470)
15/06/03 18:34:26 INFO BlockManagerInfo: Removed broadcast_60_piece0 on localhost:60836 in memory (size: 6.7 KB, free: 533.1 MB)
15/06/03 18:34:26 INFO Executor: Finished task 0.0 in stage 3930.0 (TID 72). 2627 bytes result sent to driver
15/06/03 18:34:26 INFO BlockManagerMaster: Updated info of block broadcast_60_piece0
15/06/03 18:34:26 INFO TaskSetManager: Starting task 0.0 in stage 3985.0 (TID 73, localhost, PROCESS_LOCAL, 3247 bytes)
15/06/03 18:34:26 INFO ContextCleaner: Cleaned broadcast 60
15/06/03 18:34:26 INFO Executor: Running task 0.0 in stage 3985.0 (TID 73)
15/06/03 18:34:26 INFO TaskSetManager: Finished task 0.0 in stage 3930.0 (TID 72) in 61 ms on localhost (1/1)
15/06/03 18:34:26 INFO TaskSchedulerImpl: Removed TaskSet 3930.0, whose tasks have all completed, from pool
15/06/03 18:34:26 INFO BlockManager: Removing broadcast 59
15/06/03 18:34:26 INFO BlockManager: Removing block broadcast_59_piece0
15/06/03 18:34:26 INFO MemoryStore: Block broadcast_59_piece0 of size 5675 dropped from memory (free 558706145)
15/06/03 18:34:26 INFO BlockManagerInfo: Removed broadcast_59_piece0 on localhost:60836 in memory (size: 5.5 KB, free: 533.1 MB)
15/06/03 18:34:26 INFO BlockManagerMaster: Updated info of block broadcast_59_piece0
15/06/03 18:34:26 INFO BlockManager: Removing block broadcast_59
15/06/03 18:34:26 INFO MemoryStore: Block broadcast_59 of size 9816 dropped from memory (free 558715961)
15/06/03 18:34:26 INFO ContextCleaner: Cleaned broadcast 59
15/06/03 18:34:26 INFO BlockManager: Removing broadcast 58
15/06/03 18:34:26 INFO BlockManager: Removing block broadcast_58
15/06/03 18:34:26 INFO MemoryStore: Block broadcast_58 of size 12680 dropped from memory (free 558728641)
15/06/03 18:34:26 INFO BlockManager: Removing block broadcast_58_piece0
15/06/03 18:34:26 INFO MemoryStore: Block broadcast_58_piece0 of size 6645 dropped from memory (free 558735286)
15/06/03 18:34:26 INFO BlockManagerInfo: Removed broadcast_58_piece0 on localhost:60836 in memory (size: 6.5 KB, free: 533.1 MB)
15/06/03 18:34:26 INFO BlockManagerMaster: Updated info of block broadcast_58_piece0
15/06/03 18:34:26 INFO ContextCleaner: Cleaned broadcast 58
15/06/03 18:34:26 INFO BlockManager: Removing broadcast 56
15/06/03 18:34:26 INFO BlockManager: Removing block broadcast_56
15/06/03 18:34:26 INFO MemoryStore: Block broadcast_56 of size 12784 dropped from memory (free 558748070)
15/06/03 18:34:26 INFO BlockManager: Removing block broadcast_56_piece0
15/06/03 18:34:26 INFO MemoryStore: Block broadcast_56_piece0 of size 6685 dropped from memory (free 558754755)
15/06/03 18:34:26 INFO BlockManager: Found block rdd_350_0 locally
15/06/03 18:34:26 INFO BlockManagerInfo: Removed broadcast_56_piece0 on localhost:60836 in memory (size: 6.5 KB, free: 533.1 MB)
15/06/03 18:34:26 INFO BlockManagerMaster: Updated info of block broadcast_56_piece0
15/06/03 18:34:26 INFO ContextCleaner: Cleaned broadcast 56
15/06/03 18:34:26 INFO BlockManager: Removing broadcast 55
15/06/03 18:34:26 INFO DAGScheduler: Stage 3930 (mapPartitions at VertexRDDImpl.scala:218) finished in 0.068 s
15/06/03 18:34:26 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:26 INFO DAGScheduler: running: Set(Stage 3985)
15/06/03 18:34:26 INFO DAGScheduler: waiting: Set(Stage 3987, Stage 3986)
15/06/03 18:34:26 INFO DAGScheduler: failed: Set()
15/06/03 18:34:26 INFO BlockManager: Removing block broadcast_55_piece0
15/06/03 18:34:26 INFO MemoryStore: Block broadcast_55_piece0 of size 5619 dropped from memory (free 558760374)
15/06/03 18:34:26 INFO BlockManagerInfo: Removed broadcast_55_piece0 on localhost:60836 in memory (size: 5.5 KB, free: 533.1 MB)
15/06/03 18:34:26 INFO BlockManagerMaster: Updated info of block broadcast_55_piece0
15/06/03 18:34:26 INFO BlockManager: Removing block broadcast_55
15/06/03 18:34:26 INFO MemoryStore: Block broadcast_55 of size 9584 dropped from memory (free 558769958)
15/06/03 18:34:26 INFO ContextCleaner: Cleaned broadcast 55
15/06/03 18:34:26 INFO DAGScheduler: Missing parents for Stage 3987: List(Stage 3986)
15/06/03 18:34:26 INFO BlockManager: Removing broadcast 54
15/06/03 18:34:26 INFO BlockManager: Removing block broadcast_54
15/06/03 18:34:26 INFO MemoryStore: Block broadcast_54 of size 12160 dropped from memory (free 558782118)
15/06/03 18:34:26 INFO BlockManager: Removing block broadcast_54_piece0
15/06/03 18:34:26 INFO MemoryStore: Block broadcast_54_piece0 of size 6430 dropped from memory (free 558788548)
15/06/03 18:34:26 INFO BlockManagerInfo: Removed broadcast_54_piece0 on localhost:60836 in memory (size: 6.3 KB, free: 533.2 MB)
15/06/03 18:34:26 INFO BlockManagerMaster: Updated info of block broadcast_54_piece0
15/06/03 18:34:26 INFO ContextCleaner: Cleaned broadcast 54
15/06/03 18:34:26 INFO BlockManager: Removing broadcast 53
15/06/03 18:34:26 INFO BlockManager: Removing block broadcast_53_piece0
15/06/03 18:34:26 INFO MemoryStore: Block broadcast_53_piece0 of size 6613 dropped from memory (free 558795161)
15/06/03 18:34:26 INFO DAGScheduler: Missing parents for Stage 3986: List(Stage 3985)
15/06/03 18:34:26 INFO BlockManagerInfo: Removed broadcast_53_piece0 on localhost:60836 in memory (size: 6.5 KB, free: 533.2 MB)
15/06/03 18:34:26 INFO BlockManagerMaster: Updated info of block broadcast_53_piece0
15/06/03 18:34:26 INFO BlockManager: Removing block broadcast_53
15/06/03 18:34:26 INFO MemoryStore: Block broadcast_53 of size 12496 dropped from memory (free 558807657)
15/06/03 18:34:26 INFO ContextCleaner: Cleaned broadcast 53
15/06/03 18:34:26 INFO BlockManager: Removing broadcast 64
15/06/03 18:34:26 INFO BlockManager: Removing block broadcast_64
15/06/03 18:34:26 INFO MemoryStore: Block broadcast_64 of size 13840 dropped from memory (free 558821497)
15/06/03 18:34:26 INFO BlockManager: Removing block broadcast_64_piece0
15/06/03 18:34:26 INFO MemoryStore: Block broadcast_64_piece0 of size 7080 dropped from memory (free 558828577)
15/06/03 18:34:26 INFO Executor: Finished task 0.0 in stage 3985.0 (TID 73). 2003 bytes result sent to driver
15/06/03 18:34:26 INFO BlockManagerInfo: Removed broadcast_64_piece0 on localhost:60836 in memory (size: 6.9 KB, free: 533.2 MB)
15/06/03 18:34:26 INFO BlockManagerMaster: Updated info of block broadcast_64_piece0
15/06/03 18:34:26 INFO ContextCleaner: Cleaned broadcast 64
15/06/03 18:34:26 INFO BlockManager: Removing broadcast 63
15/06/03 18:34:26 INFO TaskSetManager: Finished task 0.0 in stage 3985.0 (TID 73) in 19 ms on localhost (1/1)
15/06/03 18:34:26 INFO TaskSchedulerImpl: Removed TaskSet 3985.0, whose tasks have all completed, from pool
15/06/03 18:34:26 INFO BlockManager: Removing block broadcast_63_piece0
15/06/03 18:34:26 INFO MemoryStore: Block broadcast_63_piece0 of size 5780 dropped from memory (free 558834357)
15/06/03 18:34:26 INFO DAGScheduler: Stage 3985 (mapPartitions at VertexRDDImpl.scala:222) finished in 0.038 s
15/06/03 18:34:26 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:26 INFO DAGScheduler: running: Set()
15/06/03 18:34:26 INFO DAGScheduler: waiting: Set(Stage 3987, Stage 3986)
15/06/03 18:34:26 INFO BlockManagerInfo: Removed broadcast_63_piece0 on localhost:60836 in memory (size: 5.6 KB, free: 533.2 MB)
15/06/03 18:34:26 INFO DAGScheduler: failed: Set()
15/06/03 18:34:26 INFO BlockManagerMaster: Updated info of block broadcast_63_piece0
15/06/03 18:34:26 INFO BlockManager: Removing block broadcast_63
15/06/03 18:34:26 INFO MemoryStore: Block broadcast_63 of size 10040 dropped from memory (free 558844397)
15/06/03 18:34:26 INFO ContextCleaner: Cleaned broadcast 63
15/06/03 18:34:26 INFO BlockManager: Removing broadcast 62
15/06/03 18:34:26 INFO BlockManager: Removing block broadcast_62_piece0
15/06/03 18:34:26 INFO MemoryStore: Block broadcast_62_piece0 of size 6841 dropped from memory (free 558851238)
15/06/03 18:34:26 INFO BlockManagerInfo: Removed broadcast_62_piece0 on localhost:60836 in memory (size: 6.7 KB, free: 533.2 MB)
15/06/03 18:34:26 INFO BlockManagerMaster: Updated info of block broadcast_62_piece0
15/06/03 18:34:26 INFO BlockManager: Removing block broadcast_62
15/06/03 18:34:26 INFO MemoryStore: Block broadcast_62 of size 13208 dropped from memory (free 558864446)
15/06/03 18:34:26 INFO DAGScheduler: Missing parents for Stage 3987: List(Stage 3986)
15/06/03 18:34:26 INFO ContextCleaner: Cleaned broadcast 62
15/06/03 18:34:26 INFO BlockManager: Removing broadcast 61
15/06/03 18:34:26 INFO BlockManager: Removing block broadcast_61_piece0
15/06/03 18:34:26 INFO MemoryStore: Block broadcast_61_piece0 of size 7039 dropped from memory (free 558871485)
15/06/03 18:34:26 INFO BlockManagerInfo: Removed broadcast_61_piece0 on localhost:60836 in memory (size: 6.9 KB, free: 533.2 MB)
15/06/03 18:34:26 INFO BlockManagerMaster: Updated info of block broadcast_61_piece0
15/06/03 18:34:26 INFO BlockManager: Removing block broadcast_61
15/06/03 18:34:26 INFO MemoryStore: Block broadcast_61 of size 13552 dropped from memory (free 558885037)
15/06/03 18:34:26 INFO ContextCleaner: Cleaned broadcast 61
15/06/03 18:34:26 INFO DAGScheduler: Missing parents for Stage 3986: List()
15/06/03 18:34:26 INFO DAGScheduler: Submitting Stage 3986 (GraphImpl.aggregateMessages - preAgg MapPartitionsRDD[364] at mapPartitions at GraphImpl.scala:235), which is now runnable
15/06/03 18:34:26 INFO MemoryStore: ensureFreeSpace(10728) called with curMem=268115, maxMem=559153152
15/06/03 18:34:26 INFO MemoryStore: Block broadcast_75 stored as values in memory (estimated size 10.5 KB, free 533.0 MB)
15/06/03 18:34:26 INFO MemoryStore: ensureFreeSpace(6091) called with curMem=278843, maxMem=559153152
15/06/03 18:34:26 INFO MemoryStore: Block broadcast_75_piece0 stored as bytes in memory (estimated size 5.9 KB, free 533.0 MB)
15/06/03 18:34:26 INFO BlockManagerInfo: Added broadcast_75_piece0 in memory on localhost:60836 (size: 5.9 KB, free: 533.2 MB)
15/06/03 18:34:26 INFO BlockManagerMaster: Updated info of block broadcast_75_piece0
15/06/03 18:34:26 INFO SparkContext: Created broadcast 75 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:26 INFO DAGScheduler: Submitting 1 missing tasks from Stage 3986 (GraphImpl.aggregateMessages - preAgg MapPartitionsRDD[364] at mapPartitions at GraphImpl.scala:235)
15/06/03 18:34:26 INFO TaskSchedulerImpl: Adding task set 3986.0 with 1 tasks
15/06/03 18:34:26 INFO TaskSetManager: Starting task 0.0 in stage 3986.0 (TID 74, localhost, PROCESS_LOCAL, 2632 bytes)
15/06/03 18:34:26 INFO Executor: Running task 0.0 in stage 3986.0 (TID 74)
15/06/03 18:34:26 INFO CacheManager: Partition rdd_358_0 not found, computing it
15/06/03 18:34:26 INFO BlockManager: Found block rdd_339_0 locally
15/06/03 18:34:26 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:26 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:26 INFO MemoryStore: ensureFreeSpace(2976) called with curMem=284934, maxMem=559153152
15/06/03 18:34:26 INFO MemoryStore: Block rdd_358_0 stored as values in memory (estimated size 2.9 KB, free 533.0 MB)
15/06/03 18:34:26 INFO BlockManagerInfo: Added rdd_358_0 in memory on localhost:60836 (size: 2.9 KB, free: 533.2 MB)
15/06/03 18:34:26 INFO BlockManagerMaster: Updated info of block rdd_358_0
15/06/03 18:34:26 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:26 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:27 INFO Executor: Finished task 0.0 in stage 3986.0 (TID 74). 2788 bytes result sent to driver
15/06/03 18:34:27 INFO TaskSetManager: Finished task 0.0 in stage 3986.0 (TID 74) in 17 ms on localhost (1/1)
15/06/03 18:34:27 INFO TaskSchedulerImpl: Removed TaskSet 3986.0, whose tasks have all completed, from pool
15/06/03 18:34:27 INFO DAGScheduler: Stage 3986 (mapPartitions at GraphImpl.scala:235) finished in 0.017 s
15/06/03 18:34:27 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:27 INFO DAGScheduler: running: Set()
15/06/03 18:34:27 INFO DAGScheduler: waiting: Set(Stage 3987)
15/06/03 18:34:27 INFO DAGScheduler: failed: Set()
15/06/03 18:34:27 INFO DAGScheduler: Missing parents for Stage 3987: List()
15/06/03 18:34:27 INFO DAGScheduler: Submitting Stage 3987 (MapPartitionsRDD[368] at map at VertexRDDImpl.scala:90), which is now runnable
15/06/03 18:34:27 INFO MemoryStore: ensureFreeSpace(15424) called with curMem=287910, maxMem=559153152
15/06/03 18:34:27 INFO MemoryStore: Block broadcast_76 stored as values in memory (estimated size 15.1 KB, free 533.0 MB)
15/06/03 18:34:27 INFO MemoryStore: ensureFreeSpace(7640) called with curMem=303334, maxMem=559153152
15/06/03 18:34:27 INFO MemoryStore: Block broadcast_76_piece0 stored as bytes in memory (estimated size 7.5 KB, free 533.0 MB)
15/06/03 18:34:27 INFO BlockManagerInfo: Added broadcast_76_piece0 in memory on localhost:60836 (size: 7.5 KB, free: 533.2 MB)
15/06/03 18:34:27 INFO BlockManagerMaster: Updated info of block broadcast_76_piece0
15/06/03 18:34:27 INFO SparkContext: Created broadcast 76 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:27 INFO DAGScheduler: Submitting 1 missing tasks from Stage 3987 (MapPartitionsRDD[368] at map at VertexRDDImpl.scala:90)
15/06/03 18:34:27 INFO TaskSchedulerImpl: Adding task set 3987.0 with 1 tasks
15/06/03 18:34:27 INFO TaskSetManager: Starting task 0.0 in stage 3987.0 (TID 75, localhost, PROCESS_LOCAL, 3331 bytes)
15/06/03 18:34:27 INFO Executor: Running task 0.0 in stage 3987.0 (TID 75)
15/06/03 18:34:27 INFO CacheManager: Partition rdd_366_0 not found, computing it
15/06/03 18:34:27 INFO BlockManager: Found block rdd_352_0 locally
15/06/03 18:34:27 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:27 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:27 INFO MemoryStore: ensureFreeSpace(1664) called with curMem=310974, maxMem=559153152
15/06/03 18:34:27 INFO MemoryStore: Block rdd_366_0 stored as values in memory (estimated size 1664.0 B, free 533.0 MB)
15/06/03 18:34:27 INFO BlockManagerInfo: Added rdd_366_0 in memory on localhost:60836 (size: 1664.0 B, free: 533.2 MB)
15/06/03 18:34:27 INFO BlockManagerMaster: Updated info of block rdd_366_0
15/06/03 18:34:27 INFO Executor: Finished task 0.0 in stage 3987.0 (TID 75). 2693 bytes result sent to driver
15/06/03 18:34:27 INFO TaskSetManager: Finished task 0.0 in stage 3987.0 (TID 75) in 8 ms on localhost (1/1)
15/06/03 18:34:27 INFO TaskSchedulerImpl: Removed TaskSet 3987.0, whose tasks have all completed, from pool
15/06/03 18:34:27 INFO DAGScheduler: Stage 3987 (reduce at VertexRDDImpl.scala:90) finished in 0.008 s
15/06/03 18:34:27 INFO DAGScheduler: Job 18 finished: reduce at VertexRDDImpl.scala:90, took 0.235162 s
15/06/03 18:34:27 INFO Pregel: Pregel finished iteration 16
15/06/03 18:34:27 INFO ZippedPartitionsRDD2: Removing RDD 347 from persistence list
15/06/03 18:34:27 INFO BlockManager: Removing RDD 347
15/06/03 18:34:27 INFO ZippedPartitionsRDD2: Removing RDD 350 from persistence list
15/06/03 18:34:27 INFO BlockManager: Removing block rdd_347_0
15/06/03 18:34:27 INFO MemoryStore: Block rdd_347_0 of size 1664 dropped from memory (free 558842178)
15/06/03 18:34:27 INFO BlockManager: Removing RDD 350
15/06/03 18:34:27 INFO BlockManager: Removing block rdd_350_0
15/06/03 18:34:27 INFO MemoryStore: Block rdd_350_0 of size 1544 dropped from memory (free 558843722)
15/06/03 18:34:27 INFO ZippedPartitionsRDD2: Removing RDD 333 from persistence list
15/06/03 18:34:27 INFO BlockManager: Removing RDD 333
15/06/03 18:34:27 INFO ZippedPartitionsRDD2: Removing RDD 339 from persistence list
15/06/03 18:34:27 INFO BlockManager: Removing block rdd_333_0
15/06/03 18:34:27 INFO MemoryStore: Block rdd_333_0 of size 1496 dropped from memory (free 558845218)
15/06/03 18:34:27 INFO BlockManager: Removing RDD 339
15/06/03 18:34:27 INFO BlockManager: Removing block rdd_339_0
15/06/03 18:34:27 INFO MemoryStore: Block rdd_339_0 of size 2976 dropped from memory (free 558848194)
15/06/03 18:34:27 INFO SparkContext: Starting job: reduce at VertexRDDImpl.scala:90
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 2 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 3 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 5 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 4 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 1 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 0 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 8 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 7 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 6 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 11 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 10 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 9 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 14 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 13 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 12 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 17 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 16 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 15 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 20 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 19 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 18 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 23 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 22 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 21 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 26 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 25 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 24 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 29 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 28 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 27 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 32 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 31 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 30 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 35 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 34 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 33 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 38 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 37 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 36 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 41 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 40 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 39 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 44 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 43 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 42 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 47 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 46 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 45 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 50 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 49 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 48 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 53 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 52 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 51 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 56 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 55 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 54 is 145 bytes
15/06/03 18:34:27 INFO DAGScheduler: Registering RDD 375 (mapPartitions at VertexRDDImpl.scala:218)
15/06/03 18:34:27 INFO DAGScheduler: Registering RDD 379 (mapPartitions at VertexRDDImpl.scala:222)
15/06/03 18:34:27 INFO DAGScheduler: Registering RDD 383 (mapPartitions at GraphImpl.scala:235)
15/06/03 18:34:27 INFO DAGScheduler: Got job 19 (reduce at VertexRDDImpl.scala:90) with 1 output partitions (allowLocal=false)
15/06/03 18:34:27 INFO DAGScheduler: Final stage: Stage 4617(reduce at VertexRDDImpl.scala:90)
15/06/03 18:34:27 INFO DAGScheduler: Parents of final stage: List(Stage 4582, Stage 4604, Stage 4590, Stage 4612, Stage 4606, Stage 4598, Stage 4577, Stage 4592, Stage 4584, Stage 4614, Stage 4578, Stage 4600, Stage 4586, Stage 4608, Stage 4602, Stage 4594, Stage 4588, Stage 4616, Stage 4580, Stage 4610, Stage 4596)
15/06/03 18:34:27 INFO DAGScheduler: Missing parents: List(Stage 4616)
15/06/03 18:34:27 INFO DAGScheduler: Submitting Stage 4557 (ReplicatedVertexView.updateVertices - shippedVerts true false (broadcast) MapPartitionsRDD[375] at mapPartitions at VertexRDDImpl.scala:218), which has no missing parents
15/06/03 18:34:27 INFO MemoryStore: ensureFreeSpace(15664) called with curMem=304958, maxMem=559153152
15/06/03 18:34:27 INFO MemoryStore: Block broadcast_77 stored as values in memory (estimated size 15.3 KB, free 532.9 MB)
15/06/03 18:34:27 INFO MemoryStore: ensureFreeSpace(7768) called with curMem=320622, maxMem=559153152
15/06/03 18:34:27 INFO MemoryStore: Block broadcast_77_piece0 stored as bytes in memory (estimated size 7.6 KB, free 532.9 MB)
15/06/03 18:34:27 INFO BlockManagerInfo: Added broadcast_77_piece0 in memory on localhost:60836 (size: 7.6 KB, free: 533.2 MB)
15/06/03 18:34:27 INFO BlockManagerMaster: Updated info of block broadcast_77_piece0
15/06/03 18:34:27 INFO SparkContext: Created broadcast 77 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:27 INFO DAGScheduler: Submitting 1 missing tasks from Stage 4557 (ReplicatedVertexView.updateVertices - shippedVerts true false (broadcast) MapPartitionsRDD[375] at mapPartitions at VertexRDDImpl.scala:218)
15/06/03 18:34:27 INFO TaskSchedulerImpl: Adding task set 4557.0 with 1 tasks
15/06/03 18:34:27 INFO DAGScheduler: Submitting Stage 4615 (ReplicatedVertexView.withActiveSet - shippedActives (broadcast) MapPartitionsRDD[379] at mapPartitions at VertexRDDImpl.scala:222), which has no missing parents
15/06/03 18:34:27 INFO TaskSetManager: Starting task 0.0 in stage 4557.0 (TID 76, localhost, PROCESS_LOCAL, 3416 bytes)
15/06/03 18:34:27 INFO Executor: Running task 0.0 in stage 4557.0 (TID 76)
15/06/03 18:34:27 INFO MemoryStore: ensureFreeSpace(15320) called with curMem=328390, maxMem=559153152
15/06/03 18:34:27 INFO MemoryStore: Block broadcast_78 stored as values in memory (estimated size 15.0 KB, free 532.9 MB)
15/06/03 18:34:27 INFO MemoryStore: ensureFreeSpace(7585) called with curMem=343710, maxMem=559153152
15/06/03 18:34:27 INFO MemoryStore: Block broadcast_78_piece0 stored as bytes in memory (estimated size 7.4 KB, free 532.9 MB)
15/06/03 18:34:27 INFO BlockManagerInfo: Added broadcast_78_piece0 in memory on localhost:60836 (size: 7.4 KB, free: 533.2 MB)
15/06/03 18:34:27 INFO BlockManagerMaster: Updated info of block broadcast_78_piece0
15/06/03 18:34:27 INFO SparkContext: Created broadcast 78 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:27 INFO DAGScheduler: Submitting 1 missing tasks from Stage 4615 (ReplicatedVertexView.withActiveSet - shippedActives (broadcast) MapPartitionsRDD[379] at mapPartitions at VertexRDDImpl.scala:222)
15/06/03 18:34:27 INFO TaskSchedulerImpl: Adding task set 4615.0 with 1 tasks
15/06/03 18:34:27 INFO BlockManager: Found block rdd_352_0 locally
15/06/03 18:34:27 INFO CacheManager: Partition rdd_371_0 not found, computing it
15/06/03 18:34:27 INFO BlockManager: Found block rdd_352_0 locally
15/06/03 18:34:27 INFO CacheManager: Partition rdd_369_0 not found, computing it
15/06/03 18:34:27 INFO BlockManager: Found block rdd_352_0 locally
15/06/03 18:34:27 INFO BlockManager: Found block rdd_366_0 locally
15/06/03 18:34:27 INFO MemoryStore: ensureFreeSpace(1544) called with curMem=351295, maxMem=559153152
15/06/03 18:34:27 INFO MemoryStore: Block rdd_369_0 stored as values in memory (estimated size 1544.0 B, free 532.9 MB)
15/06/03 18:34:27 INFO BlockManagerInfo: Added rdd_369_0 in memory on localhost:60836 (size: 1544.0 B, free: 533.2 MB)
15/06/03 18:34:27 INFO BlockManagerMaster: Updated info of block rdd_369_0
15/06/03 18:34:27 INFO MemoryStore: ensureFreeSpace(1496) called with curMem=352839, maxMem=559153152
15/06/03 18:34:27 INFO MemoryStore: Block rdd_371_0 stored as values in memory (estimated size 1496.0 B, free 532.9 MB)
15/06/03 18:34:27 INFO BlockManagerInfo: Added rdd_371_0 in memory on localhost:60836 (size: 1496.0 B, free: 533.2 MB)
15/06/03 18:34:27 INFO BlockManagerMaster: Updated info of block rdd_371_0
15/06/03 18:34:27 INFO Executor: Finished task 0.0 in stage 4557.0 (TID 76). 2627 bytes result sent to driver
15/06/03 18:34:27 INFO TaskSetManager: Starting task 0.0 in stage 4615.0 (TID 77, localhost, PROCESS_LOCAL, 3352 bytes)
15/06/03 18:34:27 INFO Executor: Running task 0.0 in stage 4615.0 (TID 77)
15/06/03 18:34:27 INFO TaskSetManager: Finished task 0.0 in stage 4557.0 (TID 76) in 29 ms on localhost (1/1)
15/06/03 18:34:27 INFO TaskSchedulerImpl: Removed TaskSet 4557.0, whose tasks have all completed, from pool
15/06/03 18:34:27 INFO DAGScheduler: Stage 4557 (mapPartitions at VertexRDDImpl.scala:218) finished in 0.030 s
15/06/03 18:34:27 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:27 INFO DAGScheduler: running: Set(Stage 4615)
15/06/03 18:34:27 INFO DAGScheduler: waiting: Set(Stage 4616, Stage 4617)
15/06/03 18:34:27 INFO DAGScheduler: failed: Set()
15/06/03 18:34:27 INFO DAGScheduler: Missing parents for Stage 4616: List(Stage 4615)
15/06/03 18:34:27 INFO DAGScheduler: Missing parents for Stage 4617: List(Stage 4616)
15/06/03 18:34:27 INFO BlockManager: Found block rdd_369_0 locally
15/06/03 18:34:27 INFO Executor: Finished task 0.0 in stage 4615.0 (TID 77). 2003 bytes result sent to driver
15/06/03 18:34:27 INFO TaskSetManager: Finished task 0.0 in stage 4615.0 (TID 77) in 11 ms on localhost (1/1)
15/06/03 18:34:27 INFO TaskSchedulerImpl: Removed TaskSet 4615.0, whose tasks have all completed, from pool
15/06/03 18:34:27 INFO DAGScheduler: Stage 4615 (mapPartitions at VertexRDDImpl.scala:222) finished in 0.033 s
15/06/03 18:34:27 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:27 INFO DAGScheduler: running: Set()
15/06/03 18:34:27 INFO DAGScheduler: waiting: Set(Stage 4616, Stage 4617)
15/06/03 18:34:27 INFO DAGScheduler: failed: Set()
15/06/03 18:34:27 INFO DAGScheduler: Missing parents for Stage 4616: List()
15/06/03 18:34:27 INFO DAGScheduler: Missing parents for Stage 4617: List(Stage 4616)
15/06/03 18:34:27 INFO DAGScheduler: Submitting Stage 4616 (GraphImpl.aggregateMessages - preAgg MapPartitionsRDD[383] at mapPartitions at GraphImpl.scala:235), which is now runnable
15/06/03 18:34:27 INFO MemoryStore: ensureFreeSpace(10952) called with curMem=354335, maxMem=559153152
15/06/03 18:34:27 INFO MemoryStore: Block broadcast_79 stored as values in memory (estimated size 10.7 KB, free 532.9 MB)
15/06/03 18:34:27 INFO MemoryStore: ensureFreeSpace(6138) called with curMem=365287, maxMem=559153152
15/06/03 18:34:27 INFO MemoryStore: Block broadcast_79_piece0 stored as bytes in memory (estimated size 6.0 KB, free 532.9 MB)
15/06/03 18:34:27 INFO BlockManagerInfo: Added broadcast_79_piece0 in memory on localhost:60836 (size: 6.0 KB, free: 533.2 MB)
15/06/03 18:34:27 INFO BlockManagerMaster: Updated info of block broadcast_79_piece0
15/06/03 18:34:27 INFO SparkContext: Created broadcast 79 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:27 INFO DAGScheduler: Submitting 1 missing tasks from Stage 4616 (GraphImpl.aggregateMessages - preAgg MapPartitionsRDD[383] at mapPartitions at GraphImpl.scala:235)
15/06/03 18:34:27 INFO TaskSchedulerImpl: Adding task set 4616.0 with 1 tasks
15/06/03 18:34:27 INFO TaskSetManager: Starting task 0.0 in stage 4616.0 (TID 78, localhost, PROCESS_LOCAL, 2673 bytes)
15/06/03 18:34:27 INFO Executor: Running task 0.0 in stage 4616.0 (TID 78)
15/06/03 18:34:27 INFO CacheManager: Partition rdd_377_0 not found, computing it
15/06/03 18:34:27 INFO BlockManager: Found block rdd_358_0 locally
15/06/03 18:34:27 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:27 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
15/06/03 18:34:27 INFO MemoryStore: ensureFreeSpace(2976) called with curMem=371425, maxMem=559153152
15/06/03 18:34:27 INFO MemoryStore: Block rdd_377_0 stored as values in memory (estimated size 2.9 KB, free 532.9 MB)
15/06/03 18:34:27 INFO BlockManagerInfo: Added rdd_377_0 in memory on localhost:60836 (size: 2.9 KB, free: 533.1 MB)
15/06/03 18:34:27 INFO BlockManagerMaster: Updated info of block rdd_377_0
15/06/03 18:34:27 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:27 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:27 INFO Executor: Finished task 0.0 in stage 4616.0 (TID 78). 2788 bytes result sent to driver
15/06/03 18:34:27 INFO TaskSetManager: Finished task 0.0 in stage 4616.0 (TID 78) in 17 ms on localhost (1/1)
15/06/03 18:34:27 INFO TaskSchedulerImpl: Removed TaskSet 4616.0, whose tasks have all completed, from pool
15/06/03 18:34:27 INFO DAGScheduler: Stage 4616 (mapPartitions at GraphImpl.scala:235) finished in 0.018 s
15/06/03 18:34:27 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:27 INFO DAGScheduler: running: Set()
15/06/03 18:34:27 INFO DAGScheduler: waiting: Set(Stage 4617)
15/06/03 18:34:27 INFO DAGScheduler: failed: Set()
15/06/03 18:34:27 INFO DAGScheduler: Missing parents for Stage 4617: List()
15/06/03 18:34:27 INFO DAGScheduler: Submitting Stage 4617 (MapPartitionsRDD[387] at map at VertexRDDImpl.scala:90), which is now runnable
15/06/03 18:34:27 INFO MemoryStore: ensureFreeSpace(15952) called with curMem=374401, maxMem=559153152
15/06/03 18:34:27 INFO MemoryStore: Block broadcast_80 stored as values in memory (estimated size 15.6 KB, free 532.9 MB)
15/06/03 18:34:27 INFO MemoryStore: ensureFreeSpace(7869) called with curMem=390353, maxMem=559153152
15/06/03 18:34:27 INFO MemoryStore: Block broadcast_80_piece0 stored as bytes in memory (estimated size 7.7 KB, free 532.9 MB)
15/06/03 18:34:27 INFO BlockManagerInfo: Added broadcast_80_piece0 in memory on localhost:60836 (size: 7.7 KB, free: 533.1 MB)
15/06/03 18:34:27 INFO BlockManagerMaster: Updated info of block broadcast_80_piece0
15/06/03 18:34:27 INFO SparkContext: Created broadcast 80 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:27 INFO DAGScheduler: Submitting 1 missing tasks from Stage 4617 (MapPartitionsRDD[387] at map at VertexRDDImpl.scala:90)
15/06/03 18:34:27 INFO TaskSchedulerImpl: Adding task set 4617.0 with 1 tasks
15/06/03 18:34:27 INFO TaskSetManager: Starting task 0.0 in stage 4617.0 (TID 79, localhost, PROCESS_LOCAL, 3436 bytes)
15/06/03 18:34:27 INFO Executor: Running task 0.0 in stage 4617.0 (TID 79)
15/06/03 18:34:27 INFO CacheManager: Partition rdd_385_0 not found, computing it
15/06/03 18:34:27 INFO BlockManager: Found block rdd_371_0 locally
15/06/03 18:34:27 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:27 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:27 INFO MemoryStore: ensureFreeSpace(1664) called with curMem=398222, maxMem=559153152
15/06/03 18:34:27 INFO MemoryStore: Block rdd_385_0 stored as values in memory (estimated size 1664.0 B, free 532.9 MB)
15/06/03 18:34:27 INFO BlockManagerInfo: Added rdd_385_0 in memory on localhost:60836 (size: 1664.0 B, free: 533.1 MB)
15/06/03 18:34:27 INFO BlockManagerMaster: Updated info of block rdd_385_0
15/06/03 18:34:27 INFO Executor: Finished task 0.0 in stage 4617.0 (TID 79). 2693 bytes result sent to driver
15/06/03 18:34:27 INFO TaskSetManager: Finished task 0.0 in stage 4617.0 (TID 79) in 9 ms on localhost (1/1)
15/06/03 18:34:27 INFO TaskSchedulerImpl: Removed TaskSet 4617.0, whose tasks have all completed, from pool
15/06/03 18:34:27 INFO DAGScheduler: Stage 4617 (reduce at VertexRDDImpl.scala:90) finished in 0.009 s
15/06/03 18:34:27 INFO DAGScheduler: Job 19 finished: reduce at VertexRDDImpl.scala:90, took 0.185674 s
15/06/03 18:34:27 INFO Pregel: Pregel finished iteration 17
15/06/03 18:34:27 INFO ZippedPartitionsRDD2: Removing RDD 366 from persistence list
15/06/03 18:34:27 INFO BlockManager: Removing RDD 366
15/06/03 18:34:27 INFO ZippedPartitionsRDD2: Removing RDD 369 from persistence list
15/06/03 18:34:27 INFO BlockManager: Removing block rdd_366_0
15/06/03 18:34:27 INFO MemoryStore: Block rdd_366_0 of size 1664 dropped from memory (free 558754930)
15/06/03 18:34:27 INFO BlockManager: Removing RDD 369
15/06/03 18:34:27 INFO BlockManager: Removing block rdd_369_0
15/06/03 18:34:27 INFO MemoryStore: Block rdd_369_0 of size 1544 dropped from memory (free 558756474)
15/06/03 18:34:27 INFO ZippedPartitionsRDD2: Removing RDD 352 from persistence list
15/06/03 18:34:27 INFO BlockManager: Removing RDD 352
15/06/03 18:34:27 INFO ZippedPartitionsRDD2: Removing RDD 358 from persistence list
15/06/03 18:34:27 INFO BlockManager: Removing block rdd_352_0
15/06/03 18:34:27 INFO MemoryStore: Block rdd_352_0 of size 1496 dropped from memory (free 558757970)
15/06/03 18:34:27 INFO BlockManager: Removing RDD 358
15/06/03 18:34:27 INFO BlockManager: Removing block rdd_358_0
15/06/03 18:34:27 INFO MemoryStore: Block rdd_358_0 of size 2976 dropped from memory (free 558760946)
15/06/03 18:34:27 INFO SparkContext: Starting job: reduce at VertexRDDImpl.scala:90
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 2 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 3 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 5 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 4 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 1 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 0 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 8 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 7 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 6 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 11 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 10 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 9 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 14 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 13 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 12 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 17 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 16 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 15 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 20 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 19 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 18 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 23 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 22 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 21 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 26 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 25 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 24 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 29 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 28 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 27 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 32 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 31 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 30 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 35 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 34 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 33 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 38 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 37 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 36 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 41 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 40 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 39 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 44 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 43 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 42 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 47 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 46 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 45 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 50 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 49 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 48 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 53 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 52 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 51 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 56 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 55 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 54 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 59 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 58 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 57 is 145 bytes
15/06/03 18:34:27 INFO DAGScheduler: Registering RDD 394 (mapPartitions at VertexRDDImpl.scala:218)
15/06/03 18:34:27 INFO DAGScheduler: Registering RDD 398 (mapPartitions at VertexRDDImpl.scala:222)
15/06/03 18:34:27 INFO DAGScheduler: Registering RDD 402 (mapPartitions at GraphImpl.scala:235)
15/06/03 18:34:27 INFO DAGScheduler: Got job 20 (reduce at VertexRDDImpl.scala:90) with 1 output partitions (allowLocal=false)
15/06/03 18:34:27 INFO DAGScheduler: Final stage: Stage 5310(reduce at VertexRDDImpl.scala:90)
15/06/03 18:34:27 INFO DAGScheduler: Parents of final stage: List(Stage 5293, Stage 5279, Stage 5301, Stage 5287, Stage 5309, Stage 5273, Stage 5303, Stage 5295, Stage 5289, Stage 5281, Stage 5275, Stage 5297, Stage 5268, Stage 5283, Stage 5305, Stage 5269, Stage 5299, Stage 5291, Stage 5285, Stage 5277, Stage 5307, Stage 5271)
15/06/03 18:34:27 INFO DAGScheduler: Missing parents: List(Stage 5309)
15/06/03 18:34:27 INFO DAGScheduler: Submitting Stage 5247 (ReplicatedVertexView.updateVertices - shippedVerts true false (broadcast) MapPartitionsRDD[394] at mapPartitions at VertexRDDImpl.scala:218), which has no missing parents
15/06/03 18:34:27 INFO MemoryStore: ensureFreeSpace(16192) called with curMem=392206, maxMem=559153152
15/06/03 18:34:27 INFO MemoryStore: Block broadcast_81 stored as values in memory (estimated size 15.8 KB, free 532.9 MB)
15/06/03 18:34:27 INFO MemoryStore: ensureFreeSpace(7960) called with curMem=408398, maxMem=559153152
15/06/03 18:34:27 INFO MemoryStore: Block broadcast_81_piece0 stored as bytes in memory (estimated size 7.8 KB, free 532.9 MB)
15/06/03 18:34:27 INFO BlockManagerInfo: Added broadcast_81_piece0 in memory on localhost:60836 (size: 7.8 KB, free: 533.1 MB)
15/06/03 18:34:27 INFO BlockManagerMaster: Updated info of block broadcast_81_piece0
15/06/03 18:34:27 INFO SparkContext: Created broadcast 81 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:27 INFO DAGScheduler: Submitting 1 missing tasks from Stage 5247 (ReplicatedVertexView.updateVertices - shippedVerts true false (broadcast) MapPartitionsRDD[394] at mapPartitions at VertexRDDImpl.scala:218)
15/06/03 18:34:27 INFO TaskSchedulerImpl: Adding task set 5247.0 with 1 tasks
15/06/03 18:34:27 INFO DAGScheduler: Submitting Stage 5308 (ReplicatedVertexView.withActiveSet - shippedActives (broadcast) MapPartitionsRDD[398] at mapPartitions at VertexRDDImpl.scala:222), which has no missing parents
15/06/03 18:34:27 INFO TaskSetManager: Starting task 0.0 in stage 5247.0 (TID 80, localhost, PROCESS_LOCAL, 3521 bytes)
15/06/03 18:34:27 INFO Executor: Running task 0.0 in stage 5247.0 (TID 80)
15/06/03 18:34:27 INFO MemoryStore: ensureFreeSpace(15848) called with curMem=416358, maxMem=559153152
15/06/03 18:34:27 INFO MemoryStore: Block broadcast_82 stored as values in memory (estimated size 15.5 KB, free 532.8 MB)
15/06/03 18:34:27 INFO MemoryStore: ensureFreeSpace(7828) called with curMem=432206, maxMem=559153152
15/06/03 18:34:27 INFO MemoryStore: Block broadcast_82_piece0 stored as bytes in memory (estimated size 7.6 KB, free 532.8 MB)
15/06/03 18:34:27 INFO BlockManagerInfo: Added broadcast_82_piece0 in memory on localhost:60836 (size: 7.6 KB, free: 533.1 MB)
15/06/03 18:34:27 INFO BlockManagerMaster: Updated info of block broadcast_82_piece0
15/06/03 18:34:27 INFO SparkContext: Created broadcast 82 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:27 INFO BlockManager: Found block rdd_371_0 locally
15/06/03 18:34:27 INFO DAGScheduler: Submitting 1 missing tasks from Stage 5308 (ReplicatedVertexView.withActiveSet - shippedActives (broadcast) MapPartitionsRDD[398] at mapPartitions at VertexRDDImpl.scala:222)
15/06/03 18:34:27 INFO TaskSchedulerImpl: Adding task set 5308.0 with 1 tasks
15/06/03 18:34:27 INFO CacheManager: Partition rdd_390_0 not found, computing it
15/06/03 18:34:27 INFO BlockManager: Found block rdd_371_0 locally
15/06/03 18:34:27 INFO CacheManager: Partition rdd_388_0 not found, computing it
15/06/03 18:34:27 INFO BlockManager: Found block rdd_371_0 locally
15/06/03 18:34:27 INFO BlockManager: Found block rdd_385_0 locally
15/06/03 18:34:27 INFO MemoryStore: ensureFreeSpace(1544) called with curMem=440034, maxMem=559153152
15/06/03 18:34:27 INFO MemoryStore: Block rdd_388_0 stored as values in memory (estimated size 1544.0 B, free 532.8 MB)
15/06/03 18:34:27 INFO BlockManagerInfo: Added rdd_388_0 in memory on localhost:60836 (size: 1544.0 B, free: 533.1 MB)
15/06/03 18:34:27 INFO BlockManagerMaster: Updated info of block rdd_388_0
15/06/03 18:34:27 INFO MemoryStore: ensureFreeSpace(1496) called with curMem=441578, maxMem=559153152
15/06/03 18:34:27 INFO MemoryStore: Block rdd_390_0 stored as values in memory (estimated size 1496.0 B, free 532.8 MB)
15/06/03 18:34:27 INFO BlockManagerInfo: Added rdd_390_0 in memory on localhost:60836 (size: 1496.0 B, free: 533.1 MB)
15/06/03 18:34:27 INFO BlockManagerMaster: Updated info of block rdd_390_0
15/06/03 18:34:27 INFO Executor: Finished task 0.0 in stage 5247.0 (TID 80). 2627 bytes result sent to driver
15/06/03 18:34:27 INFO TaskSetManager: Starting task 0.0 in stage 5308.0 (TID 81, localhost, PROCESS_LOCAL, 3457 bytes)
15/06/03 18:34:27 INFO TaskSetManager: Finished task 0.0 in stage 5247.0 (TID 80) in 24 ms on localhost (1/1)
15/06/03 18:34:27 INFO Executor: Running task 0.0 in stage 5308.0 (TID 81)
15/06/03 18:34:27 INFO TaskSchedulerImpl: Removed TaskSet 5247.0, whose tasks have all completed, from pool
15/06/03 18:34:27 INFO DAGScheduler: Stage 5247 (mapPartitions at VertexRDDImpl.scala:218) finished in 0.026 s
15/06/03 18:34:27 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:27 INFO DAGScheduler: running: Set(Stage 5308)
15/06/03 18:34:27 INFO DAGScheduler: waiting: Set(Stage 5309, Stage 5310)
15/06/03 18:34:27 INFO DAGScheduler: failed: Set()
15/06/03 18:34:27 INFO DAGScheduler: Missing parents for Stage 5309: List(Stage 5308)
15/06/03 18:34:27 INFO DAGScheduler: Missing parents for Stage 5310: List(Stage 5309)
15/06/03 18:34:27 INFO BlockManager: Found block rdd_388_0 locally
15/06/03 18:34:27 INFO Executor: Finished task 0.0 in stage 5308.0 (TID 81). 2003 bytes result sent to driver
15/06/03 18:34:27 INFO TaskSetManager: Finished task 0.0 in stage 5308.0 (TID 81) in 13 ms on localhost (1/1)
15/06/03 18:34:27 INFO TaskSchedulerImpl: Removed TaskSet 5308.0, whose tasks have all completed, from pool
15/06/03 18:34:27 INFO DAGScheduler: Stage 5308 (mapPartitions at VertexRDDImpl.scala:222) finished in 0.032 s
15/06/03 18:34:27 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:27 INFO DAGScheduler: running: Set()
15/06/03 18:34:27 INFO DAGScheduler: waiting: Set(Stage 5309, Stage 5310)
15/06/03 18:34:27 INFO DAGScheduler: failed: Set()
15/06/03 18:34:27 INFO DAGScheduler: Missing parents for Stage 5309: List()
15/06/03 18:34:27 INFO DAGScheduler: Missing parents for Stage 5310: List(Stage 5309)
15/06/03 18:34:27 INFO DAGScheduler: Submitting Stage 5309 (GraphImpl.aggregateMessages - preAgg MapPartitionsRDD[402] at mapPartitions at GraphImpl.scala:235), which is now runnable
15/06/03 18:34:27 INFO MemoryStore: ensureFreeSpace(11184) called with curMem=443074, maxMem=559153152
15/06/03 18:34:27 INFO MemoryStore: Block broadcast_83 stored as values in memory (estimated size 10.9 KB, free 532.8 MB)
15/06/03 18:34:27 INFO MemoryStore: ensureFreeSpace(6108) called with curMem=454258, maxMem=559153152
15/06/03 18:34:27 INFO MemoryStore: Block broadcast_83_piece0 stored as bytes in memory (estimated size 6.0 KB, free 532.8 MB)
15/06/03 18:34:27 INFO BlockManagerInfo: Added broadcast_83_piece0 in memory on localhost:60836 (size: 6.0 KB, free: 533.1 MB)
15/06/03 18:34:27 INFO BlockManagerMaster: Updated info of block broadcast_83_piece0
15/06/03 18:34:27 INFO SparkContext: Created broadcast 83 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:27 INFO DAGScheduler: Submitting 1 missing tasks from Stage 5309 (GraphImpl.aggregateMessages - preAgg MapPartitionsRDD[402] at mapPartitions at GraphImpl.scala:235)
15/06/03 18:34:27 INFO TaskSchedulerImpl: Adding task set 5309.0 with 1 tasks
15/06/03 18:34:27 INFO TaskSetManager: Starting task 0.0 in stage 5309.0 (TID 82, localhost, PROCESS_LOCAL, 2714 bytes)
15/06/03 18:34:27 INFO Executor: Running task 0.0 in stage 5309.0 (TID 82)
15/06/03 18:34:27 INFO CacheManager: Partition rdd_396_0 not found, computing it
15/06/03 18:34:27 INFO BlockManager: Found block rdd_377_0 locally
15/06/03 18:34:27 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:27 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:27 INFO MemoryStore: ensureFreeSpace(2976) called with curMem=460366, maxMem=559153152
15/06/03 18:34:27 INFO MemoryStore: Block rdd_396_0 stored as values in memory (estimated size 2.9 KB, free 532.8 MB)
15/06/03 18:34:27 INFO BlockManagerInfo: Added rdd_396_0 in memory on localhost:60836 (size: 2.9 KB, free: 533.1 MB)
15/06/03 18:34:27 INFO BlockManagerMaster: Updated info of block rdd_396_0
15/06/03 18:34:27 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:27 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:27 INFO Executor: Finished task 0.0 in stage 5309.0 (TID 82). 2788 bytes result sent to driver
15/06/03 18:34:27 INFO TaskSetManager: Finished task 0.0 in stage 5309.0 (TID 82) in 14 ms on localhost (1/1)
15/06/03 18:34:27 INFO TaskSchedulerImpl: Removed TaskSet 5309.0, whose tasks have all completed, from pool
15/06/03 18:34:27 INFO DAGScheduler: Stage 5309 (mapPartitions at GraphImpl.scala:235) finished in 0.015 s
15/06/03 18:34:27 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:27 INFO DAGScheduler: running: Set()
15/06/03 18:34:27 INFO DAGScheduler: waiting: Set(Stage 5310)
15/06/03 18:34:27 INFO DAGScheduler: failed: Set()
15/06/03 18:34:27 INFO DAGScheduler: Missing parents for Stage 5310: List()
15/06/03 18:34:27 INFO DAGScheduler: Submitting Stage 5310 (MapPartitionsRDD[406] at map at VertexRDDImpl.scala:90), which is now runnable
15/06/03 18:34:27 INFO MemoryStore: ensureFreeSpace(16472) called with curMem=463342, maxMem=559153152
15/06/03 18:34:27 INFO MemoryStore: Block broadcast_84 stored as values in memory (estimated size 16.1 KB, free 532.8 MB)
15/06/03 18:34:27 INFO MemoryStore: ensureFreeSpace(8030) called with curMem=479814, maxMem=559153152
15/06/03 18:34:27 INFO MemoryStore: Block broadcast_84_piece0 stored as bytes in memory (estimated size 7.8 KB, free 532.8 MB)
15/06/03 18:34:27 INFO BlockManagerInfo: Added broadcast_84_piece0 in memory on localhost:60836 (size: 7.8 KB, free: 533.1 MB)
15/06/03 18:34:27 INFO BlockManagerMaster: Updated info of block broadcast_84_piece0
15/06/03 18:34:27 INFO SparkContext: Created broadcast 84 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:27 INFO DAGScheduler: Submitting 1 missing tasks from Stage 5310 (MapPartitionsRDD[406] at map at VertexRDDImpl.scala:90)
15/06/03 18:34:27 INFO TaskSchedulerImpl: Adding task set 5310.0 with 1 tasks
15/06/03 18:34:27 INFO TaskSetManager: Starting task 0.0 in stage 5310.0 (TID 83, localhost, PROCESS_LOCAL, 3541 bytes)
15/06/03 18:34:27 INFO Executor: Running task 0.0 in stage 5310.0 (TID 83)
15/06/03 18:34:27 INFO CacheManager: Partition rdd_404_0 not found, computing it
15/06/03 18:34:27 INFO BlockManager: Found block rdd_390_0 locally
15/06/03 18:34:27 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:27 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:27 INFO MemoryStore: ensureFreeSpace(1664) called with curMem=487844, maxMem=559153152
15/06/03 18:34:27 INFO MemoryStore: Block rdd_404_0 stored as values in memory (estimated size 1664.0 B, free 532.8 MB)
15/06/03 18:34:27 INFO BlockManagerInfo: Added rdd_404_0 in memory on localhost:60836 (size: 1664.0 B, free: 533.1 MB)
15/06/03 18:34:27 INFO BlockManagerMaster: Updated info of block rdd_404_0
15/06/03 18:34:27 INFO Executor: Finished task 0.0 in stage 5310.0 (TID 83). 2693 bytes result sent to driver
15/06/03 18:34:27 INFO TaskSetManager: Finished task 0.0 in stage 5310.0 (TID 83) in 9 ms on localhost (1/1)
15/06/03 18:34:27 INFO TaskSchedulerImpl: Removed TaskSet 5310.0, whose tasks have all completed, from pool
15/06/03 18:34:27 INFO DAGScheduler: Stage 5310 (reduce at VertexRDDImpl.scala:90) finished in 0.009 s
15/06/03 18:34:27 INFO DAGScheduler: Job 20 finished: reduce at VertexRDDImpl.scala:90, took 0.182664 s
15/06/03 18:34:27 INFO Pregel: Pregel finished iteration 18
15/06/03 18:34:27 INFO ZippedPartitionsRDD2: Removing RDD 385 from persistence list
15/06/03 18:34:27 INFO BlockManager: Removing RDD 385
15/06/03 18:34:27 INFO BlockManager: Removing block rdd_385_0
15/06/03 18:34:27 INFO MemoryStore: Block rdd_385_0 of size 1664 dropped from memory (free 558665308)
15/06/03 18:34:27 INFO ZippedPartitionsRDD2: Removing RDD 388 from persistence list
15/06/03 18:34:27 INFO BlockManager: Removing RDD 388
15/06/03 18:34:27 INFO BlockManager: Removing block rdd_388_0
15/06/03 18:34:27 INFO ZippedPartitionsRDD2: Removing RDD 371 from persistence list
15/06/03 18:34:27 INFO MemoryStore: Block rdd_388_0 of size 1544 dropped from memory (free 558666852)
15/06/03 18:34:27 INFO BlockManager: Removing RDD 371
15/06/03 18:34:27 INFO ZippedPartitionsRDD2: Removing RDD 377 from persistence list
15/06/03 18:34:27 INFO BlockManager: Removing block rdd_371_0
15/06/03 18:34:27 INFO MemoryStore: Block rdd_371_0 of size 1496 dropped from memory (free 558668348)
15/06/03 18:34:27 INFO BlockManager: Removing RDD 377
15/06/03 18:34:27 INFO BlockManager: Removing block rdd_377_0
15/06/03 18:34:27 INFO MemoryStore: Block rdd_377_0 of size 2976 dropped from memory (free 558671324)
15/06/03 18:34:27 INFO SparkContext: Starting job: reduce at VertexRDDImpl.scala:90
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 2 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 3 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 5 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 4 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 1 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 0 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 8 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 7 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 6 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 11 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 10 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 9 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 14 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 13 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 12 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 17 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 16 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 15 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 20 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 19 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 18 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 23 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 22 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 21 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 26 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 25 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 24 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 29 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 28 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 27 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 32 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 31 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 30 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 35 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 34 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 33 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 38 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 37 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 36 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 41 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 40 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 39 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 44 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 43 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 42 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 47 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 46 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 45 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 50 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 49 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 48 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 53 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 52 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 51 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 56 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 55 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 54 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 59 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 58 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 57 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 62 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 61 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 60 is 145 bytes
15/06/03 18:34:27 INFO DAGScheduler: Registering RDD 413 (mapPartitions at VertexRDDImpl.scala:218)
15/06/03 18:34:27 INFO DAGScheduler: Registering RDD 417 (mapPartitions at VertexRDDImpl.scala:222)
15/06/03 18:34:27 INFO DAGScheduler: Registering RDD 421 (mapPartitions at GraphImpl.scala:235)
15/06/03 18:34:27 INFO DAGScheduler: Got job 21 (reduce at VertexRDDImpl.scala:90) with 1 output partitions (allowLocal=false)
15/06/03 18:34:27 INFO DAGScheduler: Final stage: Stage 6069(reduce at VertexRDDImpl.scala:90)
15/06/03 18:34:27 INFO DAGScheduler: Parents of final stage: List(Stage 6062, Stage 6026, Stage 6056, Stage 6048, Stage 6042, Stage 6034, Stage 6064, Stage 6028, Stage 6050, Stage 6036, Stage 6058, Stage 6044, Stage 6066, Stage 6030, Stage 6060, Stage 6052, Stage 6046, Stage 6038, Stage 6068, Stage 6032, Stage 6054, Stage 6025, Stage 6040)
15/06/03 18:34:27 INFO DAGScheduler: Missing parents: List(Stage 6068)
15/06/03 18:34:27 INFO DAGScheduler: Submitting Stage 6003 (ReplicatedVertexView.updateVertices - shippedVerts true false (broadcast) MapPartitionsRDD[413] at mapPartitions at VertexRDDImpl.scala:218), which has no missing parents
15/06/03 18:34:27 INFO MemoryStore: ensureFreeSpace(16712) called with curMem=481828, maxMem=559153152
15/06/03 18:34:27 INFO MemoryStore: Block broadcast_85 stored as values in memory (estimated size 16.3 KB, free 532.8 MB)
15/06/03 18:34:27 INFO MemoryStore: ensureFreeSpace(8096) called with curMem=498540, maxMem=559153152
15/06/03 18:34:27 INFO MemoryStore: Block broadcast_85_piece0 stored as bytes in memory (estimated size 7.9 KB, free 532.8 MB)
15/06/03 18:34:27 INFO BlockManagerInfo: Added broadcast_85_piece0 in memory on localhost:60836 (size: 7.9 KB, free: 533.1 MB)
15/06/03 18:34:27 INFO BlockManagerMaster: Updated info of block broadcast_85_piece0
15/06/03 18:34:27 INFO SparkContext: Created broadcast 85 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:27 INFO DAGScheduler: Submitting 1 missing tasks from Stage 6003 (ReplicatedVertexView.updateVertices - shippedVerts true false (broadcast) MapPartitionsRDD[413] at mapPartitions at VertexRDDImpl.scala:218)
15/06/03 18:34:27 INFO TaskSchedulerImpl: Adding task set 6003.0 with 1 tasks
15/06/03 18:34:27 INFO DAGScheduler: Submitting Stage 6067 (ReplicatedVertexView.withActiveSet - shippedActives (broadcast) MapPartitionsRDD[417] at mapPartitions at VertexRDDImpl.scala:222), which has no missing parents
15/06/03 18:34:27 INFO TaskSetManager: Starting task 0.0 in stage 6003.0 (TID 84, localhost, PROCESS_LOCAL, 3626 bytes)
15/06/03 18:34:27 INFO Executor: Running task 0.0 in stage 6003.0 (TID 84)
15/06/03 18:34:27 INFO MemoryStore: ensureFreeSpace(16376) called with curMem=506636, maxMem=559153152
15/06/03 18:34:27 INFO MemoryStore: Block broadcast_86 stored as values in memory (estimated size 16.0 KB, free 532.8 MB)
15/06/03 18:34:27 INFO MemoryStore: ensureFreeSpace(7945) called with curMem=523012, maxMem=559153152
15/06/03 18:34:27 INFO MemoryStore: Block broadcast_86_piece0 stored as bytes in memory (estimated size 7.8 KB, free 532.7 MB)
15/06/03 18:34:27 INFO BlockManagerInfo: Added broadcast_86_piece0 in memory on localhost:60836 (size: 7.8 KB, free: 533.1 MB)
15/06/03 18:34:27 INFO BlockManagerMaster: Updated info of block broadcast_86_piece0
15/06/03 18:34:27 INFO SparkContext: Created broadcast 86 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:27 INFO DAGScheduler: Submitting 1 missing tasks from Stage 6067 (ReplicatedVertexView.withActiveSet - shippedActives (broadcast) MapPartitionsRDD[417] at mapPartitions at VertexRDDImpl.scala:222)
15/06/03 18:34:27 INFO BlockManager: Found block rdd_390_0 locally
15/06/03 18:34:27 INFO TaskSchedulerImpl: Adding task set 6067.0 with 1 tasks
15/06/03 18:34:27 INFO CacheManager: Partition rdd_409_0 not found, computing it
15/06/03 18:34:27 INFO BlockManager: Found block rdd_390_0 locally
15/06/03 18:34:27 INFO CacheManager: Partition rdd_407_0 not found, computing it
15/06/03 18:34:27 INFO BlockManager: Found block rdd_390_0 locally
15/06/03 18:34:27 INFO BlockManager: Found block rdd_404_0 locally
15/06/03 18:34:27 INFO MemoryStore: ensureFreeSpace(1544) called with curMem=530957, maxMem=559153152
15/06/03 18:34:27 INFO MemoryStore: Block rdd_407_0 stored as values in memory (estimated size 1544.0 B, free 532.7 MB)
15/06/03 18:34:27 INFO BlockManagerInfo: Added rdd_407_0 in memory on localhost:60836 (size: 1544.0 B, free: 533.1 MB)
15/06/03 18:34:27 INFO BlockManagerMaster: Updated info of block rdd_407_0
15/06/03 18:34:27 INFO MemoryStore: ensureFreeSpace(1496) called with curMem=532501, maxMem=559153152
15/06/03 18:34:27 INFO MemoryStore: Block rdd_409_0 stored as values in memory (estimated size 1496.0 B, free 532.7 MB)
15/06/03 18:34:27 INFO BlockManagerInfo: Added rdd_409_0 in memory on localhost:60836 (size: 1496.0 B, free: 533.1 MB)
15/06/03 18:34:27 INFO BlockManagerMaster: Updated info of block rdd_409_0
15/06/03 18:34:27 INFO Executor: Finished task 0.0 in stage 6003.0 (TID 84). 2627 bytes result sent to driver
15/06/03 18:34:27 INFO TaskSetManager: Starting task 0.0 in stage 6067.0 (TID 85, localhost, PROCESS_LOCAL, 3562 bytes)
15/06/03 18:34:27 INFO Executor: Running task 0.0 in stage 6067.0 (TID 85)
15/06/03 18:34:27 INFO TaskSetManager: Finished task 0.0 in stage 6003.0 (TID 84) in 27 ms on localhost (1/1)
15/06/03 18:34:27 INFO TaskSchedulerImpl: Removed TaskSet 6003.0, whose tasks have all completed, from pool
15/06/03 18:34:27 INFO DAGScheduler: Stage 6003 (mapPartitions at VertexRDDImpl.scala:218) finished in 0.027 s
15/06/03 18:34:27 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:27 INFO DAGScheduler: running: Set(Stage 6067)
15/06/03 18:34:27 INFO DAGScheduler: waiting: Set(Stage 6068, Stage 6069)
15/06/03 18:34:27 INFO DAGScheduler: failed: Set()
15/06/03 18:34:27 INFO DAGScheduler: Missing parents for Stage 6068: List(Stage 6067)
15/06/03 18:34:27 INFO DAGScheduler: Missing parents for Stage 6069: List(Stage 6068)
15/06/03 18:34:27 INFO BlockManager: Found block rdd_407_0 locally
15/06/03 18:34:27 INFO Executor: Finished task 0.0 in stage 6067.0 (TID 85). 2003 bytes result sent to driver
15/06/03 18:34:27 INFO TaskSetManager: Finished task 0.0 in stage 6067.0 (TID 85) in 12 ms on localhost (1/1)
15/06/03 18:34:27 INFO TaskSchedulerImpl: Removed TaskSet 6067.0, whose tasks have all completed, from pool
15/06/03 18:34:27 INFO DAGScheduler: Stage 6067 (mapPartitions at VertexRDDImpl.scala:222) finished in 0.034 s
15/06/03 18:34:27 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:27 INFO DAGScheduler: running: Set()
15/06/03 18:34:27 INFO DAGScheduler: waiting: Set(Stage 6068, Stage 6069)
15/06/03 18:34:27 INFO DAGScheduler: failed: Set()
15/06/03 18:34:27 INFO DAGScheduler: Missing parents for Stage 6068: List()
15/06/03 18:34:27 INFO DAGScheduler: Missing parents for Stage 6069: List(Stage 6068)
15/06/03 18:34:27 INFO DAGScheduler: Submitting Stage 6068 (GraphImpl.aggregateMessages - preAgg MapPartitionsRDD[421] at mapPartitions at GraphImpl.scala:235), which is now runnable
15/06/03 18:34:27 INFO MemoryStore: ensureFreeSpace(11408) called with curMem=533997, maxMem=559153152
15/06/03 18:34:27 INFO MemoryStore: Block broadcast_87 stored as values in memory (estimated size 11.1 KB, free 532.7 MB)
15/06/03 18:34:27 INFO MemoryStore: ensureFreeSpace(6232) called with curMem=545405, maxMem=559153152
15/06/03 18:34:27 INFO MemoryStore: Block broadcast_87_piece0 stored as bytes in memory (estimated size 6.1 KB, free 532.7 MB)
15/06/03 18:34:27 INFO BlockManagerInfo: Added broadcast_87_piece0 in memory on localhost:60836 (size: 6.1 KB, free: 533.1 MB)
15/06/03 18:34:27 INFO BlockManagerMaster: Updated info of block broadcast_87_piece0
15/06/03 18:34:27 INFO SparkContext: Created broadcast 87 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:27 INFO DAGScheduler: Submitting 1 missing tasks from Stage 6068 (GraphImpl.aggregateMessages - preAgg MapPartitionsRDD[421] at mapPartitions at GraphImpl.scala:235)
15/06/03 18:34:27 INFO TaskSchedulerImpl: Adding task set 6068.0 with 1 tasks
15/06/03 18:34:27 INFO TaskSetManager: Starting task 0.0 in stage 6068.0 (TID 86, localhost, PROCESS_LOCAL, 2755 bytes)
15/06/03 18:34:27 INFO Executor: Running task 0.0 in stage 6068.0 (TID 86)
15/06/03 18:34:27 INFO CacheManager: Partition rdd_415_0 not found, computing it
15/06/03 18:34:27 INFO BlockManager: Found block rdd_396_0 locally
15/06/03 18:34:27 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:27 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:27 INFO MemoryStore: ensureFreeSpace(2976) called with curMem=551637, maxMem=559153152
15/06/03 18:34:27 INFO MemoryStore: Block rdd_415_0 stored as values in memory (estimated size 2.9 KB, free 532.7 MB)
15/06/03 18:34:27 INFO BlockManagerInfo: Added rdd_415_0 in memory on localhost:60836 (size: 2.9 KB, free: 533.1 MB)
15/06/03 18:34:27 INFO BlockManagerMaster: Updated info of block rdd_415_0
15/06/03 18:34:27 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:27 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:27 INFO Executor: Finished task 0.0 in stage 6068.0 (TID 86). 2788 bytes result sent to driver
15/06/03 18:34:27 INFO TaskSetManager: Finished task 0.0 in stage 6068.0 (TID 86) in 16 ms on localhost (1/1)
15/06/03 18:34:27 INFO TaskSchedulerImpl: Removed TaskSet 6068.0, whose tasks have all completed, from pool
15/06/03 18:34:27 INFO DAGScheduler: Stage 6068 (mapPartitions at GraphImpl.scala:235) finished in 0.016 s
15/06/03 18:34:27 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:27 INFO DAGScheduler: running: Set()
15/06/03 18:34:27 INFO DAGScheduler: waiting: Set(Stage 6069)
15/06/03 18:34:27 INFO DAGScheduler: failed: Set()
15/06/03 18:34:27 INFO DAGScheduler: Missing parents for Stage 6069: List()
15/06/03 18:34:27 INFO DAGScheduler: Submitting Stage 6069 (MapPartitionsRDD[425] at map at VertexRDDImpl.scala:90), which is now runnable
15/06/03 18:34:27 INFO MemoryStore: ensureFreeSpace(17000) called with curMem=554613, maxMem=559153152
15/06/03 18:34:27 INFO MemoryStore: Block broadcast_88 stored as values in memory (estimated size 16.6 KB, free 532.7 MB)
15/06/03 18:34:27 INFO MemoryStore: ensureFreeSpace(8279) called with curMem=571613, maxMem=559153152
15/06/03 18:34:27 INFO MemoryStore: Block broadcast_88_piece0 stored as bytes in memory (estimated size 8.1 KB, free 532.7 MB)
15/06/03 18:34:27 INFO BlockManagerInfo: Added broadcast_88_piece0 in memory on localhost:60836 (size: 8.1 KB, free: 533.1 MB)
15/06/03 18:34:27 INFO BlockManagerMaster: Updated info of block broadcast_88_piece0
15/06/03 18:34:27 INFO SparkContext: Created broadcast 88 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:27 INFO DAGScheduler: Submitting 1 missing tasks from Stage 6069 (MapPartitionsRDD[425] at map at VertexRDDImpl.scala:90)
15/06/03 18:34:27 INFO TaskSchedulerImpl: Adding task set 6069.0 with 1 tasks
15/06/03 18:34:27 INFO TaskSetManager: Starting task 0.0 in stage 6069.0 (TID 87, localhost, PROCESS_LOCAL, 3646 bytes)
15/06/03 18:34:27 INFO Executor: Running task 0.0 in stage 6069.0 (TID 87)
15/06/03 18:34:27 INFO CacheManager: Partition rdd_423_0 not found, computing it
15/06/03 18:34:27 INFO BlockManager: Found block rdd_409_0 locally
15/06/03 18:34:27 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:27 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:27 INFO MemoryStore: ensureFreeSpace(1664) called with curMem=579892, maxMem=559153152
15/06/03 18:34:27 INFO MemoryStore: Block rdd_423_0 stored as values in memory (estimated size 1664.0 B, free 532.7 MB)
15/06/03 18:34:27 INFO BlockManagerInfo: Added rdd_423_0 in memory on localhost:60836 (size: 1664.0 B, free: 533.1 MB)
15/06/03 18:34:27 INFO BlockManagerMaster: Updated info of block rdd_423_0
15/06/03 18:34:27 INFO Executor: Finished task 0.0 in stage 6069.0 (TID 87). 2693 bytes result sent to driver
15/06/03 18:34:27 INFO TaskSetManager: Finished task 0.0 in stage 6069.0 (TID 87) in 10 ms on localhost (1/1)
15/06/03 18:34:27 INFO TaskSchedulerImpl: Removed TaskSet 6069.0, whose tasks have all completed, from pool
15/06/03 18:34:27 INFO DAGScheduler: Stage 6069 (reduce at VertexRDDImpl.scala:90) finished in 0.010 s
15/06/03 18:34:27 INFO DAGScheduler: Job 21 finished: reduce at VertexRDDImpl.scala:90, took 0.186025 s
15/06/03 18:34:27 INFO Pregel: Pregel finished iteration 19
15/06/03 18:34:27 INFO ZippedPartitionsRDD2: Removing RDD 404 from persistence list
15/06/03 18:34:27 INFO BlockManager: Removing RDD 404
15/06/03 18:34:27 INFO ZippedPartitionsRDD2: Removing RDD 407 from persistence list
15/06/03 18:34:27 INFO BlockManager: Removing block rdd_404_0
15/06/03 18:34:27 INFO MemoryStore: Block rdd_404_0 of size 1664 dropped from memory (free 558573260)
15/06/03 18:34:27 INFO BlockManager: Removing RDD 407
15/06/03 18:34:27 INFO BlockManager: Removing block rdd_407_0
15/06/03 18:34:27 INFO MemoryStore: Block rdd_407_0 of size 1544 dropped from memory (free 558574804)
15/06/03 18:34:27 INFO ZippedPartitionsRDD2: Removing RDD 390 from persistence list
15/06/03 18:34:27 INFO BlockManager: Removing RDD 390
15/06/03 18:34:27 INFO ZippedPartitionsRDD2: Removing RDD 396 from persistence list
15/06/03 18:34:27 INFO BlockManager: Removing block rdd_390_0
15/06/03 18:34:27 INFO MemoryStore: Block rdd_390_0 of size 1496 dropped from memory (free 558576300)
15/06/03 18:34:27 INFO BlockManager: Removing RDD 396
15/06/03 18:34:27 INFO BlockManager: Removing block rdd_396_0
15/06/03 18:34:27 INFO MemoryStore: Block rdd_396_0 of size 2976 dropped from memory (free 558579276)
15/06/03 18:34:27 INFO SparkContext: Starting job: reduce at VertexRDDImpl.scala:90
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 2 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 3 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 5 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 4 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 1 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 0 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 8 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 7 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 6 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 11 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 10 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 9 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 14 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 13 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 12 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 17 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 16 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 15 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 20 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 19 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 18 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 23 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 22 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 21 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 26 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 25 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 24 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 29 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 28 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 27 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 32 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 31 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 30 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 35 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 34 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 33 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 38 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 37 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 36 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 41 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 40 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 39 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 44 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 43 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 42 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 47 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 46 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 45 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 50 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 49 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 48 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 53 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 52 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 51 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 56 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 55 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 54 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 59 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 58 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 57 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 62 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 61 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 60 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 65 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 64 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 63 is 145 bytes
15/06/03 18:34:27 INFO DAGScheduler: Registering RDD 432 (mapPartitions at VertexRDDImpl.scala:218)
15/06/03 18:34:27 INFO DAGScheduler: Registering RDD 436 (mapPartitions at VertexRDDImpl.scala:222)
15/06/03 18:34:27 INFO DAGScheduler: Registering RDD 440 (mapPartitions at GraphImpl.scala:235)
15/06/03 18:34:27 INFO DAGScheduler: Got job 22 (reduce at VertexRDDImpl.scala:90) with 1 output partitions (allowLocal=false)
15/06/03 18:34:27 INFO DAGScheduler: Final stage: Stage 6897(reduce at VertexRDDImpl.scala:90)
15/06/03 18:34:27 INFO DAGScheduler: Parents of final stage: List(Stage 6890, Stage 6854, Stage 6876, Stage 6862, Stage 6884, Stage 6870, Stage 6892, Stage 6856, Stage 6886, Stage 6878, Stage 6872, Stage 6864, Stage 6894, Stage 6858, Stage 6880, Stage 6851, Stage 6866, Stage 6896, Stage 6888, Stage 6852, Stage 6882, Stage 6874, Stage 6868, Stage 6860)
15/06/03 18:34:27 INFO DAGScheduler: Missing parents: List(Stage 6896)
15/06/03 18:34:27 INFO DAGScheduler: Submitting Stage 6828 (ReplicatedVertexView.updateVertices - shippedVerts true false (broadcast) MapPartitionsRDD[432] at mapPartitions at VertexRDDImpl.scala:218), which has no missing parents
15/06/03 18:34:27 INFO MemoryStore: ensureFreeSpace(17240) called with curMem=573876, maxMem=559153152
15/06/03 18:34:27 INFO MemoryStore: Block broadcast_89 stored as values in memory (estimated size 16.8 KB, free 532.7 MB)
15/06/03 18:34:27 INFO MemoryStore: ensureFreeSpace(8380) called with curMem=591116, maxMem=559153152
15/06/03 18:34:27 INFO MemoryStore: Block broadcast_89_piece0 stored as bytes in memory (estimated size 8.2 KB, free 532.7 MB)
15/06/03 18:34:27 INFO BlockManagerInfo: Added broadcast_89_piece0 in memory on localhost:60836 (size: 8.2 KB, free: 533.1 MB)
15/06/03 18:34:27 INFO BlockManagerMaster: Updated info of block broadcast_89_piece0
15/06/03 18:34:27 INFO SparkContext: Created broadcast 89 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:27 INFO DAGScheduler: Submitting 1 missing tasks from Stage 6828 (ReplicatedVertexView.updateVertices - shippedVerts true false (broadcast) MapPartitionsRDD[432] at mapPartitions at VertexRDDImpl.scala:218)
15/06/03 18:34:27 INFO TaskSchedulerImpl: Adding task set 6828.0 with 1 tasks
15/06/03 18:34:27 INFO DAGScheduler: Submitting Stage 6895 (ReplicatedVertexView.withActiveSet - shippedActives (broadcast) MapPartitionsRDD[436] at mapPartitions at VertexRDDImpl.scala:222), which has no missing parents
15/06/03 18:34:27 INFO TaskSetManager: Starting task 0.0 in stage 6828.0 (TID 88, localhost, PROCESS_LOCAL, 3731 bytes)
15/06/03 18:34:27 INFO Executor: Running task 0.0 in stage 6828.0 (TID 88)
15/06/03 18:34:27 INFO MemoryStore: ensureFreeSpace(16896) called with curMem=599496, maxMem=559153152
15/06/03 18:34:27 INFO MemoryStore: Block broadcast_90 stored as values in memory (estimated size 16.5 KB, free 532.7 MB)
15/06/03 18:34:27 INFO BlockManager: Found block rdd_409_0 locally
15/06/03 18:34:27 INFO MemoryStore: ensureFreeSpace(8237) called with curMem=616392, maxMem=559153152
15/06/03 18:34:27 INFO MemoryStore: Block broadcast_90_piece0 stored as bytes in memory (estimated size 8.0 KB, free 532.7 MB)
15/06/03 18:34:27 INFO CacheManager: Partition rdd_428_0 not found, computing it
15/06/03 18:34:27 INFO BlockManager: Found block rdd_409_0 locally
15/06/03 18:34:27 INFO CacheManager: Partition rdd_426_0 not found, computing it
15/06/03 18:34:27 INFO BlockManager: Found block rdd_409_0 locally
15/06/03 18:34:27 INFO BlockManager: Found block rdd_423_0 locally
15/06/03 18:34:27 INFO BlockManagerInfo: Added broadcast_90_piece0 in memory on localhost:60836 (size: 8.0 KB, free: 533.1 MB)
15/06/03 18:34:27 INFO BlockManagerMaster: Updated info of block broadcast_90_piece0
15/06/03 18:34:27 INFO MemoryStore: ensureFreeSpace(1544) called with curMem=624629, maxMem=559153152
15/06/03 18:34:27 INFO MemoryStore: Block rdd_426_0 stored as values in memory (estimated size 1544.0 B, free 532.7 MB)
15/06/03 18:34:27 INFO SparkContext: Created broadcast 90 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:27 INFO DAGScheduler: Submitting 1 missing tasks from Stage 6895 (ReplicatedVertexView.withActiveSet - shippedActives (broadcast) MapPartitionsRDD[436] at mapPartitions at VertexRDDImpl.scala:222)
15/06/03 18:34:27 INFO BlockManagerInfo: Added rdd_426_0 in memory on localhost:60836 (size: 1544.0 B, free: 533.1 MB)
15/06/03 18:34:27 INFO TaskSchedulerImpl: Adding task set 6895.0 with 1 tasks
15/06/03 18:34:27 INFO BlockManagerMaster: Updated info of block rdd_426_0
15/06/03 18:34:27 INFO MemoryStore: ensureFreeSpace(1496) called with curMem=626173, maxMem=559153152
15/06/03 18:34:27 INFO MemoryStore: Block rdd_428_0 stored as values in memory (estimated size 1496.0 B, free 532.7 MB)
15/06/03 18:34:27 INFO BlockManagerInfo: Added rdd_428_0 in memory on localhost:60836 (size: 1496.0 B, free: 533.1 MB)
15/06/03 18:34:27 INFO BlockManagerMaster: Updated info of block rdd_428_0
15/06/03 18:34:27 INFO Executor: Finished task 0.0 in stage 6828.0 (TID 88). 2627 bytes result sent to driver
15/06/03 18:34:27 INFO TaskSetManager: Starting task 0.0 in stage 6895.0 (TID 89, localhost, PROCESS_LOCAL, 3667 bytes)
15/06/03 18:34:27 INFO Executor: Running task 0.0 in stage 6895.0 (TID 89)
15/06/03 18:34:27 INFO TaskSetManager: Finished task 0.0 in stage 6828.0 (TID 88) in 22 ms on localhost (1/1)
15/06/03 18:34:27 INFO TaskSchedulerImpl: Removed TaskSet 6828.0, whose tasks have all completed, from pool
15/06/03 18:34:27 INFO DAGScheduler: Stage 6828 (mapPartitions at VertexRDDImpl.scala:218) finished in 0.023 s
15/06/03 18:34:27 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:27 INFO DAGScheduler: running: Set(Stage 6895)
15/06/03 18:34:27 INFO DAGScheduler: waiting: Set(Stage 6897, Stage 6896)
15/06/03 18:34:27 INFO DAGScheduler: failed: Set()
15/06/03 18:34:27 INFO DAGScheduler: Missing parents for Stage 6897: List(Stage 6896)
15/06/03 18:34:27 INFO DAGScheduler: Missing parents for Stage 6896: List(Stage 6895)
15/06/03 18:34:27 INFO BlockManager: Found block rdd_426_0 locally
15/06/03 18:34:27 INFO Executor: Finished task 0.0 in stage 6895.0 (TID 89). 2003 bytes result sent to driver
15/06/03 18:34:27 INFO TaskSetManager: Finished task 0.0 in stage 6895.0 (TID 89) in 15 ms on localhost (1/1)
15/06/03 18:34:27 INFO TaskSchedulerImpl: Removed TaskSet 6895.0, whose tasks have all completed, from pool
15/06/03 18:34:27 INFO DAGScheduler: Stage 6895 (mapPartitions at VertexRDDImpl.scala:222) finished in 0.027 s
15/06/03 18:34:27 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:27 INFO DAGScheduler: running: Set()
15/06/03 18:34:27 INFO DAGScheduler: waiting: Set(Stage 6897, Stage 6896)
15/06/03 18:34:27 INFO DAGScheduler: failed: Set()
15/06/03 18:34:27 INFO DAGScheduler: Missing parents for Stage 6897: List(Stage 6896)
15/06/03 18:34:27 INFO DAGScheduler: Missing parents for Stage 6896: List()
15/06/03 18:34:27 INFO DAGScheduler: Submitting Stage 6896 (GraphImpl.aggregateMessages - preAgg MapPartitionsRDD[440] at mapPartitions at GraphImpl.scala:235), which is now runnable
15/06/03 18:34:27 INFO MemoryStore: ensureFreeSpace(11640) called with curMem=627669, maxMem=559153152
15/06/03 18:34:27 INFO MemoryStore: Block broadcast_91 stored as values in memory (estimated size 11.4 KB, free 532.6 MB)
15/06/03 18:34:27 INFO MemoryStore: ensureFreeSpace(6413) called with curMem=639309, maxMem=559153152
15/06/03 18:34:27 INFO MemoryStore: Block broadcast_91_piece0 stored as bytes in memory (estimated size 6.3 KB, free 532.6 MB)
15/06/03 18:34:27 INFO BlockManagerInfo: Added broadcast_91_piece0 in memory on localhost:60836 (size: 6.3 KB, free: 533.1 MB)
15/06/03 18:34:27 INFO BlockManagerMaster: Updated info of block broadcast_91_piece0
15/06/03 18:34:27 INFO SparkContext: Created broadcast 91 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:27 INFO DAGScheduler: Submitting 1 missing tasks from Stage 6896 (GraphImpl.aggregateMessages - preAgg MapPartitionsRDD[440] at mapPartitions at GraphImpl.scala:235)
15/06/03 18:34:27 INFO TaskSchedulerImpl: Adding task set 6896.0 with 1 tasks
15/06/03 18:34:27 INFO TaskSetManager: Starting task 0.0 in stage 6896.0 (TID 90, localhost, PROCESS_LOCAL, 2796 bytes)
15/06/03 18:34:27 INFO Executor: Running task 0.0 in stage 6896.0 (TID 90)
15/06/03 18:34:27 INFO CacheManager: Partition rdd_434_0 not found, computing it
15/06/03 18:34:27 INFO BlockManager: Found block rdd_415_0 locally
15/06/03 18:34:27 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:27 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:27 INFO MemoryStore: ensureFreeSpace(2976) called with curMem=645722, maxMem=559153152
15/06/03 18:34:27 INFO MemoryStore: Block rdd_434_0 stored as values in memory (estimated size 2.9 KB, free 532.6 MB)
15/06/03 18:34:27 INFO BlockManagerInfo: Added rdd_434_0 in memory on localhost:60836 (size: 2.9 KB, free: 533.1 MB)
15/06/03 18:34:27 INFO BlockManagerMaster: Updated info of block rdd_434_0
15/06/03 18:34:27 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:27 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:27 INFO Executor: Finished task 0.0 in stage 6896.0 (TID 90). 2788 bytes result sent to driver
15/06/03 18:34:27 INFO TaskSetManager: Finished task 0.0 in stage 6896.0 (TID 90) in 15 ms on localhost (1/1)
15/06/03 18:34:27 INFO TaskSchedulerImpl: Removed TaskSet 6896.0, whose tasks have all completed, from pool
15/06/03 18:34:27 INFO DAGScheduler: Stage 6896 (mapPartitions at GraphImpl.scala:235) finished in 0.016 s
15/06/03 18:34:27 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:27 INFO DAGScheduler: running: Set()
15/06/03 18:34:27 INFO DAGScheduler: waiting: Set(Stage 6897)
15/06/03 18:34:27 INFO DAGScheduler: failed: Set()
15/06/03 18:34:27 INFO DAGScheduler: Missing parents for Stage 6897: List()
15/06/03 18:34:27 INFO DAGScheduler: Submitting Stage 6897 (MapPartitionsRDD[444] at map at VertexRDDImpl.scala:90), which is now runnable
15/06/03 18:34:27 INFO MemoryStore: ensureFreeSpace(17528) called with curMem=648698, maxMem=559153152
15/06/03 18:34:27 INFO MemoryStore: Block broadcast_92 stored as values in memory (estimated size 17.1 KB, free 532.6 MB)
15/06/03 18:34:27 INFO MemoryStore: ensureFreeSpace(8572) called with curMem=666226, maxMem=559153152
15/06/03 18:34:27 INFO MemoryStore: Block broadcast_92_piece0 stored as bytes in memory (estimated size 8.4 KB, free 532.6 MB)
15/06/03 18:34:27 INFO BlockManagerInfo: Added broadcast_92_piece0 in memory on localhost:60836 (size: 8.4 KB, free: 533.1 MB)
15/06/03 18:34:27 INFO BlockManagerMaster: Updated info of block broadcast_92_piece0
15/06/03 18:34:27 INFO SparkContext: Created broadcast 92 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:27 INFO DAGScheduler: Submitting 1 missing tasks from Stage 6897 (MapPartitionsRDD[444] at map at VertexRDDImpl.scala:90)
15/06/03 18:34:27 INFO TaskSchedulerImpl: Adding task set 6897.0 with 1 tasks
15/06/03 18:34:27 INFO TaskSetManager: Starting task 0.0 in stage 6897.0 (TID 91, localhost, PROCESS_LOCAL, 3751 bytes)
15/06/03 18:34:27 INFO Executor: Running task 0.0 in stage 6897.0 (TID 91)
15/06/03 18:34:27 INFO CacheManager: Partition rdd_442_0 not found, computing it
15/06/03 18:34:27 INFO BlockManager: Found block rdd_428_0 locally
15/06/03 18:34:27 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:27 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:27 INFO MemoryStore: ensureFreeSpace(1664) called with curMem=674798, maxMem=559153152
15/06/03 18:34:27 INFO MemoryStore: Block rdd_442_0 stored as values in memory (estimated size 1664.0 B, free 532.6 MB)
15/06/03 18:34:27 INFO BlockManagerInfo: Added rdd_442_0 in memory on localhost:60836 (size: 1664.0 B, free: 533.1 MB)
15/06/03 18:34:27 INFO BlockManagerMaster: Updated info of block rdd_442_0
15/06/03 18:34:27 INFO Executor: Finished task 0.0 in stage 6897.0 (TID 91). 2693 bytes result sent to driver
15/06/03 18:34:27 INFO TaskSetManager: Finished task 0.0 in stage 6897.0 (TID 91) in 7 ms on localhost (1/1)
15/06/03 18:34:27 INFO TaskSchedulerImpl: Removed TaskSet 6897.0, whose tasks have all completed, from pool
15/06/03 18:34:27 INFO DAGScheduler: Stage 6897 (reduce at VertexRDDImpl.scala:90) finished in 0.007 s
15/06/03 18:34:27 INFO DAGScheduler: Job 22 finished: reduce at VertexRDDImpl.scala:90, took 0.189090 s
15/06/03 18:34:27 INFO Pregel: Pregel finished iteration 20
15/06/03 18:34:27 INFO ZippedPartitionsRDD2: Removing RDD 423 from persistence list
15/06/03 18:34:27 INFO BlockManager: Removing RDD 423
15/06/03 18:34:27 INFO ZippedPartitionsRDD2: Removing RDD 426 from persistence list
15/06/03 18:34:27 INFO BlockManager: Removing block rdd_423_0
15/06/03 18:34:27 INFO MemoryStore: Block rdd_423_0 of size 1664 dropped from memory (free 558478354)
15/06/03 18:34:27 INFO BlockManager: Removing RDD 426
15/06/03 18:34:27 INFO ZippedPartitionsRDD2: Removing RDD 409 from persistence list
15/06/03 18:34:27 INFO BlockManager: Removing block rdd_426_0
15/06/03 18:34:27 INFO MemoryStore: Block rdd_426_0 of size 1544 dropped from memory (free 558479898)
15/06/03 18:34:27 INFO BlockManager: Removing RDD 409
15/06/03 18:34:27 INFO ZippedPartitionsRDD2: Removing RDD 415 from persistence list
15/06/03 18:34:27 INFO BlockManager: Removing block rdd_409_0
15/06/03 18:34:27 INFO MemoryStore: Block rdd_409_0 of size 1496 dropped from memory (free 558481394)
15/06/03 18:34:27 INFO BlockManager: Removing RDD 415
15/06/03 18:34:27 INFO BlockManager: Removing block rdd_415_0
15/06/03 18:34:27 INFO MemoryStore: Block rdd_415_0 of size 2976 dropped from memory (free 558484370)
15/06/03 18:34:27 INFO SparkContext: Starting job: reduce at VertexRDDImpl.scala:90
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 2 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 3 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 5 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 4 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 1 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 0 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 8 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 7 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 6 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 11 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 10 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 9 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 14 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 13 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 12 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 17 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 16 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 15 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 20 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 19 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 18 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 23 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 22 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 21 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 26 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 25 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 24 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 29 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 28 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 27 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 32 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 31 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 30 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 35 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 34 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 33 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 38 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 37 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 36 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 41 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 40 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 39 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 44 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 43 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 42 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 47 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 46 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 45 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 50 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 49 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 48 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 53 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 52 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 51 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 56 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 55 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 54 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 59 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 58 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 57 is 145 bytes
15/06/03 18:34:27 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 62 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 61 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 60 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 65 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 64 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 63 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 68 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 67 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 66 is 145 bytes
15/06/03 18:34:28 INFO DAGScheduler: Registering RDD 451 (mapPartitions at VertexRDDImpl.scala:218)
15/06/03 18:34:28 INFO DAGScheduler: Registering RDD 455 (mapPartitions at VertexRDDImpl.scala:222)
15/06/03 18:34:28 INFO DAGScheduler: Registering RDD 459 (mapPartitions at GraphImpl.scala:235)
15/06/03 18:34:28 INFO DAGScheduler: Got job 23 (reduce at VertexRDDImpl.scala:90) with 1 output partitions (allowLocal=false)
15/06/03 18:34:28 INFO DAGScheduler: Final stage: Stage 7797(reduce at VertexRDDImpl.scala:90)
15/06/03 18:34:28 INFO DAGScheduler: Parents of final stage: List(Stage 7754, Stage 7776, Stage 7762, Stage 7784, Stage 7770, Stage 7749, Stage 7792, Stage 7756, Stage 7786, Stage 7750, Stage 7778, Stage 7772, Stage 7764, Stage 7794, Stage 7758, Stage 7780, Stage 7766, Stage 7796, Stage 7788, Stage 7752, Stage 7782, Stage 7774, Stage 7768, Stage 7760, Stage 7790)
15/06/03 18:34:28 INFO DAGScheduler: Missing parents: List(Stage 7796)
15/06/03 18:34:28 INFO DAGScheduler: Submitting Stage 7725 (ReplicatedVertexView.updateVertices - shippedVerts true false (broadcast) MapPartitionsRDD[451] at mapPartitions at VertexRDDImpl.scala:218), which has no missing parents
15/06/03 18:34:28 INFO MemoryStore: ensureFreeSpace(17768) called with curMem=668782, maxMem=559153152
15/06/03 18:34:28 INFO MemoryStore: Block broadcast_93 stored as values in memory (estimated size 17.4 KB, free 532.6 MB)
15/06/03 18:34:28 INFO MemoryStore: ensureFreeSpace(8595) called with curMem=686550, maxMem=559153152
15/06/03 18:34:28 INFO MemoryStore: Block broadcast_93_piece0 stored as bytes in memory (estimated size 8.4 KB, free 532.6 MB)
15/06/03 18:34:28 INFO BlockManagerInfo: Added broadcast_93_piece0 in memory on localhost:60836 (size: 8.4 KB, free: 533.0 MB)
15/06/03 18:34:28 INFO BlockManagerMaster: Updated info of block broadcast_93_piece0
15/06/03 18:34:28 INFO SparkContext: Created broadcast 93 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:28 INFO DAGScheduler: Submitting 1 missing tasks from Stage 7725 (ReplicatedVertexView.updateVertices - shippedVerts true false (broadcast) MapPartitionsRDD[451] at mapPartitions at VertexRDDImpl.scala:218)
15/06/03 18:34:28 INFO TaskSchedulerImpl: Adding task set 7725.0 with 1 tasks
15/06/03 18:34:28 INFO DAGScheduler: Submitting Stage 7795 (ReplicatedVertexView.withActiveSet - shippedActives (broadcast) MapPartitionsRDD[455] at mapPartitions at VertexRDDImpl.scala:222), which has no missing parents
15/06/03 18:34:28 INFO TaskSetManager: Starting task 0.0 in stage 7725.0 (TID 92, localhost, PROCESS_LOCAL, 3836 bytes)
15/06/03 18:34:28 INFO Executor: Running task 0.0 in stage 7725.0 (TID 92)
15/06/03 18:34:28 INFO MemoryStore: ensureFreeSpace(17424) called with curMem=695145, maxMem=559153152
15/06/03 18:34:28 INFO MemoryStore: Block broadcast_94 stored as values in memory (estimated size 17.0 KB, free 532.6 MB)
15/06/03 18:34:28 INFO MemoryStore: ensureFreeSpace(8445) called with curMem=712569, maxMem=559153152
15/06/03 18:34:28 INFO MemoryStore: Block broadcast_94_piece0 stored as bytes in memory (estimated size 8.2 KB, free 532.6 MB)
15/06/03 18:34:28 INFO BlockManagerInfo: Added broadcast_94_piece0 in memory on localhost:60836 (size: 8.2 KB, free: 533.0 MB)
15/06/03 18:34:28 INFO BlockManagerMaster: Updated info of block broadcast_94_piece0
15/06/03 18:34:28 INFO SparkContext: Created broadcast 94 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:28 INFO DAGScheduler: Submitting 1 missing tasks from Stage 7795 (ReplicatedVertexView.withActiveSet - shippedActives (broadcast) MapPartitionsRDD[455] at mapPartitions at VertexRDDImpl.scala:222)
15/06/03 18:34:28 INFO TaskSchedulerImpl: Adding task set 7795.0 with 1 tasks
15/06/03 18:34:28 INFO BlockManager: Found block rdd_428_0 locally
15/06/03 18:34:28 INFO BlockManager: Removing broadcast 76
15/06/03 18:34:28 INFO CacheManager: Partition rdd_447_0 not found, computing it
15/06/03 18:34:28 INFO BlockManager: Removing block broadcast_76_piece0
15/06/03 18:34:28 INFO BlockManager: Found block rdd_428_0 locally
15/06/03 18:34:28 INFO MemoryStore: Block broadcast_76_piece0 of size 7640 dropped from memory (free 558439778)
15/06/03 18:34:28 INFO CacheManager: Partition rdd_445_0 not found, computing it
15/06/03 18:34:28 INFO BlockManager: Found block rdd_428_0 locally
15/06/03 18:34:28 INFO BlockManager: Found block rdd_442_0 locally
15/06/03 18:34:28 INFO BlockManagerInfo: Removed broadcast_76_piece0 on localhost:60836 in memory (size: 7.5 KB, free: 533.0 MB)
15/06/03 18:34:28 INFO BlockManagerMaster: Updated info of block broadcast_76_piece0
15/06/03 18:34:28 INFO BlockManager: Removing block broadcast_76
15/06/03 18:34:28 INFO MemoryStore: Block broadcast_76 of size 15424 dropped from memory (free 558455202)
15/06/03 18:34:28 INFO MemoryStore: ensureFreeSpace(1544) called with curMem=697950, maxMem=559153152
15/06/03 18:34:28 INFO MemoryStore: Block rdd_445_0 stored as values in memory (estimated size 1544.0 B, free 532.6 MB)
15/06/03 18:34:28 INFO ContextCleaner: Cleaned broadcast 76
15/06/03 18:34:28 INFO BlockManagerInfo: Added rdd_445_0 in memory on localhost:60836 (size: 1544.0 B, free: 533.0 MB)
15/06/03 18:34:28 INFO BlockManagerMaster: Updated info of block rdd_445_0
15/06/03 18:34:28 INFO BlockManager: Removing broadcast 75
15/06/03 18:34:28 INFO MemoryStore: ensureFreeSpace(1496) called with curMem=699494, maxMem=559153152
15/06/03 18:34:28 INFO BlockManager: Removing block broadcast_75_piece0
15/06/03 18:34:28 INFO MemoryStore: Block rdd_447_0 stored as values in memory (estimated size 1496.0 B, free 532.6 MB)
15/06/03 18:34:28 INFO MemoryStore: Block broadcast_75_piece0 of size 6091 dropped from memory (free 558458253)
15/06/03 18:34:28 INFO BlockManagerInfo: Added rdd_447_0 in memory on localhost:60836 (size: 1496.0 B, free: 533.0 MB)
15/06/03 18:34:28 INFO BlockManagerMaster: Updated info of block rdd_447_0
15/06/03 18:34:28 INFO BlockManagerInfo: Removed broadcast_75_piece0 on localhost:60836 in memory (size: 5.9 KB, free: 533.1 MB)
15/06/03 18:34:28 INFO BlockManagerMaster: Updated info of block broadcast_75_piece0
15/06/03 18:34:28 INFO BlockManager: Removing block broadcast_75
15/06/03 18:34:28 INFO MemoryStore: Block broadcast_75 of size 10728 dropped from memory (free 558468981)
15/06/03 18:34:28 INFO ContextCleaner: Cleaned broadcast 75
15/06/03 18:34:28 INFO BlockManager: Removing broadcast 74
15/06/03 18:34:28 INFO BlockManager: Removing block broadcast_74_piece0
15/06/03 18:34:28 INFO MemoryStore: Block broadcast_74_piece0 of size 7428 dropped from memory (free 558476409)
15/06/03 18:34:28 INFO BlockManagerInfo: Removed broadcast_74_piece0 on localhost:60836 in memory (size: 7.3 KB, free: 533.1 MB)
15/06/03 18:34:28 INFO BlockManagerMaster: Updated info of block broadcast_74_piece0
15/06/03 18:34:28 INFO BlockManager: Removing block broadcast_74
15/06/03 18:34:28 INFO MemoryStore: Block broadcast_74 of size 14792 dropped from memory (free 558491201)
15/06/03 18:34:28 INFO ContextCleaner: Cleaned broadcast 74
15/06/03 18:34:28 INFO BlockManager: Removing broadcast 92
15/06/03 18:34:28 INFO BlockManager: Removing block broadcast_92
15/06/03 18:34:28 INFO MemoryStore: Block broadcast_92 of size 17528 dropped from memory (free 558508729)
15/06/03 18:34:28 INFO BlockManager: Removing block broadcast_92_piece0
15/06/03 18:34:28 INFO MemoryStore: Block broadcast_92_piece0 of size 8572 dropped from memory (free 558517301)
15/06/03 18:34:28 INFO BlockManagerInfo: Removed broadcast_92_piece0 on localhost:60836 in memory (size: 8.4 KB, free: 533.1 MB)
15/06/03 18:34:28 INFO BlockManagerMaster: Updated info of block broadcast_92_piece0
15/06/03 18:34:28 INFO ContextCleaner: Cleaned broadcast 92
15/06/03 18:34:28 INFO BlockManager: Removing broadcast 91
15/06/03 18:34:28 INFO BlockManager: Removing block broadcast_91_piece0
15/06/03 18:34:28 INFO MemoryStore: Block broadcast_91_piece0 of size 6413 dropped from memory (free 558523714)
15/06/03 18:34:28 INFO BlockManagerInfo: Removed broadcast_91_piece0 on localhost:60836 in memory (size: 6.3 KB, free: 533.1 MB)
15/06/03 18:34:28 INFO BlockManagerMaster: Updated info of block broadcast_91_piece0
15/06/03 18:34:28 INFO BlockManager: Removing block broadcast_91
15/06/03 18:34:28 INFO MemoryStore: Block broadcast_91 of size 11640 dropped from memory (free 558535354)
15/06/03 18:34:28 INFO ContextCleaner: Cleaned broadcast 91
15/06/03 18:34:28 INFO BlockManager: Removing broadcast 90
15/06/03 18:34:28 INFO BlockManager: Removing block broadcast_90
15/06/03 18:34:28 INFO MemoryStore: Block broadcast_90 of size 16896 dropped from memory (free 558552250)
15/06/03 18:34:28 INFO BlockManager: Removing block broadcast_90_piece0
15/06/03 18:34:28 INFO MemoryStore: Block broadcast_90_piece0 of size 8237 dropped from memory (free 558560487)
15/06/03 18:34:28 INFO BlockManagerInfo: Removed broadcast_90_piece0 on localhost:60836 in memory (size: 8.0 KB, free: 533.1 MB)
15/06/03 18:34:28 INFO Executor: Finished task 0.0 in stage 7725.0 (TID 92). 2627 bytes result sent to driver
15/06/03 18:34:28 INFO BlockManagerMaster: Updated info of block broadcast_90_piece0
15/06/03 18:34:28 INFO ContextCleaner: Cleaned broadcast 90
15/06/03 18:34:28 INFO TaskSetManager: Starting task 0.0 in stage 7795.0 (TID 93, localhost, PROCESS_LOCAL, 3772 bytes)
15/06/03 18:34:28 INFO TaskSetManager: Finished task 0.0 in stage 7725.0 (TID 92) in 62 ms on localhost (1/1)
15/06/03 18:34:28 INFO TaskSchedulerImpl: Removed TaskSet 7725.0, whose tasks have all completed, from pool
15/06/03 18:34:28 INFO BlockManager: Removing broadcast 89
15/06/03 18:34:28 INFO BlockManager: Removing block broadcast_89_piece0
15/06/03 18:34:28 INFO MemoryStore: Block broadcast_89_piece0 of size 8380 dropped from memory (free 558568867)
15/06/03 18:34:28 INFO Executor: Running task 0.0 in stage 7795.0 (TID 93)
15/06/03 18:34:28 INFO DAGScheduler: Stage 7725 (mapPartitions at VertexRDDImpl.scala:218) finished in 0.063 s
15/06/03 18:34:28 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:28 INFO DAGScheduler: running: Set(Stage 7795)
15/06/03 18:34:28 INFO DAGScheduler: waiting: Set(Stage 7797, Stage 7796)
15/06/03 18:34:28 INFO DAGScheduler: failed: Set()
15/06/03 18:34:28 INFO BlockManagerInfo: Removed broadcast_89_piece0 on localhost:60836 in memory (size: 8.2 KB, free: 533.1 MB)
15/06/03 18:34:28 INFO BlockManagerMaster: Updated info of block broadcast_89_piece0
15/06/03 18:34:28 INFO BlockManager: Removing block broadcast_89
15/06/03 18:34:28 INFO MemoryStore: Block broadcast_89 of size 17240 dropped from memory (free 558586107)
15/06/03 18:34:28 INFO ContextCleaner: Cleaned broadcast 89
15/06/03 18:34:28 INFO BlockManager: Removing broadcast 88
15/06/03 18:34:28 INFO BlockManager: Removing block broadcast_88_piece0
15/06/03 18:34:28 INFO MemoryStore: Block broadcast_88_piece0 of size 8279 dropped from memory (free 558594386)
15/06/03 18:34:28 INFO DAGScheduler: Missing parents for Stage 7797: List(Stage 7796)
15/06/03 18:34:28 INFO BlockManagerInfo: Removed broadcast_88_piece0 on localhost:60836 in memory (size: 8.1 KB, free: 533.1 MB)
15/06/03 18:34:28 INFO BlockManagerMaster: Updated info of block broadcast_88_piece0
15/06/03 18:34:28 INFO BlockManager: Removing block broadcast_88
15/06/03 18:34:28 INFO MemoryStore: Block broadcast_88 of size 17000 dropped from memory (free 558611386)
15/06/03 18:34:28 INFO ContextCleaner: Cleaned broadcast 88
15/06/03 18:34:28 INFO BlockManager: Removing broadcast 87
15/06/03 18:34:28 INFO BlockManager: Removing block broadcast_87_piece0
15/06/03 18:34:28 INFO MemoryStore: Block broadcast_87_piece0 of size 6232 dropped from memory (free 558617618)
15/06/03 18:34:28 INFO BlockManagerInfo: Removed broadcast_87_piece0 on localhost:60836 in memory (size: 6.1 KB, free: 533.1 MB)
15/06/03 18:34:28 INFO BlockManagerMaster: Updated info of block broadcast_87_piece0
15/06/03 18:34:28 INFO BlockManager: Removing block broadcast_87
15/06/03 18:34:28 INFO MemoryStore: Block broadcast_87 of size 11408 dropped from memory (free 558629026)
15/06/03 18:34:28 INFO ContextCleaner: Cleaned broadcast 87
15/06/03 18:34:28 INFO DAGScheduler: Missing parents for Stage 7796: List(Stage 7795)
15/06/03 18:34:28 INFO BlockManager: Removing broadcast 86
15/06/03 18:34:28 INFO BlockManager: Removing block broadcast_86_piece0
15/06/03 18:34:28 INFO MemoryStore: Block broadcast_86_piece0 of size 7945 dropped from memory (free 558636971)
15/06/03 18:34:28 INFO BlockManagerInfo: Removed broadcast_86_piece0 on localhost:60836 in memory (size: 7.8 KB, free: 533.1 MB)
15/06/03 18:34:28 INFO BlockManagerMaster: Updated info of block broadcast_86_piece0
15/06/03 18:34:28 INFO BlockManager: Removing block broadcast_86
15/06/03 18:34:28 INFO MemoryStore: Block broadcast_86 of size 16376 dropped from memory (free 558653347)
15/06/03 18:34:28 INFO BlockManager: Found block rdd_445_0 locally
15/06/03 18:34:28 INFO ContextCleaner: Cleaned broadcast 86
15/06/03 18:34:28 INFO BlockManager: Removing broadcast 82
15/06/03 18:34:28 INFO BlockManager: Removing block broadcast_82
15/06/03 18:34:28 INFO MemoryStore: Block broadcast_82 of size 15848 dropped from memory (free 558669195)
15/06/03 18:34:28 INFO BlockManager: Removing block broadcast_82_piece0
15/06/03 18:34:28 INFO MemoryStore: Block broadcast_82_piece0 of size 7828 dropped from memory (free 558677023)
15/06/03 18:34:28 INFO BlockManagerInfo: Removed broadcast_82_piece0 on localhost:60836 in memory (size: 7.6 KB, free: 533.1 MB)
15/06/03 18:34:28 INFO BlockManagerMaster: Updated info of block broadcast_82_piece0
15/06/03 18:34:28 INFO ContextCleaner: Cleaned broadcast 82
15/06/03 18:34:28 INFO BlockManager: Removing broadcast 81
15/06/03 18:34:28 INFO BlockManager: Removing block broadcast_81
15/06/03 18:34:28 INFO MemoryStore: Block broadcast_81 of size 16192 dropped from memory (free 558693215)
15/06/03 18:34:28 INFO BlockManager: Removing block broadcast_81_piece0
15/06/03 18:34:28 INFO MemoryStore: Block broadcast_81_piece0 of size 7960 dropped from memory (free 558701175)
15/06/03 18:34:28 INFO BlockManagerInfo: Removed broadcast_81_piece0 on localhost:60836 in memory (size: 7.8 KB, free: 533.1 MB)
15/06/03 18:34:28 INFO BlockManagerMaster: Updated info of block broadcast_81_piece0
15/06/03 18:34:28 INFO ContextCleaner: Cleaned broadcast 81
15/06/03 18:34:28 INFO BlockManager: Removing broadcast 80
15/06/03 18:34:28 INFO BlockManager: Removing block broadcast_80
15/06/03 18:34:28 INFO MemoryStore: Block broadcast_80 of size 15952 dropped from memory (free 558717127)
15/06/03 18:34:28 INFO BlockManager: Removing block broadcast_80_piece0
15/06/03 18:34:28 INFO MemoryStore: Block broadcast_80_piece0 of size 7869 dropped from memory (free 558724996)
15/06/03 18:34:28 INFO BlockManagerInfo: Removed broadcast_80_piece0 on localhost:60836 in memory (size: 7.7 KB, free: 533.1 MB)
15/06/03 18:34:28 INFO Executor: Finished task 0.0 in stage 7795.0 (TID 93). 2003 bytes result sent to driver
15/06/03 18:34:28 INFO BlockManagerMaster: Updated info of block broadcast_80_piece0
15/06/03 18:34:28 INFO ContextCleaner: Cleaned broadcast 80
15/06/03 18:34:28 INFO TaskSetManager: Finished task 0.0 in stage 7795.0 (TID 93) in 25 ms on localhost (1/1)
15/06/03 18:34:28 INFO TaskSchedulerImpl: Removed TaskSet 7795.0, whose tasks have all completed, from pool
15/06/03 18:34:28 INFO BlockManager: Removing broadcast 79
15/06/03 18:34:28 INFO BlockManager: Removing block broadcast_79_piece0
15/06/03 18:34:28 INFO MemoryStore: Block broadcast_79_piece0 of size 6138 dropped from memory (free 558731134)
15/06/03 18:34:28 INFO DAGScheduler: Stage 7795 (mapPartitions at VertexRDDImpl.scala:222) finished in 0.039 s
15/06/03 18:34:28 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:28 INFO DAGScheduler: running: Set()
15/06/03 18:34:28 INFO DAGScheduler: waiting: Set(Stage 7797, Stage 7796)
15/06/03 18:34:28 INFO DAGScheduler: failed: Set()
15/06/03 18:34:28 INFO BlockManagerInfo: Removed broadcast_79_piece0 on localhost:60836 in memory (size: 6.0 KB, free: 533.1 MB)
15/06/03 18:34:28 INFO BlockManagerMaster: Updated info of block broadcast_79_piece0
15/06/03 18:34:28 INFO BlockManager: Removing block broadcast_79
15/06/03 18:34:28 INFO MemoryStore: Block broadcast_79 of size 10952 dropped from memory (free 558742086)
15/06/03 18:34:28 INFO ContextCleaner: Cleaned broadcast 79
15/06/03 18:34:28 INFO BlockManager: Removing broadcast 78
15/06/03 18:34:28 INFO BlockManager: Removing block broadcast_78
15/06/03 18:34:28 INFO MemoryStore: Block broadcast_78 of size 15320 dropped from memory (free 558757406)
15/06/03 18:34:28 INFO BlockManager: Removing block broadcast_78_piece0
15/06/03 18:34:28 INFO MemoryStore: Block broadcast_78_piece0 of size 7585 dropped from memory (free 558764991)
15/06/03 18:34:28 INFO DAGScheduler: Missing parents for Stage 7797: List(Stage 7796)
15/06/03 18:34:28 INFO BlockManagerInfo: Removed broadcast_78_piece0 on localhost:60836 in memory (size: 7.4 KB, free: 533.1 MB)
15/06/03 18:34:28 INFO BlockManagerMaster: Updated info of block broadcast_78_piece0
15/06/03 18:34:28 INFO ContextCleaner: Cleaned broadcast 78
15/06/03 18:34:28 INFO BlockManager: Removing broadcast 77
15/06/03 18:34:28 INFO BlockManager: Removing block broadcast_77
15/06/03 18:34:28 INFO MemoryStore: Block broadcast_77 of size 15664 dropped from memory (free 558780655)
15/06/03 18:34:28 INFO BlockManager: Removing block broadcast_77_piece0
15/06/03 18:34:28 INFO MemoryStore: Block broadcast_77_piece0 of size 7768 dropped from memory (free 558788423)
15/06/03 18:34:28 INFO BlockManagerInfo: Removed broadcast_77_piece0 on localhost:60836 in memory (size: 7.6 KB, free: 533.2 MB)
15/06/03 18:34:28 INFO BlockManagerMaster: Updated info of block broadcast_77_piece0
15/06/03 18:34:28 INFO ContextCleaner: Cleaned broadcast 77
15/06/03 18:34:28 INFO DAGScheduler: Missing parents for Stage 7796: List()
15/06/03 18:34:28 INFO DAGScheduler: Submitting Stage 7796 (GraphImpl.aggregateMessages - preAgg MapPartitionsRDD[459] at mapPartitions at GraphImpl.scala:235), which is now runnable
15/06/03 18:34:28 INFO BlockManager: Removing broadcast 85
15/06/03 18:34:28 INFO BlockManager: Removing block broadcast_85_piece0
15/06/03 18:34:28 INFO MemoryStore: Block broadcast_85_piece0 of size 8096 dropped from memory (free 558796519)
15/06/03 18:34:28 INFO BlockManagerInfo: Removed broadcast_85_piece0 on localhost:60836 in memory (size: 7.9 KB, free: 533.2 MB)
15/06/03 18:34:28 INFO BlockManagerMaster: Updated info of block broadcast_85_piece0
15/06/03 18:34:28 INFO BlockManager: Removing block broadcast_85
15/06/03 18:34:28 INFO MemoryStore: Block broadcast_85 of size 16712 dropped from memory (free 558813231)
15/06/03 18:34:28 INFO ContextCleaner: Cleaned broadcast 85
15/06/03 18:34:28 INFO BlockManager: Removing broadcast 84
15/06/03 18:34:28 INFO BlockManager: Removing block broadcast_84
15/06/03 18:34:28 INFO MemoryStore: Block broadcast_84 of size 16472 dropped from memory (free 558829703)
15/06/03 18:34:28 INFO BlockManager: Removing block broadcast_84_piece0
15/06/03 18:34:28 INFO MemoryStore: Block broadcast_84_piece0 of size 8030 dropped from memory (free 558837733)
15/06/03 18:34:28 INFO BlockManagerInfo: Removed broadcast_84_piece0 on localhost:60836 in memory (size: 7.8 KB, free: 533.2 MB)
15/06/03 18:34:28 INFO BlockManagerMaster: Updated info of block broadcast_84_piece0
15/06/03 18:34:28 INFO MemoryStore: ensureFreeSpace(11864) called with curMem=315419, maxMem=559153152
15/06/03 18:34:28 INFO MemoryStore: Block broadcast_95 stored as values in memory (estimated size 11.6 KB, free 532.9 MB)
15/06/03 18:34:28 INFO ContextCleaner: Cleaned broadcast 84
15/06/03 18:34:28 INFO BlockManager: Removing broadcast 83
15/06/03 18:34:28 INFO BlockManager: Removing block broadcast_83
15/06/03 18:34:28 INFO MemoryStore: Block broadcast_83 of size 11184 dropped from memory (free 558837053)
15/06/03 18:34:28 INFO BlockManager: Removing block broadcast_83_piece0
15/06/03 18:34:28 INFO MemoryStore: Block broadcast_83_piece0 of size 6108 dropped from memory (free 558843161)
15/06/03 18:34:28 INFO MemoryStore: ensureFreeSpace(6452) called with curMem=309991, maxMem=559153152
15/06/03 18:34:28 INFO MemoryStore: Block broadcast_95_piece0 stored as bytes in memory (estimated size 6.3 KB, free 532.9 MB)
15/06/03 18:34:28 INFO BlockManagerInfo: Removed broadcast_83_piece0 on localhost:60836 in memory (size: 6.0 KB, free: 533.2 MB)
15/06/03 18:34:28 INFO BlockManagerMaster: Updated info of block broadcast_83_piece0
15/06/03 18:34:28 INFO BlockManagerInfo: Added broadcast_95_piece0 in memory on localhost:60836 (size: 6.3 KB, free: 533.2 MB)
15/06/03 18:34:28 INFO ContextCleaner: Cleaned broadcast 83
15/06/03 18:34:28 INFO BlockManagerMaster: Updated info of block broadcast_95_piece0
15/06/03 18:34:28 INFO SparkContext: Created broadcast 95 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:28 INFO DAGScheduler: Submitting 1 missing tasks from Stage 7796 (GraphImpl.aggregateMessages - preAgg MapPartitionsRDD[459] at mapPartitions at GraphImpl.scala:235)
15/06/03 18:34:28 INFO TaskSchedulerImpl: Adding task set 7796.0 with 1 tasks
15/06/03 18:34:28 INFO TaskSetManager: Starting task 0.0 in stage 7796.0 (TID 94, localhost, PROCESS_LOCAL, 2837 bytes)
15/06/03 18:34:28 INFO Executor: Running task 0.0 in stage 7796.0 (TID 94)
15/06/03 18:34:28 INFO CacheManager: Partition rdd_453_0 not found, computing it
15/06/03 18:34:28 INFO BlockManager: Found block rdd_434_0 locally
15/06/03 18:34:28 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:28 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:28 INFO MemoryStore: ensureFreeSpace(2976) called with curMem=316443, maxMem=559153152
15/06/03 18:34:28 INFO MemoryStore: Block rdd_453_0 stored as values in memory (estimated size 2.9 KB, free 532.9 MB)
15/06/03 18:34:28 INFO BlockManagerInfo: Added rdd_453_0 in memory on localhost:60836 (size: 2.9 KB, free: 533.2 MB)
15/06/03 18:34:28 INFO BlockManagerMaster: Updated info of block rdd_453_0
15/06/03 18:34:28 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:28 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:28 INFO Executor: Finished task 0.0 in stage 7796.0 (TID 94). 2788 bytes result sent to driver
15/06/03 18:34:28 INFO TaskSetManager: Finished task 0.0 in stage 7796.0 (TID 94) in 13 ms on localhost (1/1)
15/06/03 18:34:28 INFO TaskSchedulerImpl: Removed TaskSet 7796.0, whose tasks have all completed, from pool
15/06/03 18:34:28 INFO DAGScheduler: Stage 7796 (mapPartitions at GraphImpl.scala:235) finished in 0.013 s
15/06/03 18:34:28 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:28 INFO DAGScheduler: running: Set()
15/06/03 18:34:28 INFO DAGScheduler: waiting: Set(Stage 7797)
15/06/03 18:34:28 INFO DAGScheduler: failed: Set()
15/06/03 18:34:28 INFO DAGScheduler: Missing parents for Stage 7797: List()
15/06/03 18:34:28 INFO DAGScheduler: Submitting Stage 7797 (MapPartitionsRDD[463] at map at VertexRDDImpl.scala:90), which is now runnable
15/06/03 18:34:28 INFO MemoryStore: ensureFreeSpace(18056) called with curMem=319419, maxMem=559153152
15/06/03 18:34:28 INFO MemoryStore: Block broadcast_96 stored as values in memory (estimated size 17.6 KB, free 532.9 MB)
15/06/03 18:34:28 INFO MemoryStore: ensureFreeSpace(8654) called with curMem=337475, maxMem=559153152
15/06/03 18:34:28 INFO MemoryStore: Block broadcast_96_piece0 stored as bytes in memory (estimated size 8.5 KB, free 532.9 MB)
15/06/03 18:34:28 INFO BlockManagerInfo: Added broadcast_96_piece0 in memory on localhost:60836 (size: 8.5 KB, free: 533.2 MB)
15/06/03 18:34:28 INFO BlockManagerMaster: Updated info of block broadcast_96_piece0
15/06/03 18:34:28 INFO SparkContext: Created broadcast 96 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:28 INFO DAGScheduler: Submitting 1 missing tasks from Stage 7797 (MapPartitionsRDD[463] at map at VertexRDDImpl.scala:90)
15/06/03 18:34:28 INFO TaskSchedulerImpl: Adding task set 7797.0 with 1 tasks
15/06/03 18:34:28 INFO TaskSetManager: Starting task 0.0 in stage 7797.0 (TID 95, localhost, PROCESS_LOCAL, 3856 bytes)
15/06/03 18:34:28 INFO Executor: Running task 0.0 in stage 7797.0 (TID 95)
15/06/03 18:34:28 INFO CacheManager: Partition rdd_461_0 not found, computing it
15/06/03 18:34:28 INFO BlockManager: Found block rdd_447_0 locally
15/06/03 18:34:28 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:28 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:28 INFO MemoryStore: ensureFreeSpace(1664) called with curMem=346129, maxMem=559153152
15/06/03 18:34:28 INFO MemoryStore: Block rdd_461_0 stored as values in memory (estimated size 1664.0 B, free 532.9 MB)
15/06/03 18:34:28 INFO BlockManagerInfo: Added rdd_461_0 in memory on localhost:60836 (size: 1664.0 B, free: 533.2 MB)
15/06/03 18:34:28 INFO BlockManagerMaster: Updated info of block rdd_461_0
15/06/03 18:34:28 INFO Executor: Finished task 0.0 in stage 7797.0 (TID 95). 2693 bytes result sent to driver
15/06/03 18:34:28 INFO TaskSetManager: Finished task 0.0 in stage 7797.0 (TID 95) in 9 ms on localhost (1/1)
15/06/03 18:34:28 INFO TaskSchedulerImpl: Removed TaskSet 7797.0, whose tasks have all completed, from pool
15/06/03 18:34:28 INFO DAGScheduler: Stage 7797 (reduce at VertexRDDImpl.scala:90) finished in 0.010 s
15/06/03 18:34:28 INFO DAGScheduler: Job 23 finished: reduce at VertexRDDImpl.scala:90, took 0.268847 s
15/06/03 18:34:28 INFO Pregel: Pregel finished iteration 21
15/06/03 18:34:28 INFO ZippedPartitionsRDD2: Removing RDD 442 from persistence list
15/06/03 18:34:28 INFO BlockManager: Removing RDD 442
15/06/03 18:34:28 INFO ZippedPartitionsRDD2: Removing RDD 445 from persistence list
15/06/03 18:34:28 INFO BlockManager: Removing block rdd_442_0
15/06/03 18:34:28 INFO MemoryStore: Block rdd_442_0 of size 1664 dropped from memory (free 558807023)
15/06/03 18:34:28 INFO ZippedPartitionsRDD2: Removing RDD 428 from persistence list
15/06/03 18:34:28 INFO BlockManager: Removing RDD 428
15/06/03 18:34:28 INFO BlockManager: Removing block rdd_428_0
15/06/03 18:34:28 INFO MemoryStore: Block rdd_428_0 of size 1496 dropped from memory (free 558808519)
15/06/03 18:34:28 INFO BlockManager: Removing RDD 445
15/06/03 18:34:28 INFO BlockManager: Removing block rdd_445_0
15/06/03 18:34:28 INFO MemoryStore: Block rdd_445_0 of size 1544 dropped from memory (free 558810063)
15/06/03 18:34:28 INFO ZippedPartitionsRDD2: Removing RDD 434 from persistence list
15/06/03 18:34:28 INFO BlockManager: Removing RDD 434
15/06/03 18:34:28 INFO BlockManager: Removing block rdd_434_0
15/06/03 18:34:28 INFO MemoryStore: Block rdd_434_0 of size 2976 dropped from memory (free 558813039)
15/06/03 18:34:28 INFO SparkContext: Starting job: reduce at VertexRDDImpl.scala:90
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 2 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 3 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 5 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 4 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 1 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 0 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 8 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 7 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 6 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 11 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 10 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 9 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 14 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 13 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 12 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 17 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 16 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 15 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 20 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 19 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 18 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 23 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 22 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 21 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 26 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 25 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 24 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 29 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 28 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 27 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 32 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 31 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 30 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 35 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 34 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 33 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 38 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 37 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 36 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 41 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 40 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 39 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 44 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 43 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 42 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 47 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 46 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 45 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 50 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 49 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 48 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 53 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 52 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 51 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 56 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 55 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 54 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 59 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 58 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 57 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 62 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 61 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 60 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 65 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 64 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 63 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 68 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 67 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 66 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 71 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 70 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 69 is 145 bytes
15/06/03 18:34:28 INFO DAGScheduler: Registering RDD 470 (mapPartitions at VertexRDDImpl.scala:218)
15/06/03 18:34:28 INFO DAGScheduler: Registering RDD 474 (mapPartitions at VertexRDDImpl.scala:222)
15/06/03 18:34:28 INFO DAGScheduler: Registering RDD 478 (mapPartitions at GraphImpl.scala:235)
15/06/03 18:34:28 INFO DAGScheduler: Got job 24 (reduce at VertexRDDImpl.scala:90) with 1 output partitions (allowLocal=false)
15/06/03 18:34:28 INFO DAGScheduler: Final stage: Stage 8772(reduce at VertexRDDImpl.scala:90)
15/06/03 18:34:28 INFO DAGScheduler: Parents of final stage: List(Stage 8763, Stage 8727, Stage 8749, Stage 8771, Stage 8735, Stage 8765, Stage 8757, Stage 8751, Stage 8743, Stage 8722, Stage 8737, Stage 8729, Stage 8759, Stage 8723, Stage 8745, Stage 8767, Stage 8731, Stage 8761, Stage 8753, Stage 8747, Stage 8739, Stage 8769, Stage 8733, Stage 8725, Stage 8755, Stage 8741)
15/06/03 18:34:28 INFO DAGScheduler: Missing parents: List(Stage 8771)
15/06/03 18:34:28 INFO DAGScheduler: Submitting Stage 8697 (ReplicatedVertexView.updateVertices - shippedVerts true false (broadcast) MapPartitionsRDD[470] at mapPartitions at VertexRDDImpl.scala:218), which has no missing parents
15/06/03 18:34:28 INFO MemoryStore: ensureFreeSpace(18296) called with curMem=340113, maxMem=559153152
15/06/03 18:34:28 INFO MemoryStore: Block broadcast_97 stored as values in memory (estimated size 17.9 KB, free 532.9 MB)
15/06/03 18:34:28 INFO MemoryStore: ensureFreeSpace(8733) called with curMem=358409, maxMem=559153152
15/06/03 18:34:28 INFO MemoryStore: Block broadcast_97_piece0 stored as bytes in memory (estimated size 8.5 KB, free 532.9 MB)
15/06/03 18:34:28 INFO BlockManagerInfo: Added broadcast_97_piece0 in memory on localhost:60836 (size: 8.5 KB, free: 533.2 MB)
15/06/03 18:34:28 INFO BlockManagerMaster: Updated info of block broadcast_97_piece0
15/06/03 18:34:28 INFO SparkContext: Created broadcast 97 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:28 INFO DAGScheduler: Submitting 1 missing tasks from Stage 8697 (ReplicatedVertexView.updateVertices - shippedVerts true false (broadcast) MapPartitionsRDD[470] at mapPartitions at VertexRDDImpl.scala:218)
15/06/03 18:34:28 INFO TaskSchedulerImpl: Adding task set 8697.0 with 1 tasks
15/06/03 18:34:28 INFO DAGScheduler: Submitting Stage 8770 (ReplicatedVertexView.withActiveSet - shippedActives (broadcast) MapPartitionsRDD[474] at mapPartitions at VertexRDDImpl.scala:222), which has no missing parents
15/06/03 18:34:28 INFO TaskSetManager: Starting task 0.0 in stage 8697.0 (TID 96, localhost, PROCESS_LOCAL, 3941 bytes)
15/06/03 18:34:28 INFO Executor: Running task 0.0 in stage 8697.0 (TID 96)
15/06/03 18:34:28 INFO MemoryStore: ensureFreeSpace(17952) called with curMem=367142, maxMem=559153152
15/06/03 18:34:28 INFO MemoryStore: Block broadcast_98 stored as values in memory (estimated size 17.5 KB, free 532.9 MB)
15/06/03 18:34:28 INFO MemoryStore: ensureFreeSpace(8606) called with curMem=385094, maxMem=559153152
15/06/03 18:34:28 INFO MemoryStore: Block broadcast_98_piece0 stored as bytes in memory (estimated size 8.4 KB, free 532.9 MB)
15/06/03 18:34:28 INFO BlockManagerInfo: Added broadcast_98_piece0 in memory on localhost:60836 (size: 8.4 KB, free: 533.1 MB)
15/06/03 18:34:28 INFO BlockManagerMaster: Updated info of block broadcast_98_piece0
15/06/03 18:34:28 INFO SparkContext: Created broadcast 98 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:28 INFO DAGScheduler: Submitting 1 missing tasks from Stage 8770 (ReplicatedVertexView.withActiveSet - shippedActives (broadcast) MapPartitionsRDD[474] at mapPartitions at VertexRDDImpl.scala:222)
15/06/03 18:34:28 INFO TaskSchedulerImpl: Adding task set 8770.0 with 1 tasks
15/06/03 18:34:28 INFO BlockManager: Found block rdd_447_0 locally
15/06/03 18:34:28 INFO CacheManager: Partition rdd_466_0 not found, computing it
15/06/03 18:34:28 INFO BlockManager: Found block rdd_447_0 locally
15/06/03 18:34:28 INFO CacheManager: Partition rdd_464_0 not found, computing it
15/06/03 18:34:28 INFO BlockManager: Found block rdd_447_0 locally
15/06/03 18:34:28 INFO BlockManager: Found block rdd_461_0 locally
15/06/03 18:34:28 INFO MemoryStore: ensureFreeSpace(1544) called with curMem=393700, maxMem=559153152
15/06/03 18:34:28 INFO MemoryStore: Block rdd_464_0 stored as values in memory (estimated size 1544.0 B, free 532.9 MB)
15/06/03 18:34:28 INFO BlockManagerInfo: Added rdd_464_0 in memory on localhost:60836 (size: 1544.0 B, free: 533.1 MB)
15/06/03 18:34:28 INFO BlockManagerMaster: Updated info of block rdd_464_0
15/06/03 18:34:28 INFO MemoryStore: ensureFreeSpace(1496) called with curMem=395244, maxMem=559153152
15/06/03 18:34:28 INFO MemoryStore: Block rdd_466_0 stored as values in memory (estimated size 1496.0 B, free 532.9 MB)
15/06/03 18:34:28 INFO BlockManagerInfo: Added rdd_466_0 in memory on localhost:60836 (size: 1496.0 B, free: 533.1 MB)
15/06/03 18:34:28 INFO BlockManagerMaster: Updated info of block rdd_466_0
15/06/03 18:34:28 INFO Executor: Finished task 0.0 in stage 8697.0 (TID 96). 2627 bytes result sent to driver
15/06/03 18:34:28 INFO TaskSetManager: Starting task 0.0 in stage 8770.0 (TID 97, localhost, PROCESS_LOCAL, 3877 bytes)
15/06/03 18:34:28 INFO Executor: Running task 0.0 in stage 8770.0 (TID 97)
15/06/03 18:34:28 INFO TaskSetManager: Finished task 0.0 in stage 8697.0 (TID 96) in 22 ms on localhost (1/1)
15/06/03 18:34:28 INFO TaskSchedulerImpl: Removed TaskSet 8697.0, whose tasks have all completed, from pool
15/06/03 18:34:28 INFO DAGScheduler: Stage 8697 (mapPartitions at VertexRDDImpl.scala:218) finished in 0.024 s
15/06/03 18:34:28 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:28 INFO DAGScheduler: running: Set(Stage 8770)
15/06/03 18:34:28 INFO DAGScheduler: waiting: Set(Stage 8771, Stage 8772)
15/06/03 18:34:28 INFO DAGScheduler: failed: Set()
15/06/03 18:34:28 INFO DAGScheduler: Missing parents for Stage 8771: List(Stage 8770)
15/06/03 18:34:28 INFO DAGScheduler: Missing parents for Stage 8772: List(Stage 8771)
15/06/03 18:34:28 INFO BlockManager: Found block rdd_464_0 locally
15/06/03 18:34:28 INFO Executor: Finished task 0.0 in stage 8770.0 (TID 97). 2003 bytes result sent to driver
15/06/03 18:34:28 INFO TaskSetManager: Finished task 0.0 in stage 8770.0 (TID 97) in 13 ms on localhost (1/1)
15/06/03 18:34:28 INFO TaskSchedulerImpl: Removed TaskSet 8770.0, whose tasks have all completed, from pool
15/06/03 18:34:28 INFO DAGScheduler: Stage 8770 (mapPartitions at VertexRDDImpl.scala:222) finished in 0.027 s
15/06/03 18:34:28 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:28 INFO DAGScheduler: running: Set()
15/06/03 18:34:28 INFO DAGScheduler: waiting: Set(Stage 8771, Stage 8772)
15/06/03 18:34:28 INFO DAGScheduler: failed: Set()
15/06/03 18:34:28 INFO DAGScheduler: Missing parents for Stage 8771: List()
15/06/03 18:34:28 INFO DAGScheduler: Missing parents for Stage 8772: List(Stage 8771)
15/06/03 18:34:28 INFO DAGScheduler: Submitting Stage 8771 (GraphImpl.aggregateMessages - preAgg MapPartitionsRDD[478] at mapPartitions at GraphImpl.scala:235), which is now runnable
15/06/03 18:34:28 INFO MemoryStore: ensureFreeSpace(12096) called with curMem=396740, maxMem=559153152
15/06/03 18:34:28 INFO MemoryStore: Block broadcast_99 stored as values in memory (estimated size 11.8 KB, free 532.9 MB)
15/06/03 18:34:28 INFO MemoryStore: ensureFreeSpace(6516) called with curMem=408836, maxMem=559153152
15/06/03 18:34:28 INFO MemoryStore: Block broadcast_99_piece0 stored as bytes in memory (estimated size 6.4 KB, free 532.9 MB)
15/06/03 18:34:28 INFO BlockManagerInfo: Added broadcast_99_piece0 in memory on localhost:60836 (size: 6.4 KB, free: 533.1 MB)
15/06/03 18:34:28 INFO BlockManagerMaster: Updated info of block broadcast_99_piece0
15/06/03 18:34:28 INFO SparkContext: Created broadcast 99 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:28 INFO DAGScheduler: Submitting 1 missing tasks from Stage 8771 (GraphImpl.aggregateMessages - preAgg MapPartitionsRDD[478] at mapPartitions at GraphImpl.scala:235)
15/06/03 18:34:28 INFO TaskSchedulerImpl: Adding task set 8771.0 with 1 tasks
15/06/03 18:34:28 INFO TaskSetManager: Starting task 0.0 in stage 8771.0 (TID 98, localhost, PROCESS_LOCAL, 2878 bytes)
15/06/03 18:34:28 INFO Executor: Running task 0.0 in stage 8771.0 (TID 98)
15/06/03 18:34:28 INFO CacheManager: Partition rdd_472_0 not found, computing it
15/06/03 18:34:28 INFO BlockManager: Found block rdd_453_0 locally
15/06/03 18:34:28 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:28 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:28 INFO MemoryStore: ensureFreeSpace(2976) called with curMem=415352, maxMem=559153152
15/06/03 18:34:28 INFO MemoryStore: Block rdd_472_0 stored as values in memory (estimated size 2.9 KB, free 532.9 MB)
15/06/03 18:34:28 INFO BlockManagerInfo: Added rdd_472_0 in memory on localhost:60836 (size: 2.9 KB, free: 533.1 MB)
15/06/03 18:34:28 INFO BlockManagerMaster: Updated info of block rdd_472_0
15/06/03 18:34:28 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:28 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:28 INFO Executor: Finished task 0.0 in stage 8771.0 (TID 98). 2788 bytes result sent to driver
15/06/03 18:34:28 INFO TaskSetManager: Finished task 0.0 in stage 8771.0 (TID 98) in 14 ms on localhost (1/1)
15/06/03 18:34:28 INFO TaskSchedulerImpl: Removed TaskSet 8771.0, whose tasks have all completed, from pool
15/06/03 18:34:28 INFO DAGScheduler: Stage 8771 (mapPartitions at GraphImpl.scala:235) finished in 0.014 s
15/06/03 18:34:28 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:28 INFO DAGScheduler: running: Set()
15/06/03 18:34:28 INFO DAGScheduler: waiting: Set(Stage 8772)
15/06/03 18:34:28 INFO DAGScheduler: failed: Set()
15/06/03 18:34:28 INFO DAGScheduler: Missing parents for Stage 8772: List()
15/06/03 18:34:28 INFO DAGScheduler: Submitting Stage 8772 (MapPartitionsRDD[482] at map at VertexRDDImpl.scala:90), which is now runnable
15/06/03 18:34:28 INFO MemoryStore: ensureFreeSpace(18584) called with curMem=418328, maxMem=559153152
15/06/03 18:34:28 INFO MemoryStore: Block broadcast_100 stored as values in memory (estimated size 18.1 KB, free 532.8 MB)
15/06/03 18:34:28 INFO MemoryStore: ensureFreeSpace(8901) called with curMem=436912, maxMem=559153152
15/06/03 18:34:28 INFO MemoryStore: Block broadcast_100_piece0 stored as bytes in memory (estimated size 8.7 KB, free 532.8 MB)
15/06/03 18:34:28 INFO BlockManagerInfo: Added broadcast_100_piece0 in memory on localhost:60836 (size: 8.7 KB, free: 533.1 MB)
15/06/03 18:34:28 INFO BlockManagerMaster: Updated info of block broadcast_100_piece0
15/06/03 18:34:28 INFO SparkContext: Created broadcast 100 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:28 INFO DAGScheduler: Submitting 1 missing tasks from Stage 8772 (MapPartitionsRDD[482] at map at VertexRDDImpl.scala:90)
15/06/03 18:34:28 INFO TaskSchedulerImpl: Adding task set 8772.0 with 1 tasks
15/06/03 18:34:28 INFO TaskSetManager: Starting task 0.0 in stage 8772.0 (TID 99, localhost, PROCESS_LOCAL, 3961 bytes)
15/06/03 18:34:28 INFO Executor: Running task 0.0 in stage 8772.0 (TID 99)
15/06/03 18:34:28 INFO CacheManager: Partition rdd_480_0 not found, computing it
15/06/03 18:34:28 INFO BlockManager: Found block rdd_466_0 locally
15/06/03 18:34:28 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:28 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:28 INFO MemoryStore: ensureFreeSpace(1664) called with curMem=445813, maxMem=559153152
15/06/03 18:34:28 INFO MemoryStore: Block rdd_480_0 stored as values in memory (estimated size 1664.0 B, free 532.8 MB)
15/06/03 18:34:28 INFO BlockManagerInfo: Added rdd_480_0 in memory on localhost:60836 (size: 1664.0 B, free: 533.1 MB)
15/06/03 18:34:28 INFO BlockManagerMaster: Updated info of block rdd_480_0
15/06/03 18:34:28 INFO Executor: Finished task 0.0 in stage 8772.0 (TID 99). 2693 bytes result sent to driver
15/06/03 18:34:28 INFO TaskSetManager: Finished task 0.0 in stage 8772.0 (TID 99) in 7 ms on localhost (1/1)
15/06/03 18:34:28 INFO TaskSchedulerImpl: Removed TaskSet 8772.0, whose tasks have all completed, from pool
15/06/03 18:34:28 INFO DAGScheduler: Stage 8772 (reduce at VertexRDDImpl.scala:90) finished in 0.007 s
15/06/03 18:34:28 INFO DAGScheduler: Job 24 finished: reduce at VertexRDDImpl.scala:90, took 0.232335 s
15/06/03 18:34:28 INFO Pregel: Pregel finished iteration 22
15/06/03 18:34:28 INFO ZippedPartitionsRDD2: Removing RDD 461 from persistence list
15/06/03 18:34:28 INFO BlockManager: Removing RDD 461
15/06/03 18:34:28 INFO BlockManager: Removing block rdd_461_0
15/06/03 18:34:28 INFO MemoryStore: Block rdd_461_0 of size 1664 dropped from memory (free 558707339)
15/06/03 18:34:28 INFO ZippedPartitionsRDD2: Removing RDD 464 from persistence list
15/06/03 18:34:28 INFO BlockManager: Removing RDD 464
15/06/03 18:34:28 INFO BlockManager: Removing block rdd_464_0
15/06/03 18:34:28 INFO MemoryStore: Block rdd_464_0 of size 1544 dropped from memory (free 558708883)
15/06/03 18:34:28 INFO ZippedPartitionsRDD2: Removing RDD 447 from persistence list
15/06/03 18:34:28 INFO BlockManager: Removing RDD 447
15/06/03 18:34:28 INFO ZippedPartitionsRDD2: Removing RDD 453 from persistence list
15/06/03 18:34:28 INFO BlockManager: Removing block rdd_447_0
15/06/03 18:34:28 INFO MemoryStore: Block rdd_447_0 of size 1496 dropped from memory (free 558710379)
15/06/03 18:34:28 INFO BlockManager: Removing RDD 453
15/06/03 18:34:28 INFO BlockManager: Removing block rdd_453_0
15/06/03 18:34:28 INFO MemoryStore: Block rdd_453_0 of size 2976 dropped from memory (free 558713355)
15/06/03 18:34:28 INFO SparkContext: Starting job: reduce at VertexRDDImpl.scala:90
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 2 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 3 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 5 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 4 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 1 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 0 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 8 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 7 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 6 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 11 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 10 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 9 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 14 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 13 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 12 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 17 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 16 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 15 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 20 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 19 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 18 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 23 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 22 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 21 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 26 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 25 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 24 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 29 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 28 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 27 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 32 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 31 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 30 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 35 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 34 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 33 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 38 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 37 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 36 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 41 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 40 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 39 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 44 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 43 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 42 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 47 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 46 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 45 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 50 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 49 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 48 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 53 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 52 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 51 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 56 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 55 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 54 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 59 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 58 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 57 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 62 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 61 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 60 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 65 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 64 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 63 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 68 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 67 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 66 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 71 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 70 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 69 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 74 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 73 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 72 is 145 bytes
15/06/03 18:34:28 INFO DAGScheduler: Registering RDD 489 (mapPartitions at VertexRDDImpl.scala:218)
15/06/03 18:34:28 INFO DAGScheduler: Registering RDD 493 (mapPartitions at VertexRDDImpl.scala:222)
15/06/03 18:34:28 INFO DAGScheduler: Registering RDD 497 (mapPartitions at GraphImpl.scala:235)
15/06/03 18:34:28 INFO DAGScheduler: Got job 25 (reduce at VertexRDDImpl.scala:90) with 1 output partitions (allowLocal=false)
15/06/03 18:34:28 INFO DAGScheduler: Final stage: Stage 9825(reduce at VertexRDDImpl.scala:90)
15/06/03 18:34:28 INFO DAGScheduler: Parents of final stage: List(Stage 9794, Stage 9786, Stage 9816, Stage 9780, Stage 9802, Stage 9773, Stage 9824, Stage 9788, Stage 9818, Stage 9810, Stage 9774, Stage 9804, Stage 9796, Stage 9790, Stage 9782, Stage 9812, Stage 9776, Stage 9798, Stage 9820, Stage 9784, Stage 9806, Stage 9792, Stage 9822, Stage 9814, Stage 9778, Stage 9808, Stage 9800)
15/06/03 18:34:28 INFO DAGScheduler: Missing parents: List(Stage 9824)
15/06/03 18:34:28 INFO DAGScheduler: Submitting Stage 9747 (ReplicatedVertexView.updateVertices - shippedVerts true false (broadcast) MapPartitionsRDD[489] at mapPartitions at VertexRDDImpl.scala:218), which has no missing parents
15/06/03 18:34:28 INFO MemoryStore: ensureFreeSpace(18824) called with curMem=439797, maxMem=559153152
15/06/03 18:34:28 INFO MemoryStore: Block broadcast_101 stored as values in memory (estimated size 18.4 KB, free 532.8 MB)
15/06/03 18:34:28 INFO MemoryStore: ensureFreeSpace(8915) called with curMem=458621, maxMem=559153152
15/06/03 18:34:28 INFO MemoryStore: Block broadcast_101_piece0 stored as bytes in memory (estimated size 8.7 KB, free 532.8 MB)
15/06/03 18:34:28 INFO BlockManagerInfo: Added broadcast_101_piece0 in memory on localhost:60836 (size: 8.7 KB, free: 533.1 MB)
15/06/03 18:34:28 INFO BlockManagerMaster: Updated info of block broadcast_101_piece0
15/06/03 18:34:28 INFO SparkContext: Created broadcast 101 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:28 INFO DAGScheduler: Submitting 1 missing tasks from Stage 9747 (ReplicatedVertexView.updateVertices - shippedVerts true false (broadcast) MapPartitionsRDD[489] at mapPartitions at VertexRDDImpl.scala:218)
15/06/03 18:34:28 INFO TaskSchedulerImpl: Adding task set 9747.0 with 1 tasks
15/06/03 18:34:28 INFO DAGScheduler: Submitting Stage 9823 (ReplicatedVertexView.withActiveSet - shippedActives (broadcast) MapPartitionsRDD[493] at mapPartitions at VertexRDDImpl.scala:222), which has no missing parents
15/06/03 18:34:28 INFO TaskSetManager: Starting task 0.0 in stage 9747.0 (TID 100, localhost, PROCESS_LOCAL, 4046 bytes)
15/06/03 18:34:28 INFO Executor: Running task 0.0 in stage 9747.0 (TID 100)
15/06/03 18:34:28 INFO MemoryStore: ensureFreeSpace(18480) called with curMem=467536, maxMem=559153152
15/06/03 18:34:28 INFO MemoryStore: Block broadcast_102 stored as values in memory (estimated size 18.0 KB, free 532.8 MB)
15/06/03 18:34:28 INFO MemoryStore: ensureFreeSpace(8772) called with curMem=486016, maxMem=559153152
15/06/03 18:34:28 INFO MemoryStore: Block broadcast_102_piece0 stored as bytes in memory (estimated size 8.6 KB, free 532.8 MB)
15/06/03 18:34:28 INFO BlockManagerInfo: Added broadcast_102_piece0 in memory on localhost:60836 (size: 8.6 KB, free: 533.1 MB)
15/06/03 18:34:28 INFO BlockManagerMaster: Updated info of block broadcast_102_piece0
15/06/03 18:34:28 INFO SparkContext: Created broadcast 102 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:28 INFO DAGScheduler: Submitting 1 missing tasks from Stage 9823 (ReplicatedVertexView.withActiveSet - shippedActives (broadcast) MapPartitionsRDD[493] at mapPartitions at VertexRDDImpl.scala:222)
15/06/03 18:34:28 INFO TaskSchedulerImpl: Adding task set 9823.0 with 1 tasks
15/06/03 18:34:28 INFO BlockManager: Found block rdd_466_0 locally
15/06/03 18:34:28 INFO CacheManager: Partition rdd_485_0 not found, computing it
15/06/03 18:34:28 INFO BlockManager: Found block rdd_466_0 locally
15/06/03 18:34:28 INFO CacheManager: Partition rdd_483_0 not found, computing it
15/06/03 18:34:28 INFO BlockManager: Found block rdd_466_0 locally
15/06/03 18:34:28 INFO BlockManager: Found block rdd_480_0 locally
15/06/03 18:34:28 INFO MemoryStore: ensureFreeSpace(1544) called with curMem=494788, maxMem=559153152
15/06/03 18:34:28 INFO MemoryStore: Block rdd_483_0 stored as values in memory (estimated size 1544.0 B, free 532.8 MB)
15/06/03 18:34:28 INFO BlockManagerInfo: Added rdd_483_0 in memory on localhost:60836 (size: 1544.0 B, free: 533.1 MB)
15/06/03 18:34:28 INFO BlockManagerMaster: Updated info of block rdd_483_0
15/06/03 18:34:28 INFO MemoryStore: ensureFreeSpace(1496) called with curMem=496332, maxMem=559153152
15/06/03 18:34:28 INFO MemoryStore: Block rdd_485_0 stored as values in memory (estimated size 1496.0 B, free 532.8 MB)
15/06/03 18:34:28 INFO BlockManagerInfo: Added rdd_485_0 in memory on localhost:60836 (size: 1496.0 B, free: 533.1 MB)
15/06/03 18:34:28 INFO BlockManagerMaster: Updated info of block rdd_485_0
15/06/03 18:34:28 INFO Executor: Finished task 0.0 in stage 9747.0 (TID 100). 2627 bytes result sent to driver
15/06/03 18:34:28 INFO TaskSetManager: Starting task 0.0 in stage 9823.0 (TID 101, localhost, PROCESS_LOCAL, 3982 bytes)
15/06/03 18:34:28 INFO TaskSetManager: Finished task 0.0 in stage 9747.0 (TID 100) in 25 ms on localhost (1/1)
15/06/03 18:34:28 INFO TaskSchedulerImpl: Removed TaskSet 9747.0, whose tasks have all completed, from pool
15/06/03 18:34:28 INFO Executor: Running task 0.0 in stage 9823.0 (TID 101)
15/06/03 18:34:28 INFO DAGScheduler: Stage 9747 (mapPartitions at VertexRDDImpl.scala:218) finished in 0.026 s
15/06/03 18:34:28 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:28 INFO DAGScheduler: running: Set(Stage 9823)
15/06/03 18:34:28 INFO DAGScheduler: waiting: Set(Stage 9824, Stage 9825)
15/06/03 18:34:28 INFO DAGScheduler: failed: Set()
15/06/03 18:34:28 INFO DAGScheduler: Missing parents for Stage 9824: List(Stage 9823)
15/06/03 18:34:28 INFO DAGScheduler: Missing parents for Stage 9825: List(Stage 9824)
15/06/03 18:34:28 INFO BlockManager: Found block rdd_483_0 locally
15/06/03 18:34:28 INFO Executor: Finished task 0.0 in stage 9823.0 (TID 101). 2003 bytes result sent to driver
15/06/03 18:34:28 INFO TaskSetManager: Finished task 0.0 in stage 9823.0 (TID 101) in 13 ms on localhost (1/1)
15/06/03 18:34:28 INFO TaskSchedulerImpl: Removed TaskSet 9823.0, whose tasks have all completed, from pool
15/06/03 18:34:28 INFO DAGScheduler: Stage 9823 (mapPartitions at VertexRDDImpl.scala:222) finished in 0.031 s
15/06/03 18:34:28 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:28 INFO DAGScheduler: running: Set()
15/06/03 18:34:28 INFO DAGScheduler: waiting: Set(Stage 9824, Stage 9825)
15/06/03 18:34:28 INFO DAGScheduler: failed: Set()
15/06/03 18:34:28 INFO DAGScheduler: Missing parents for Stage 9824: List()
15/06/03 18:34:28 INFO DAGScheduler: Missing parents for Stage 9825: List(Stage 9824)
15/06/03 18:34:28 INFO DAGScheduler: Submitting Stage 9824 (GraphImpl.aggregateMessages - preAgg MapPartitionsRDD[497] at mapPartitions at GraphImpl.scala:235), which is now runnable
15/06/03 18:34:28 INFO MemoryStore: ensureFreeSpace(12320) called with curMem=497828, maxMem=559153152
15/06/03 18:34:28 INFO MemoryStore: Block broadcast_103 stored as values in memory (estimated size 12.0 KB, free 532.8 MB)
15/06/03 18:34:28 INFO MemoryStore: ensureFreeSpace(6678) called with curMem=510148, maxMem=559153152
15/06/03 18:34:28 INFO MemoryStore: Block broadcast_103_piece0 stored as bytes in memory (estimated size 6.5 KB, free 532.8 MB)
15/06/03 18:34:28 INFO BlockManagerInfo: Added broadcast_103_piece0 in memory on localhost:60836 (size: 6.5 KB, free: 533.1 MB)
15/06/03 18:34:28 INFO BlockManagerMaster: Updated info of block broadcast_103_piece0
15/06/03 18:34:28 INFO SparkContext: Created broadcast 103 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:28 INFO DAGScheduler: Submitting 1 missing tasks from Stage 9824 (GraphImpl.aggregateMessages - preAgg MapPartitionsRDD[497] at mapPartitions at GraphImpl.scala:235)
15/06/03 18:34:28 INFO TaskSchedulerImpl: Adding task set 9824.0 with 1 tasks
15/06/03 18:34:28 INFO TaskSetManager: Starting task 0.0 in stage 9824.0 (TID 102, localhost, PROCESS_LOCAL, 2919 bytes)
15/06/03 18:34:28 INFO Executor: Running task 0.0 in stage 9824.0 (TID 102)
15/06/03 18:34:28 INFO CacheManager: Partition rdd_491_0 not found, computing it
15/06/03 18:34:28 INFO BlockManager: Found block rdd_472_0 locally
15/06/03 18:34:28 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:28 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:28 INFO MemoryStore: ensureFreeSpace(2976) called with curMem=516826, maxMem=559153152
15/06/03 18:34:28 INFO MemoryStore: Block rdd_491_0 stored as values in memory (estimated size 2.9 KB, free 532.8 MB)
15/06/03 18:34:28 INFO BlockManagerInfo: Added rdd_491_0 in memory on localhost:60836 (size: 2.9 KB, free: 533.1 MB)
15/06/03 18:34:28 INFO BlockManagerMaster: Updated info of block rdd_491_0
15/06/03 18:34:28 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:28 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:28 INFO Executor: Finished task 0.0 in stage 9824.0 (TID 102). 2788 bytes result sent to driver
15/06/03 18:34:28 INFO TaskSetManager: Finished task 0.0 in stage 9824.0 (TID 102) in 14 ms on localhost (1/1)
15/06/03 18:34:28 INFO TaskSchedulerImpl: Removed TaskSet 9824.0, whose tasks have all completed, from pool
15/06/03 18:34:28 INFO DAGScheduler: Stage 9824 (mapPartitions at GraphImpl.scala:235) finished in 0.014 s
15/06/03 18:34:28 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:28 INFO DAGScheduler: running: Set()
15/06/03 18:34:28 INFO DAGScheduler: waiting: Set(Stage 9825)
15/06/03 18:34:28 INFO DAGScheduler: failed: Set()
15/06/03 18:34:28 INFO DAGScheduler: Missing parents for Stage 9825: List()
15/06/03 18:34:28 INFO DAGScheduler: Submitting Stage 9825 (MapPartitionsRDD[501] at map at VertexRDDImpl.scala:90), which is now runnable
15/06/03 18:34:28 INFO MemoryStore: ensureFreeSpace(19112) called with curMem=519802, maxMem=559153152
15/06/03 18:34:28 INFO MemoryStore: Block broadcast_104 stored as values in memory (estimated size 18.7 KB, free 532.7 MB)
15/06/03 18:34:28 INFO MemoryStore: ensureFreeSpace(9080) called with curMem=538914, maxMem=559153152
15/06/03 18:34:28 INFO MemoryStore: Block broadcast_104_piece0 stored as bytes in memory (estimated size 8.9 KB, free 532.7 MB)
15/06/03 18:34:28 INFO BlockManagerInfo: Added broadcast_104_piece0 in memory on localhost:60836 (size: 8.9 KB, free: 533.1 MB)
15/06/03 18:34:28 INFO BlockManagerMaster: Updated info of block broadcast_104_piece0
15/06/03 18:34:28 INFO SparkContext: Created broadcast 104 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:28 INFO DAGScheduler: Submitting 1 missing tasks from Stage 9825 (MapPartitionsRDD[501] at map at VertexRDDImpl.scala:90)
15/06/03 18:34:28 INFO TaskSchedulerImpl: Adding task set 9825.0 with 1 tasks
15/06/03 18:34:28 INFO TaskSetManager: Starting task 0.0 in stage 9825.0 (TID 103, localhost, PROCESS_LOCAL, 4066 bytes)
15/06/03 18:34:28 INFO Executor: Running task 0.0 in stage 9825.0 (TID 103)
15/06/03 18:34:28 INFO CacheManager: Partition rdd_499_0 not found, computing it
15/06/03 18:34:28 INFO BlockManager: Found block rdd_485_0 locally
15/06/03 18:34:28 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:28 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:28 INFO MemoryStore: ensureFreeSpace(1664) called with curMem=547994, maxMem=559153152
15/06/03 18:34:28 INFO MemoryStore: Block rdd_499_0 stored as values in memory (estimated size 1664.0 B, free 532.7 MB)
15/06/03 18:34:28 INFO BlockManagerInfo: Added rdd_499_0 in memory on localhost:60836 (size: 1664.0 B, free: 533.1 MB)
15/06/03 18:34:28 INFO BlockManagerMaster: Updated info of block rdd_499_0
15/06/03 18:34:28 INFO Executor: Finished task 0.0 in stage 9825.0 (TID 103). 2693 bytes result sent to driver
15/06/03 18:34:28 INFO TaskSetManager: Finished task 0.0 in stage 9825.0 (TID 103) in 9 ms on localhost (1/1)
15/06/03 18:34:28 INFO TaskSchedulerImpl: Removed TaskSet 9825.0, whose tasks have all completed, from pool
15/06/03 18:34:28 INFO DAGScheduler: Stage 9825 (reduce at VertexRDDImpl.scala:90) finished in 0.011 s
15/06/03 18:34:28 INFO DAGScheduler: Job 25 finished: reduce at VertexRDDImpl.scala:90, took 0.222677 s
15/06/03 18:34:28 INFO Pregel: Pregel finished iteration 23
15/06/03 18:34:28 INFO ZippedPartitionsRDD2: Removing RDD 480 from persistence list
15/06/03 18:34:28 INFO BlockManager: Removing RDD 480
15/06/03 18:34:28 INFO ZippedPartitionsRDD2: Removing RDD 483 from persistence list
15/06/03 18:34:28 INFO BlockManager: Removing block rdd_480_0
15/06/03 18:34:28 INFO MemoryStore: Block rdd_480_0 of size 1664 dropped from memory (free 558605158)
15/06/03 18:34:28 INFO BlockManager: Removing RDD 483
15/06/03 18:34:28 INFO BlockManager: Removing block rdd_483_0
15/06/03 18:34:28 INFO MemoryStore: Block rdd_483_0 of size 1544 dropped from memory (free 558606702)
15/06/03 18:34:28 INFO ZippedPartitionsRDD2: Removing RDD 466 from persistence list
15/06/03 18:34:28 INFO BlockManager: Removing RDD 466
15/06/03 18:34:28 INFO ZippedPartitionsRDD2: Removing RDD 472 from persistence list
15/06/03 18:34:28 INFO BlockManager: Removing block rdd_466_0
15/06/03 18:34:28 INFO MemoryStore: Block rdd_466_0 of size 1496 dropped from memory (free 558608198)
15/06/03 18:34:28 INFO BlockManager: Removing RDD 472
15/06/03 18:34:28 INFO BlockManager: Removing block rdd_472_0
15/06/03 18:34:28 INFO MemoryStore: Block rdd_472_0 of size 2976 dropped from memory (free 558611174)
15/06/03 18:34:28 INFO SparkContext: Starting job: reduce at VertexRDDImpl.scala:90
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 2 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 3 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 5 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 4 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 1 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 0 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 8 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 7 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 6 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 11 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 10 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 9 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 14 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 13 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 12 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 17 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 16 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 15 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 20 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 19 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 18 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 23 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 22 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 21 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 26 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 25 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 24 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 29 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 28 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 27 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 32 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 31 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 30 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 35 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 34 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 33 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 38 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 37 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 36 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 41 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 40 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 39 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 44 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 43 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 42 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 47 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 46 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 45 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 50 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 49 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 48 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 53 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 52 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 51 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 56 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 55 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 54 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 59 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 58 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 57 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 62 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 61 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 60 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 65 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 64 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 63 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 68 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 67 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 66 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 71 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 70 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 69 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 74 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 73 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 72 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 77 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 76 is 145 bytes
15/06/03 18:34:28 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 75 is 145 bytes
15/06/03 18:34:28 INFO DAGScheduler: Registering RDD 508 (mapPartitions at VertexRDDImpl.scala:218)
15/06/03 18:34:28 INFO DAGScheduler: Registering RDD 512 (mapPartitions at VertexRDDImpl.scala:222)
15/06/03 18:34:28 INFO DAGScheduler: Registering RDD 516 (mapPartitions at GraphImpl.scala:235)
15/06/03 18:34:28 INFO DAGScheduler: Got job 26 (reduce at VertexRDDImpl.scala:90) with 1 output partitions (allowLocal=false)
15/06/03 18:34:28 INFO DAGScheduler: Final stage: Stage 10959(reduce at VertexRDDImpl.scala:90)
15/06/03 18:34:28 INFO DAGScheduler: Parents of final stage: List(Stage 10934, Stage 10926, Stage 10905, Stage 10956, Stage 10906, Stage 10928, Stage 10958, Stage 10950, Stage 10930, Stage 10922, Stage 10952, Stage 10924, Stage 10946, Stage 10918, Stage 10948, Stage 10940, Stage 10920, Stage 10942, Stage 10914, Stage 10944, Stage 10936, Stage 10916, Stage 10908, Stage 10938, Stage 10910, Stage 10932, Stage 10954, Stage 10912)
15/06/03 18:34:28 INFO DAGScheduler: Missing parents: List(Stage 10958)
15/06/03 18:34:28 INFO DAGScheduler: Submitting Stage 10878 (ReplicatedVertexView.updateVertices - shippedVerts true false (broadcast) MapPartitionsRDD[508] at mapPartitions at VertexRDDImpl.scala:218), which has no missing parents
15/06/03 18:34:28 INFO MemoryStore: ensureFreeSpace(19352) called with curMem=541978, maxMem=559153152
15/06/03 18:34:28 INFO MemoryStore: Block broadcast_105 stored as values in memory (estimated size 18.9 KB, free 532.7 MB)
15/06/03 18:34:28 INFO MemoryStore: ensureFreeSpace(9152) called with curMem=561330, maxMem=559153152
15/06/03 18:34:28 INFO MemoryStore: Block broadcast_105_piece0 stored as bytes in memory (estimated size 8.9 KB, free 532.7 MB)
15/06/03 18:34:28 INFO BlockManagerInfo: Added broadcast_105_piece0 in memory on localhost:60836 (size: 8.9 KB, free: 533.1 MB)
15/06/03 18:34:28 INFO BlockManagerMaster: Updated info of block broadcast_105_piece0
15/06/03 18:34:28 INFO SparkContext: Created broadcast 105 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:28 INFO DAGScheduler: Submitting 1 missing tasks from Stage 10878 (ReplicatedVertexView.updateVertices - shippedVerts true false (broadcast) MapPartitionsRDD[508] at mapPartitions at VertexRDDImpl.scala:218)
15/06/03 18:34:28 INFO TaskSchedulerImpl: Adding task set 10878.0 with 1 tasks
15/06/03 18:34:28 INFO DAGScheduler: Submitting Stage 10957 (ReplicatedVertexView.withActiveSet - shippedActives (broadcast) MapPartitionsRDD[512] at mapPartitions at VertexRDDImpl.scala:222), which has no missing parents
15/06/03 18:34:28 INFO TaskSetManager: Starting task 0.0 in stage 10878.0 (TID 104, localhost, PROCESS_LOCAL, 4151 bytes)
15/06/03 18:34:28 INFO Executor: Running task 0.0 in stage 10878.0 (TID 104)
15/06/03 18:34:28 INFO MemoryStore: ensureFreeSpace(19008) called with curMem=570482, maxMem=559153152
15/06/03 18:34:28 INFO MemoryStore: Block broadcast_106 stored as values in memory (estimated size 18.6 KB, free 532.7 MB)
15/06/03 18:34:28 INFO MemoryStore: ensureFreeSpace(9035) called with curMem=589490, maxMem=559153152
15/06/03 18:34:28 INFO BlockManager: Found block rdd_485_0 locally
15/06/03 18:34:28 INFO CacheManager: Partition rdd_504_0 not found, computing it
15/06/03 18:34:28 INFO BlockManager: Found block rdd_485_0 locally
15/06/03 18:34:28 INFO MemoryStore: Block broadcast_106_piece0 stored as bytes in memory (estimated size 8.8 KB, free 532.7 MB)
15/06/03 18:34:28 INFO CacheManager: Partition rdd_502_0 not found, computing it
15/06/03 18:34:28 INFO BlockManager: Found block rdd_485_0 locally
15/06/03 18:34:28 INFO BlockManager: Found block rdd_499_0 locally
15/06/03 18:34:28 INFO BlockManagerInfo: Added broadcast_106_piece0 in memory on localhost:60836 (size: 8.8 KB, free: 533.1 MB)
15/06/03 18:34:28 INFO BlockManagerMaster: Updated info of block broadcast_106_piece0
15/06/03 18:34:28 INFO MemoryStore: ensureFreeSpace(1544) called with curMem=598525, maxMem=559153152
15/06/03 18:34:28 INFO MemoryStore: Block rdd_502_0 stored as values in memory (estimated size 1544.0 B, free 532.7 MB)
15/06/03 18:34:28 INFO SparkContext: Created broadcast 106 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:28 INFO BlockManagerInfo: Added rdd_502_0 in memory on localhost:60836 (size: 1544.0 B, free: 533.1 MB)
15/06/03 18:34:28 INFO BlockManagerMaster: Updated info of block rdd_502_0
15/06/03 18:34:28 INFO DAGScheduler: Submitting 1 missing tasks from Stage 10957 (ReplicatedVertexView.withActiveSet - shippedActives (broadcast) MapPartitionsRDD[512] at mapPartitions at VertexRDDImpl.scala:222)
15/06/03 18:34:28 INFO TaskSchedulerImpl: Adding task set 10957.0 with 1 tasks
15/06/03 18:34:28 INFO MemoryStore: ensureFreeSpace(1496) called with curMem=600069, maxMem=559153152
15/06/03 18:34:28 INFO MemoryStore: Block rdd_504_0 stored as values in memory (estimated size 1496.0 B, free 532.7 MB)
15/06/03 18:34:28 INFO BlockManagerInfo: Added rdd_504_0 in memory on localhost:60836 (size: 1496.0 B, free: 533.1 MB)
15/06/03 18:34:28 INFO BlockManagerMaster: Updated info of block rdd_504_0
15/06/03 18:34:28 INFO Executor: Finished task 0.0 in stage 10878.0 (TID 104). 2627 bytes result sent to driver
15/06/03 18:34:28 INFO TaskSetManager: Starting task 0.0 in stage 10957.0 (TID 105, localhost, PROCESS_LOCAL, 4087 bytes)
15/06/03 18:34:28 INFO Executor: Running task 0.0 in stage 10957.0 (TID 105)
15/06/03 18:34:28 INFO TaskSetManager: Finished task 0.0 in stage 10878.0 (TID 104) in 31 ms on localhost (1/1)
15/06/03 18:34:28 INFO TaskSchedulerImpl: Removed TaskSet 10878.0, whose tasks have all completed, from pool
15/06/03 18:34:28 INFO DAGScheduler: Stage 10878 (mapPartitions at VertexRDDImpl.scala:218) finished in 0.032 s
15/06/03 18:34:28 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:28 INFO DAGScheduler: running: Set(Stage 10957)
15/06/03 18:34:28 INFO DAGScheduler: waiting: Set(Stage 10959, Stage 10958)
15/06/03 18:34:28 INFO DAGScheduler: failed: Set()
15/06/03 18:34:28 INFO DAGScheduler: Missing parents for Stage 10959: List(Stage 10958)
15/06/03 18:34:28 INFO DAGScheduler: Missing parents for Stage 10958: List(Stage 10957)
15/06/03 18:34:28 INFO BlockManager: Found block rdd_502_0 locally
15/06/03 18:34:28 INFO Executor: Finished task 0.0 in stage 10957.0 (TID 105). 2003 bytes result sent to driver
15/06/03 18:34:28 INFO TaskSetManager: Finished task 0.0 in stage 10957.0 (TID 105) in 15 ms on localhost (1/1)
15/06/03 18:34:28 INFO TaskSchedulerImpl: Removed TaskSet 10957.0, whose tasks have all completed, from pool
15/06/03 18:34:28 INFO DAGScheduler: Stage 10957 (mapPartitions at VertexRDDImpl.scala:222) finished in 0.034 s
15/06/03 18:34:28 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:28 INFO DAGScheduler: running: Set()
15/06/03 18:34:28 INFO DAGScheduler: waiting: Set(Stage 10959, Stage 10958)
15/06/03 18:34:28 INFO DAGScheduler: failed: Set()
15/06/03 18:34:28 INFO DAGScheduler: Missing parents for Stage 10959: List(Stage 10958)
15/06/03 18:34:28 INFO DAGScheduler: Missing parents for Stage 10958: List()
15/06/03 18:34:28 INFO DAGScheduler: Submitting Stage 10958 (GraphImpl.aggregateMessages - preAgg MapPartitionsRDD[516] at mapPartitions at GraphImpl.scala:235), which is now runnable
15/06/03 18:34:28 INFO MemoryStore: ensureFreeSpace(12552) called with curMem=601565, maxMem=559153152
15/06/03 18:34:28 INFO MemoryStore: Block broadcast_107 stored as values in memory (estimated size 12.3 KB, free 532.7 MB)
15/06/03 18:34:29 INFO MemoryStore: ensureFreeSpace(6594) called with curMem=614117, maxMem=559153152
15/06/03 18:34:29 INFO MemoryStore: Block broadcast_107_piece0 stored as bytes in memory (estimated size 6.4 KB, free 532.7 MB)
15/06/03 18:34:29 INFO BlockManagerInfo: Added broadcast_107_piece0 in memory on localhost:60836 (size: 6.4 KB, free: 533.1 MB)
15/06/03 18:34:29 INFO BlockManagerMaster: Updated info of block broadcast_107_piece0
15/06/03 18:34:29 INFO SparkContext: Created broadcast 107 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:29 INFO DAGScheduler: Submitting 1 missing tasks from Stage 10958 (GraphImpl.aggregateMessages - preAgg MapPartitionsRDD[516] at mapPartitions at GraphImpl.scala:235)
15/06/03 18:34:29 INFO TaskSchedulerImpl: Adding task set 10958.0 with 1 tasks
15/06/03 18:34:29 INFO TaskSetManager: Starting task 0.0 in stage 10958.0 (TID 106, localhost, PROCESS_LOCAL, 2960 bytes)
15/06/03 18:34:29 INFO Executor: Running task 0.0 in stage 10958.0 (TID 106)
15/06/03 18:34:29 INFO CacheManager: Partition rdd_510_0 not found, computing it
15/06/03 18:34:29 INFO BlockManager: Found block rdd_491_0 locally
15/06/03 18:34:29 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:29 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:29 INFO MemoryStore: ensureFreeSpace(2976) called with curMem=620711, maxMem=559153152
15/06/03 18:34:29 INFO MemoryStore: Block rdd_510_0 stored as values in memory (estimated size 2.9 KB, free 532.7 MB)
15/06/03 18:34:29 INFO BlockManagerInfo: Added rdd_510_0 in memory on localhost:60836 (size: 2.9 KB, free: 533.1 MB)
15/06/03 18:34:29 INFO BlockManagerMaster: Updated info of block rdd_510_0
15/06/03 18:34:29 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:29 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:29 INFO Executor: Finished task 0.0 in stage 10958.0 (TID 106). 2788 bytes result sent to driver
15/06/03 18:34:29 INFO TaskSetManager: Finished task 0.0 in stage 10958.0 (TID 106) in 26 ms on localhost (1/1)
15/06/03 18:34:29 INFO TaskSchedulerImpl: Removed TaskSet 10958.0, whose tasks have all completed, from pool
15/06/03 18:34:29 INFO DAGScheduler: Stage 10958 (mapPartitions at GraphImpl.scala:235) finished in 0.024 s
15/06/03 18:34:29 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:29 INFO DAGScheduler: running: Set()
15/06/03 18:34:29 INFO DAGScheduler: waiting: Set(Stage 10959)
15/06/03 18:34:29 INFO DAGScheduler: failed: Set()
15/06/03 18:34:29 INFO DAGScheduler: Missing parents for Stage 10959: List()
15/06/03 18:34:29 INFO DAGScheduler: Submitting Stage 10959 (MapPartitionsRDD[520] at map at VertexRDDImpl.scala:90), which is now runnable
15/06/03 18:34:29 INFO MemoryStore: ensureFreeSpace(19640) called with curMem=623687, maxMem=559153152
15/06/03 18:34:29 INFO MemoryStore: Block broadcast_108 stored as values in memory (estimated size 19.2 KB, free 532.6 MB)
15/06/03 18:34:29 INFO MemoryStore: ensureFreeSpace(9297) called with curMem=643327, maxMem=559153152
15/06/03 18:34:29 INFO MemoryStore: Block broadcast_108_piece0 stored as bytes in memory (estimated size 9.1 KB, free 532.6 MB)
15/06/03 18:34:29 INFO BlockManagerInfo: Added broadcast_108_piece0 in memory on localhost:60836 (size: 9.1 KB, free: 533.1 MB)
15/06/03 18:34:29 INFO BlockManagerMaster: Updated info of block broadcast_108_piece0
15/06/03 18:34:29 INFO SparkContext: Created broadcast 108 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:29 INFO DAGScheduler: Submitting 1 missing tasks from Stage 10959 (MapPartitionsRDD[520] at map at VertexRDDImpl.scala:90)
15/06/03 18:34:29 INFO TaskSchedulerImpl: Adding task set 10959.0 with 1 tasks
15/06/03 18:34:29 INFO TaskSetManager: Starting task 0.0 in stage 10959.0 (TID 107, localhost, PROCESS_LOCAL, 4171 bytes)
15/06/03 18:34:29 INFO Executor: Running task 0.0 in stage 10959.0 (TID 107)
15/06/03 18:34:29 INFO CacheManager: Partition rdd_518_0 not found, computing it
15/06/03 18:34:29 INFO BlockManager: Found block rdd_504_0 locally
15/06/03 18:34:29 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:29 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:29 INFO MemoryStore: ensureFreeSpace(1664) called with curMem=652624, maxMem=559153152
15/06/03 18:34:29 INFO MemoryStore: Block rdd_518_0 stored as values in memory (estimated size 1664.0 B, free 532.6 MB)
15/06/03 18:34:29 INFO BlockManagerInfo: Added rdd_518_0 in memory on localhost:60836 (size: 1664.0 B, free: 533.1 MB)
15/06/03 18:34:29 INFO BlockManagerMaster: Updated info of block rdd_518_0
15/06/03 18:34:29 INFO Executor: Finished task 0.0 in stage 10959.0 (TID 107). 2693 bytes result sent to driver
15/06/03 18:34:29 INFO TaskSetManager: Finished task 0.0 in stage 10959.0 (TID 107) in 13 ms on localhost (1/1)
15/06/03 18:34:29 INFO TaskSchedulerImpl: Removed TaskSet 10959.0, whose tasks have all completed, from pool
15/06/03 18:34:29 INFO DAGScheduler: Stage 10959 (reduce at VertexRDDImpl.scala:90) finished in 0.014 s
15/06/03 18:34:29 INFO DAGScheduler: Job 26 finished: reduce at VertexRDDImpl.scala:90, took 0.351283 s
15/06/03 18:34:29 INFO Pregel: Pregel finished iteration 24
15/06/03 18:34:29 INFO ZippedPartitionsRDD2: Removing RDD 499 from persistence list
15/06/03 18:34:29 INFO BlockManager: Removing RDD 499
15/06/03 18:34:29 INFO BlockManager: Removing block rdd_499_0
15/06/03 18:34:29 INFO MemoryStore: Block rdd_499_0 of size 1664 dropped from memory (free 558500528)
15/06/03 18:34:29 INFO ZippedPartitionsRDD2: Removing RDD 502 from persistence list
15/06/03 18:34:29 INFO BlockManager: Removing RDD 502
15/06/03 18:34:29 INFO ZippedPartitionsRDD2: Removing RDD 485 from persistence list
15/06/03 18:34:29 INFO BlockManager: Removing block rdd_502_0
15/06/03 18:34:29 INFO MemoryStore: Block rdd_502_0 of size 1544 dropped from memory (free 558502072)
15/06/03 18:34:29 INFO BlockManager: Removing RDD 485
15/06/03 18:34:29 INFO ZippedPartitionsRDD2: Removing RDD 491 from persistence list
15/06/03 18:34:29 INFO BlockManager: Removing block rdd_485_0
15/06/03 18:34:29 INFO MemoryStore: Block rdd_485_0 of size 1496 dropped from memory (free 558503568)
15/06/03 18:34:29 INFO BlockManager: Removing RDD 491
15/06/03 18:34:29 INFO BlockManager: Removing block rdd_491_0
15/06/03 18:34:29 INFO MemoryStore: Block rdd_491_0 of size 2976 dropped from memory (free 558506544)
15/06/03 18:34:29 INFO SparkContext: Starting job: reduce at VertexRDDImpl.scala:90
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 2 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 3 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 5 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 4 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 1 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 0 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 8 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 7 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 6 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 11 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 10 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 9 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 14 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 13 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 12 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 17 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 16 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 15 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 20 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 19 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 18 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 23 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 22 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 21 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 26 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 25 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 24 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 29 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 28 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 27 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 32 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 31 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 30 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 35 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 34 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 33 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 38 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 37 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 36 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 41 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 40 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 39 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 44 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 43 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 42 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 47 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 46 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 45 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 50 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 49 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 48 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 53 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 52 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 51 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 56 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 55 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 54 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 59 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 58 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 57 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 62 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 61 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 60 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 65 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 64 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 63 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 68 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 67 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 66 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 71 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 70 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 69 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 74 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 73 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 72 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 77 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 76 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 75 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 80 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 79 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 78 is 145 bytes
15/06/03 18:34:29 INFO DAGScheduler: Registering RDD 527 (mapPartitions at VertexRDDImpl.scala:218)
15/06/03 18:34:29 INFO DAGScheduler: Registering RDD 531 (mapPartitions at VertexRDDImpl.scala:222)
15/06/03 18:34:29 INFO DAGScheduler: Registering RDD 535 (mapPartitions at GraphImpl.scala:235)
15/06/03 18:34:29 INFO DAGScheduler: Got job 27 (reduce at VertexRDDImpl.scala:90) with 1 output partitions (allowLocal=false)
15/06/03 18:34:29 INFO DAGScheduler: Final stage: Stage 12177(reduce at VertexRDDImpl.scala:90)
15/06/03 18:34:29 INFO DAGScheduler: Parents of final stage: List(Stage 12124, Stage 12146, Stage 12176, Stage 12168, Stage 12126, Stage 12148, Stage 12140, Stage 12170, Stage 12142, Stage 12121, Stage 12172, Stage 12164, Stage 12144, Stage 12136, Stage 12166, Stage 12138, Stage 12160, Stage 12132, Stage 12162, Stage 12154, Stage 12134, Stage 12156, Stage 12128, Stage 12158, Stage 12150, Stage 12130, Stage 12122, Stage 12152, Stage 12174)
15/06/03 18:34:29 INFO DAGScheduler: Missing parents: List(Stage 12176)
15/06/03 18:34:29 INFO DAGScheduler: Submitting Stage 12093 (ReplicatedVertexView.updateVertices - shippedVerts true false (broadcast) MapPartitionsRDD[527] at mapPartitions at VertexRDDImpl.scala:218), which has no missing parents
15/06/03 18:34:29 INFO MemoryStore: ensureFreeSpace(19880) called with curMem=646608, maxMem=559153152
15/06/03 18:34:29 INFO MemoryStore: Block broadcast_109 stored as values in memory (estimated size 19.4 KB, free 532.6 MB)
15/06/03 18:34:29 INFO MemoryStore: ensureFreeSpace(9320) called with curMem=666488, maxMem=559153152
15/06/03 18:34:29 INFO MemoryStore: Block broadcast_109_piece0 stored as bytes in memory (estimated size 9.1 KB, free 532.6 MB)
15/06/03 18:34:29 INFO BlockManagerInfo: Added broadcast_109_piece0 in memory on localhost:60836 (size: 9.1 KB, free: 533.1 MB)
15/06/03 18:34:29 INFO BlockManagerMaster: Updated info of block broadcast_109_piece0
15/06/03 18:34:29 INFO SparkContext: Created broadcast 109 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:29 INFO BlockManager: Removing broadcast 100
15/06/03 18:34:29 INFO BlockManager: Removing block broadcast_100
15/06/03 18:34:29 INFO MemoryStore: Block broadcast_100 of size 18584 dropped from memory (free 558495928)
15/06/03 18:34:29 INFO BlockManager: Removing block broadcast_100_piece0
15/06/03 18:34:29 INFO MemoryStore: Block broadcast_100_piece0 of size 8901 dropped from memory (free 558504829)
15/06/03 18:34:29 INFO BlockManagerInfo: Removed broadcast_100_piece0 on localhost:60836 in memory (size: 8.7 KB, free: 533.1 MB)
15/06/03 18:34:29 INFO BlockManagerMaster: Updated info of block broadcast_100_piece0
15/06/03 18:34:29 INFO ContextCleaner: Cleaned broadcast 100
15/06/03 18:34:29 INFO DAGScheduler: Submitting 1 missing tasks from Stage 12093 (ReplicatedVertexView.updateVertices - shippedVerts true false (broadcast) MapPartitionsRDD[527] at mapPartitions at VertexRDDImpl.scala:218)
15/06/03 18:34:29 INFO BlockManager: Removing broadcast 99
15/06/03 18:34:29 INFO TaskSchedulerImpl: Adding task set 12093.0 with 1 tasks
15/06/03 18:34:29 INFO BlockManager: Removing block broadcast_99_piece0
15/06/03 18:34:29 INFO MemoryStore: Block broadcast_99_piece0 of size 6516 dropped from memory (free 558511345)
15/06/03 18:34:29 INFO BlockManagerInfo: Removed broadcast_99_piece0 on localhost:60836 in memory (size: 6.4 KB, free: 533.1 MB)
15/06/03 18:34:29 INFO BlockManagerMaster: Updated info of block broadcast_99_piece0
15/06/03 18:34:29 INFO BlockManager: Removing block broadcast_99
15/06/03 18:34:29 INFO MemoryStore: Block broadcast_99 of size 12096 dropped from memory (free 558523441)
15/06/03 18:34:29 INFO ContextCleaner: Cleaned broadcast 99
15/06/03 18:34:29 INFO BlockManager: Removing broadcast 98
15/06/03 18:34:29 INFO BlockManager: Removing block broadcast_98_piece0
15/06/03 18:34:29 INFO MemoryStore: Block broadcast_98_piece0 of size 8606 dropped from memory (free 558532047)
15/06/03 18:34:29 INFO BlockManagerInfo: Removed broadcast_98_piece0 on localhost:60836 in memory (size: 8.4 KB, free: 533.1 MB)
15/06/03 18:34:29 INFO TaskSetManager: Starting task 0.0 in stage 12093.0 (TID 108, localhost, PROCESS_LOCAL, 4256 bytes)
15/06/03 18:34:29 INFO BlockManagerMaster: Updated info of block broadcast_98_piece0
15/06/03 18:34:29 INFO BlockManager: Removing block broadcast_98
15/06/03 18:34:29 INFO MemoryStore: Block broadcast_98 of size 17952 dropped from memory (free 558549999)
15/06/03 18:34:29 INFO Executor: Running task 0.0 in stage 12093.0 (TID 108)
15/06/03 18:34:29 INFO ContextCleaner: Cleaned broadcast 98
15/06/03 18:34:29 INFO BlockManager: Removing broadcast 97
15/06/03 18:34:29 INFO BlockManager: Removing block broadcast_97
15/06/03 18:34:29 INFO MemoryStore: Block broadcast_97 of size 18296 dropped from memory (free 558568295)
15/06/03 18:34:29 INFO BlockManager: Removing block broadcast_97_piece0
15/06/03 18:34:29 INFO MemoryStore: Block broadcast_97_piece0 of size 8733 dropped from memory (free 558577028)
15/06/03 18:34:29 INFO BlockManagerInfo: Removed broadcast_97_piece0 on localhost:60836 in memory (size: 8.5 KB, free: 533.1 MB)
15/06/03 18:34:29 INFO BlockManagerMaster: Updated info of block broadcast_97_piece0
15/06/03 18:34:29 INFO ContextCleaner: Cleaned broadcast 97
15/06/03 18:34:29 INFO DAGScheduler: Submitting Stage 12175 (ReplicatedVertexView.withActiveSet - shippedActives (broadcast) MapPartitionsRDD[531] at mapPartitions at VertexRDDImpl.scala:222), which has no missing parents
15/06/03 18:34:29 INFO BlockManager: Removing broadcast 96
15/06/03 18:34:29 INFO BlockManager: Removing block broadcast_96_piece0
15/06/03 18:34:29 INFO MemoryStore: Block broadcast_96_piece0 of size 8654 dropped from memory (free 558585682)
15/06/03 18:34:29 INFO BlockManagerInfo: Removed broadcast_96_piece0 on localhost:60836 in memory (size: 8.5 KB, free: 533.1 MB)
15/06/03 18:34:29 INFO BlockManagerMaster: Updated info of block broadcast_96_piece0
15/06/03 18:34:29 INFO BlockManager: Removing block broadcast_96
15/06/03 18:34:29 INFO MemoryStore: Block broadcast_96 of size 18056 dropped from memory (free 558603738)
15/06/03 18:34:29 INFO ContextCleaner: Cleaned broadcast 96
15/06/03 18:34:29 INFO BlockManager: Removing broadcast 95
15/06/03 18:34:29 INFO BlockManager: Removing block broadcast_95
15/06/03 18:34:29 INFO MemoryStore: Block broadcast_95 of size 11864 dropped from memory (free 558615602)
15/06/03 18:34:29 INFO BlockManager: Removing block broadcast_95_piece0
15/06/03 18:34:29 INFO MemoryStore: Block broadcast_95_piece0 of size 6452 dropped from memory (free 558622054)
15/06/03 18:34:29 INFO BlockManagerInfo: Removed broadcast_95_piece0 on localhost:60836 in memory (size: 6.3 KB, free: 533.1 MB)
15/06/03 18:34:29 INFO BlockManagerMaster: Updated info of block broadcast_95_piece0
15/06/03 18:34:29 INFO ContextCleaner: Cleaned broadcast 95
15/06/03 18:34:29 INFO BlockManager: Removing broadcast 94
15/06/03 18:34:29 INFO BlockManager: Removing block broadcast_94_piece0
15/06/03 18:34:29 INFO MemoryStore: Block broadcast_94_piece0 of size 8445 dropped from memory (free 558630499)
15/06/03 18:34:29 INFO BlockManagerInfo: Removed broadcast_94_piece0 on localhost:60836 in memory (size: 8.2 KB, free: 533.1 MB)
15/06/03 18:34:29 INFO BlockManagerMaster: Updated info of block broadcast_94_piece0
15/06/03 18:34:29 INFO BlockManager: Removing block broadcast_94
15/06/03 18:34:29 INFO MemoryStore: Block broadcast_94 of size 17424 dropped from memory (free 558647923)
15/06/03 18:34:29 INFO ContextCleaner: Cleaned broadcast 94
15/06/03 18:34:29 INFO BlockManager: Removing broadcast 108
15/06/03 18:34:29 INFO BlockManager: Removing block broadcast_108
15/06/03 18:34:29 INFO BlockManager: Found block rdd_504_0 locally
15/06/03 18:34:29 INFO MemoryStore: Block broadcast_108 of size 19640 dropped from memory (free 558667563)
15/06/03 18:34:29 INFO BlockManager: Removing block broadcast_108_piece0
15/06/03 18:34:29 INFO CacheManager: Partition rdd_523_0 not found, computing it
15/06/03 18:34:29 INFO MemoryStore: Block broadcast_108_piece0 of size 9297 dropped from memory (free 558676860)
15/06/03 18:34:29 INFO BlockManager: Found block rdd_504_0 locally
15/06/03 18:34:29 INFO BlockManagerInfo: Removed broadcast_108_piece0 on localhost:60836 in memory (size: 9.1 KB, free: 533.1 MB)
15/06/03 18:34:29 INFO BlockManagerMaster: Updated info of block broadcast_108_piece0
15/06/03 18:34:29 INFO CacheManager: Partition rdd_521_0 not found, computing it
15/06/03 18:34:29 INFO BlockManager: Found block rdd_504_0 locally
15/06/03 18:34:29 INFO ContextCleaner: Cleaned broadcast 108
15/06/03 18:34:29 INFO BlockManager: Found block rdd_518_0 locally
15/06/03 18:34:29 INFO MemoryStore: ensureFreeSpace(1544) called with curMem=476292, maxMem=559153152
15/06/03 18:34:29 INFO BlockManager: Removing broadcast 107
15/06/03 18:34:29 INFO MemoryStore: Block rdd_521_0 stored as values in memory (estimated size 1544.0 B, free 532.8 MB)
15/06/03 18:34:29 INFO BlockManager: Removing block broadcast_107
15/06/03 18:34:29 INFO MemoryStore: Block broadcast_107 of size 12552 dropped from memory (free 558687868)
15/06/03 18:34:29 INFO BlockManager: Removing block broadcast_107_piece0
15/06/03 18:34:29 INFO MemoryStore: Block broadcast_107_piece0 of size 6594 dropped from memory (free 558694462)
15/06/03 18:34:29 INFO BlockManagerInfo: Added rdd_521_0 in memory on localhost:60836 (size: 1544.0 B, free: 533.1 MB)
15/06/03 18:34:29 INFO BlockManagerMaster: Updated info of block rdd_521_0
15/06/03 18:34:29 INFO BlockManagerInfo: Removed broadcast_107_piece0 on localhost:60836 in memory (size: 6.4 KB, free: 533.1 MB)
15/06/03 18:34:29 INFO BlockManagerMaster: Updated info of block broadcast_107_piece0
15/06/03 18:34:29 INFO MemoryStore: ensureFreeSpace(19536) called with curMem=458690, maxMem=559153152
15/06/03 18:34:29 INFO MemoryStore: Block broadcast_110 stored as values in memory (estimated size 19.1 KB, free 532.8 MB)
15/06/03 18:34:29 INFO ContextCleaner: Cleaned broadcast 107
15/06/03 18:34:29 INFO MemoryStore: ensureFreeSpace(1496) called with curMem=478226, maxMem=559153152
15/06/03 18:34:29 INFO MemoryStore: Block rdd_523_0 stored as values in memory (estimated size 1496.0 B, free 532.8 MB)
15/06/03 18:34:29 INFO MemoryStore: ensureFreeSpace(9166) called with curMem=479722, maxMem=559153152
15/06/03 18:34:29 INFO BlockManagerInfo: Added rdd_523_0 in memory on localhost:60836 (size: 1496.0 B, free: 533.1 MB)
15/06/03 18:34:29 INFO BlockManager: Removing broadcast 106
15/06/03 18:34:29 INFO MemoryStore: Block broadcast_110_piece0 stored as bytes in memory (estimated size 9.0 KB, free 532.8 MB)
15/06/03 18:34:29 INFO BlockManagerMaster: Updated info of block rdd_523_0
15/06/03 18:34:29 INFO BlockManager: Removing block broadcast_106
15/06/03 18:34:29 INFO MemoryStore: Block broadcast_106 of size 19008 dropped from memory (free 558683272)
15/06/03 18:34:29 INFO BlockManager: Removing block broadcast_106_piece0
15/06/03 18:34:29 INFO MemoryStore: Block broadcast_106_piece0 of size 9035 dropped from memory (free 558692307)
15/06/03 18:34:29 INFO BlockManagerInfo: Removed broadcast_106_piece0 on localhost:60836 in memory (size: 8.8 KB, free: 533.1 MB)
15/06/03 18:34:29 INFO BlockManagerMaster: Updated info of block broadcast_106_piece0
15/06/03 18:34:29 INFO BlockManagerInfo: Added broadcast_110_piece0 in memory on localhost:60836 (size: 9.0 KB, free: 533.1 MB)
15/06/03 18:34:29 INFO ContextCleaner: Cleaned broadcast 106
15/06/03 18:34:29 INFO BlockManagerMaster: Updated info of block broadcast_110_piece0
15/06/03 18:34:29 INFO BlockManager: Removing broadcast 105
15/06/03 18:34:29 INFO BlockManager: Removing block broadcast_105_piece0
15/06/03 18:34:29 INFO MemoryStore: Block broadcast_105_piece0 of size 9152 dropped from memory (free 558701459)
15/06/03 18:34:29 INFO SparkContext: Created broadcast 110 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:29 INFO DAGScheduler: Submitting 1 missing tasks from Stage 12175 (ReplicatedVertexView.withActiveSet - shippedActives (broadcast) MapPartitionsRDD[531] at mapPartitions at VertexRDDImpl.scala:222)
15/06/03 18:34:29 INFO TaskSchedulerImpl: Adding task set 12175.0 with 1 tasks
15/06/03 18:34:29 INFO BlockManagerInfo: Removed broadcast_105_piece0 on localhost:60836 in memory (size: 8.9 KB, free: 533.1 MB)
15/06/03 18:34:29 INFO BlockManagerMaster: Updated info of block broadcast_105_piece0
15/06/03 18:34:29 INFO BlockManager: Removing block broadcast_105
15/06/03 18:34:29 INFO MemoryStore: Block broadcast_105 of size 19352 dropped from memory (free 558720811)
15/06/03 18:34:29 INFO ContextCleaner: Cleaned broadcast 105
15/06/03 18:34:29 INFO BlockManager: Removing broadcast 104
15/06/03 18:34:29 INFO BlockManager: Removing block broadcast_104_piece0
15/06/03 18:34:29 INFO MemoryStore: Block broadcast_104_piece0 of size 9080 dropped from memory (free 558729891)
15/06/03 18:34:29 INFO BlockManagerInfo: Removed broadcast_104_piece0 on localhost:60836 in memory (size: 8.9 KB, free: 533.1 MB)
15/06/03 18:34:29 INFO BlockManagerMaster: Updated info of block broadcast_104_piece0
15/06/03 18:34:29 INFO BlockManager: Removing block broadcast_104
15/06/03 18:34:29 INFO MemoryStore: Block broadcast_104 of size 19112 dropped from memory (free 558749003)
15/06/03 18:34:29 INFO ContextCleaner: Cleaned broadcast 104
15/06/03 18:34:29 INFO BlockManager: Removing broadcast 103
15/06/03 18:34:29 INFO BlockManager: Removing block broadcast_103
15/06/03 18:34:29 INFO MemoryStore: Block broadcast_103 of size 12320 dropped from memory (free 558761323)
15/06/03 18:34:29 INFO BlockManager: Removing block broadcast_103_piece0
15/06/03 18:34:29 INFO MemoryStore: Block broadcast_103_piece0 of size 6678 dropped from memory (free 558768001)
15/06/03 18:34:29 INFO BlockManagerInfo: Removed broadcast_103_piece0 on localhost:60836 in memory (size: 6.5 KB, free: 533.1 MB)
15/06/03 18:34:29 INFO BlockManagerMaster: Updated info of block broadcast_103_piece0
15/06/03 18:34:29 INFO ContextCleaner: Cleaned broadcast 103
15/06/03 18:34:29 INFO BlockManager: Removing broadcast 102
15/06/03 18:34:29 INFO BlockManager: Removing block broadcast_102_piece0
15/06/03 18:34:29 INFO MemoryStore: Block broadcast_102_piece0 of size 8772 dropped from memory (free 558776773)
15/06/03 18:34:29 INFO BlockManagerInfo: Removed broadcast_102_piece0 on localhost:60836 in memory (size: 8.6 KB, free: 533.2 MB)
15/06/03 18:34:29 INFO BlockManagerMaster: Updated info of block broadcast_102_piece0
15/06/03 18:34:29 INFO BlockManager: Removing block broadcast_102
15/06/03 18:34:29 INFO MemoryStore: Block broadcast_102 of size 18480 dropped from memory (free 558795253)
15/06/03 18:34:29 INFO ContextCleaner: Cleaned broadcast 102
15/06/03 18:34:29 INFO BlockManager: Removing broadcast 101
15/06/03 18:34:29 INFO BlockManager: Removing block broadcast_101
15/06/03 18:34:29 INFO MemoryStore: Block broadcast_101 of size 18824 dropped from memory (free 558814077)
15/06/03 18:34:29 INFO BlockManager: Removing block broadcast_101_piece0
15/06/03 18:34:29 INFO MemoryStore: Block broadcast_101_piece0 of size 8915 dropped from memory (free 558822992)
15/06/03 18:34:29 INFO BlockManagerInfo: Removed broadcast_101_piece0 on localhost:60836 in memory (size: 8.7 KB, free: 533.2 MB)
15/06/03 18:34:29 INFO BlockManagerMaster: Updated info of block broadcast_101_piece0
15/06/03 18:34:29 INFO ContextCleaner: Cleaned broadcast 101
15/06/03 18:34:29 INFO Executor: Finished task 0.0 in stage 12093.0 (TID 108). 2627 bytes result sent to driver
15/06/03 18:34:29 INFO TaskSetManager: Starting task 0.0 in stage 12175.0 (TID 109, localhost, PROCESS_LOCAL, 4192 bytes)
15/06/03 18:34:29 INFO Executor: Running task 0.0 in stage 12175.0 (TID 109)
15/06/03 18:34:29 INFO TaskSetManager: Finished task 0.0 in stage 12093.0 (TID 108) in 38 ms on localhost (1/1)
15/06/03 18:34:29 INFO TaskSchedulerImpl: Removed TaskSet 12093.0, whose tasks have all completed, from pool
15/06/03 18:34:29 INFO DAGScheduler: Stage 12093 (mapPartitions at VertexRDDImpl.scala:218) finished in 0.037 s
15/06/03 18:34:29 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:29 INFO DAGScheduler: running: Set(Stage 12175)
15/06/03 18:34:29 INFO DAGScheduler: waiting: Set(Stage 12176, Stage 12177)
15/06/03 18:34:29 INFO DAGScheduler: failed: Set()
15/06/03 18:34:29 INFO DAGScheduler: Missing parents for Stage 12176: List(Stage 12175)
15/06/03 18:34:29 INFO DAGScheduler: Missing parents for Stage 12177: List(Stage 12176)
15/06/03 18:34:29 INFO BlockManager: Found block rdd_521_0 locally
15/06/03 18:34:29 INFO Executor: Finished task 0.0 in stage 12175.0 (TID 109). 2003 bytes result sent to driver
15/06/03 18:34:29 INFO TaskSetManager: Finished task 0.0 in stage 12175.0 (TID 109) in 17 ms on localhost (1/1)
15/06/03 18:34:29 INFO TaskSchedulerImpl: Removed TaskSet 12175.0, whose tasks have all completed, from pool
15/06/03 18:34:29 INFO DAGScheduler: Stage 12175 (mapPartitions at VertexRDDImpl.scala:222) finished in 0.035 s
15/06/03 18:34:29 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:29 INFO DAGScheduler: running: Set()
15/06/03 18:34:29 INFO DAGScheduler: waiting: Set(Stage 12176, Stage 12177)
15/06/03 18:34:29 INFO DAGScheduler: failed: Set()
15/06/03 18:34:29 INFO DAGScheduler: Missing parents for Stage 12176: List()
15/06/03 18:34:29 INFO DAGScheduler: Missing parents for Stage 12177: List(Stage 12176)
15/06/03 18:34:29 INFO DAGScheduler: Submitting Stage 12176 (GraphImpl.aggregateMessages - preAgg MapPartitionsRDD[535] at mapPartitions at GraphImpl.scala:235), which is now runnable
15/06/03 18:34:29 INFO MemoryStore: ensureFreeSpace(12776) called with curMem=330160, maxMem=559153152
15/06/03 18:34:29 INFO MemoryStore: Block broadcast_111 stored as values in memory (estimated size 12.5 KB, free 532.9 MB)
15/06/03 18:34:29 INFO MemoryStore: ensureFreeSpace(6791) called with curMem=342936, maxMem=559153152
15/06/03 18:34:29 INFO MemoryStore: Block broadcast_111_piece0 stored as bytes in memory (estimated size 6.6 KB, free 532.9 MB)
15/06/03 18:34:29 INFO BlockManagerInfo: Added broadcast_111_piece0 in memory on localhost:60836 (size: 6.6 KB, free: 533.2 MB)
15/06/03 18:34:29 INFO BlockManagerMaster: Updated info of block broadcast_111_piece0
15/06/03 18:34:29 INFO SparkContext: Created broadcast 111 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:29 INFO DAGScheduler: Submitting 1 missing tasks from Stage 12176 (GraphImpl.aggregateMessages - preAgg MapPartitionsRDD[535] at mapPartitions at GraphImpl.scala:235)
15/06/03 18:34:29 INFO TaskSchedulerImpl: Adding task set 12176.0 with 1 tasks
15/06/03 18:34:29 INFO TaskSetManager: Starting task 0.0 in stage 12176.0 (TID 110, localhost, PROCESS_LOCAL, 3001 bytes)
15/06/03 18:34:29 INFO Executor: Running task 0.0 in stage 12176.0 (TID 110)
15/06/03 18:34:29 INFO CacheManager: Partition rdd_529_0 not found, computing it
15/06/03 18:34:29 INFO BlockManager: Found block rdd_510_0 locally
15/06/03 18:34:29 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:29 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:29 INFO MemoryStore: ensureFreeSpace(2976) called with curMem=349727, maxMem=559153152
15/06/03 18:34:29 INFO MemoryStore: Block rdd_529_0 stored as values in memory (estimated size 2.9 KB, free 532.9 MB)
15/06/03 18:34:29 INFO BlockManagerInfo: Added rdd_529_0 in memory on localhost:60836 (size: 2.9 KB, free: 533.2 MB)
15/06/03 18:34:29 INFO BlockManagerMaster: Updated info of block rdd_529_0
15/06/03 18:34:29 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:29 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:29 INFO Executor: Finished task 0.0 in stage 12176.0 (TID 110). 2788 bytes result sent to driver
15/06/03 18:34:29 INFO TaskSetManager: Finished task 0.0 in stage 12176.0 (TID 110) in 20 ms on localhost (1/1)
15/06/03 18:34:29 INFO TaskSchedulerImpl: Removed TaskSet 12176.0, whose tasks have all completed, from pool
15/06/03 18:34:29 INFO DAGScheduler: Stage 12176 (mapPartitions at GraphImpl.scala:235) finished in 0.020 s
15/06/03 18:34:29 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:29 INFO DAGScheduler: running: Set()
15/06/03 18:34:29 INFO DAGScheduler: waiting: Set(Stage 12177)
15/06/03 18:34:29 INFO DAGScheduler: failed: Set()
15/06/03 18:34:29 INFO DAGScheduler: Missing parents for Stage 12177: List()
15/06/03 18:34:29 INFO DAGScheduler: Submitting Stage 12177 (MapPartitionsRDD[539] at map at VertexRDDImpl.scala:90), which is now runnable
15/06/03 18:34:29 INFO MemoryStore: ensureFreeSpace(20168) called with curMem=352703, maxMem=559153152
15/06/03 18:34:29 INFO MemoryStore: Block broadcast_112 stored as values in memory (estimated size 19.7 KB, free 532.9 MB)
15/06/03 18:34:29 INFO MemoryStore: ensureFreeSpace(9404) called with curMem=372871, maxMem=559153152
15/06/03 18:34:29 INFO MemoryStore: Block broadcast_112_piece0 stored as bytes in memory (estimated size 9.2 KB, free 532.9 MB)
15/06/03 18:34:29 INFO BlockManagerInfo: Added broadcast_112_piece0 in memory on localhost:60836 (size: 9.2 KB, free: 533.1 MB)
15/06/03 18:34:29 INFO BlockManagerMaster: Updated info of block broadcast_112_piece0
15/06/03 18:34:29 INFO SparkContext: Created broadcast 112 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:29 INFO DAGScheduler: Submitting 1 missing tasks from Stage 12177 (MapPartitionsRDD[539] at map at VertexRDDImpl.scala:90)
15/06/03 18:34:29 INFO TaskSchedulerImpl: Adding task set 12177.0 with 1 tasks
15/06/03 18:34:29 INFO TaskSetManager: Starting task 0.0 in stage 12177.0 (TID 111, localhost, PROCESS_LOCAL, 4276 bytes)
15/06/03 18:34:29 INFO Executor: Running task 0.0 in stage 12177.0 (TID 111)
15/06/03 18:34:29 INFO CacheManager: Partition rdd_537_0 not found, computing it
15/06/03 18:34:29 INFO BlockManager: Found block rdd_523_0 locally
15/06/03 18:34:29 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:29 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:29 INFO MemoryStore: ensureFreeSpace(1664) called with curMem=382275, maxMem=559153152
15/06/03 18:34:29 INFO MemoryStore: Block rdd_537_0 stored as values in memory (estimated size 1664.0 B, free 532.9 MB)
15/06/03 18:34:29 INFO BlockManagerInfo: Added rdd_537_0 in memory on localhost:60836 (size: 1664.0 B, free: 533.1 MB)
15/06/03 18:34:29 INFO BlockManagerMaster: Updated info of block rdd_537_0
15/06/03 18:34:29 INFO Executor: Finished task 0.0 in stage 12177.0 (TID 111). 2693 bytes result sent to driver
15/06/03 18:34:29 INFO TaskSetManager: Finished task 0.0 in stage 12177.0 (TID 111) in 12 ms on localhost (1/1)
15/06/03 18:34:29 INFO TaskSchedulerImpl: Removed TaskSet 12177.0, whose tasks have all completed, from pool
15/06/03 18:34:29 INFO DAGScheduler: Stage 12177 (reduce at VertexRDDImpl.scala:90) finished in 0.012 s
15/06/03 18:34:29 INFO DAGScheduler: Job 27 finished: reduce at VertexRDDImpl.scala:90, took 0.374022 s
15/06/03 18:34:29 INFO Pregel: Pregel finished iteration 25
15/06/03 18:34:29 INFO ZippedPartitionsRDD2: Removing RDD 518 from persistence list
15/06/03 18:34:29 INFO BlockManager: Removing RDD 518
15/06/03 18:34:29 INFO BlockManager: Removing block rdd_518_0
15/06/03 18:34:29 INFO MemoryStore: Block rdd_518_0 of size 1664 dropped from memory (free 558770877)
15/06/03 18:34:29 INFO ZippedPartitionsRDD2: Removing RDD 521 from persistence list
15/06/03 18:34:29 INFO BlockManager: Removing RDD 521
15/06/03 18:34:29 INFO ZippedPartitionsRDD2: Removing RDD 504 from persistence list
15/06/03 18:34:29 INFO BlockManager: Removing block rdd_521_0
15/06/03 18:34:29 INFO MemoryStore: Block rdd_521_0 of size 1544 dropped from memory (free 558772421)
15/06/03 18:34:29 INFO BlockManager: Removing RDD 504
15/06/03 18:34:29 INFO BlockManager: Removing block rdd_504_0
15/06/03 18:34:29 INFO MemoryStore: Block rdd_504_0 of size 1496 dropped from memory (free 558773917)
15/06/03 18:34:29 INFO ZippedPartitionsRDD2: Removing RDD 510 from persistence list
15/06/03 18:34:29 INFO BlockManager: Removing RDD 510
15/06/03 18:34:29 INFO BlockManager: Removing block rdd_510_0
15/06/03 18:34:29 INFO MemoryStore: Block rdd_510_0 of size 2976 dropped from memory (free 558776893)
15/06/03 18:34:29 INFO SparkContext: Starting job: reduce at VertexRDDImpl.scala:90
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 2 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 3 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 5 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 4 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 1 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 0 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 8 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 7 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 6 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 11 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 10 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 9 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 14 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 13 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 12 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 17 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 16 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 15 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 20 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 19 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 18 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 23 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 22 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 21 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 26 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 25 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 24 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 29 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 28 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 27 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 32 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 31 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 30 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 35 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 34 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 33 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 38 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 37 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 36 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 41 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 40 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 39 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 44 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 43 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 42 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 47 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 46 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 45 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 50 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 49 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 48 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 53 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 52 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 51 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 56 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 55 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 54 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 59 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 58 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 57 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 62 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 61 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 60 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 65 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 64 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 63 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 68 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 67 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 66 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 71 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 70 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 69 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 74 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 73 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 72 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 77 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 76 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 75 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 80 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 79 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 78 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 83 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 82 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 81 is 145 bytes
15/06/03 18:34:29 INFO DAGScheduler: Registering RDD 546 (mapPartitions at VertexRDDImpl.scala:218)
15/06/03 18:34:29 INFO DAGScheduler: Registering RDD 550 (mapPartitions at VertexRDDImpl.scala:222)
15/06/03 18:34:29 INFO DAGScheduler: Registering RDD 554 (mapPartitions at GraphImpl.scala:235)
15/06/03 18:34:29 INFO DAGScheduler: Got job 28 (reduce at VertexRDDImpl.scala:90) with 1 output partitions (allowLocal=false)
15/06/03 18:34:29 INFO DAGScheduler: Final stage: Stage 13482(reduce at VertexRDDImpl.scala:90)
15/06/03 18:34:29 INFO DAGScheduler: Parents of final stage: List(Stage 13437, Stage 13467, Stage 13439, Stage 13469, Stage 13461, Stage 13441, Stage 13433, Stage 13463, Stage 13435, Stage 13457, Stage 13479, Stage 13429, Stage 13459, Stage 13451, Stage 13481, Stage 13431, Stage 13453, Stage 13424, Stage 13475, Stage 13425, Stage 13455, Stage 13447, Stage 13477, Stage 13427, Stage 13449, Stage 13471, Stage 13443, Stage 13473, Stage 13465, Stage 13445)
15/06/03 18:34:29 INFO DAGScheduler: Missing parents: List(Stage 13481)
15/06/03 18:34:29 INFO DAGScheduler: Submitting Stage 13395 (ReplicatedVertexView.updateVertices - shippedVerts true false (broadcast) MapPartitionsRDD[546] at mapPartitions at VertexRDDImpl.scala:218), which has no missing parents
15/06/03 18:34:29 INFO MemoryStore: ensureFreeSpace(20408) called with curMem=376259, maxMem=559153152
15/06/03 18:34:29 INFO MemoryStore: Block broadcast_113 stored as values in memory (estimated size 19.9 KB, free 532.9 MB)
15/06/03 18:34:29 INFO MemoryStore: ensureFreeSpace(9481) called with curMem=396667, maxMem=559153152
15/06/03 18:34:29 INFO MemoryStore: Block broadcast_113_piece0 stored as bytes in memory (estimated size 9.3 KB, free 532.9 MB)
15/06/03 18:34:29 INFO BlockManagerInfo: Added broadcast_113_piece0 in memory on localhost:60836 (size: 9.3 KB, free: 533.1 MB)
15/06/03 18:34:29 INFO BlockManagerMaster: Updated info of block broadcast_113_piece0
15/06/03 18:34:29 INFO SparkContext: Created broadcast 113 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:29 INFO DAGScheduler: Submitting 1 missing tasks from Stage 13395 (ReplicatedVertexView.updateVertices - shippedVerts true false (broadcast) MapPartitionsRDD[546] at mapPartitions at VertexRDDImpl.scala:218)
15/06/03 18:34:29 INFO TaskSchedulerImpl: Adding task set 13395.0 with 1 tasks
15/06/03 18:34:29 INFO DAGScheduler: Submitting Stage 13480 (ReplicatedVertexView.withActiveSet - shippedActives (broadcast) MapPartitionsRDD[550] at mapPartitions at VertexRDDImpl.scala:222), which has no missing parents
15/06/03 18:34:29 INFO TaskSetManager: Starting task 0.0 in stage 13395.0 (TID 112, localhost, PROCESS_LOCAL, 4361 bytes)
15/06/03 18:34:29 INFO Executor: Running task 0.0 in stage 13395.0 (TID 112)
15/06/03 18:34:29 INFO MemoryStore: ensureFreeSpace(20064) called with curMem=406148, maxMem=559153152
15/06/03 18:34:29 INFO MemoryStore: Block broadcast_114 stored as values in memory (estimated size 19.6 KB, free 532.8 MB)
15/06/03 18:34:29 INFO BlockManager: Found block rdd_523_0 locally
15/06/03 18:34:29 INFO MemoryStore: ensureFreeSpace(9370) called with curMem=426212, maxMem=559153152
15/06/03 18:34:29 INFO MemoryStore: Block broadcast_114_piece0 stored as bytes in memory (estimated size 9.2 KB, free 532.8 MB)
15/06/03 18:34:29 INFO CacheManager: Partition rdd_542_0 not found, computing it
15/06/03 18:34:29 INFO BlockManager: Found block rdd_523_0 locally
15/06/03 18:34:29 INFO BlockManagerInfo: Added broadcast_114_piece0 in memory on localhost:60836 (size: 9.2 KB, free: 533.1 MB)
15/06/03 18:34:29 INFO BlockManagerMaster: Updated info of block broadcast_114_piece0
15/06/03 18:34:29 INFO CacheManager: Partition rdd_540_0 not found, computing it
15/06/03 18:34:29 INFO BlockManager: Found block rdd_523_0 locally
15/06/03 18:34:29 INFO BlockManager: Found block rdd_537_0 locally
15/06/03 18:34:29 INFO SparkContext: Created broadcast 114 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:29 INFO DAGScheduler: Submitting 1 missing tasks from Stage 13480 (ReplicatedVertexView.withActiveSet - shippedActives (broadcast) MapPartitionsRDD[550] at mapPartitions at VertexRDDImpl.scala:222)
15/06/03 18:34:29 INFO TaskSchedulerImpl: Adding task set 13480.0 with 1 tasks
15/06/03 18:34:29 INFO MemoryStore: ensureFreeSpace(1544) called with curMem=435582, maxMem=559153152
15/06/03 18:34:29 INFO MemoryStore: Block rdd_540_0 stored as values in memory (estimated size 1544.0 B, free 532.8 MB)
15/06/03 18:34:29 INFO BlockManagerInfo: Added rdd_540_0 in memory on localhost:60836 (size: 1544.0 B, free: 533.1 MB)
15/06/03 18:34:29 INFO BlockManagerMaster: Updated info of block rdd_540_0
15/06/03 18:34:29 INFO MemoryStore: ensureFreeSpace(1496) called with curMem=437126, maxMem=559153152
15/06/03 18:34:29 INFO MemoryStore: Block rdd_542_0 stored as values in memory (estimated size 1496.0 B, free 532.8 MB)
15/06/03 18:34:29 INFO BlockManagerInfo: Added rdd_542_0 in memory on localhost:60836 (size: 1496.0 B, free: 533.1 MB)
15/06/03 18:34:29 INFO BlockManagerMaster: Updated info of block rdd_542_0
15/06/03 18:34:29 INFO Executor: Finished task 0.0 in stage 13395.0 (TID 112). 2627 bytes result sent to driver
15/06/03 18:34:29 INFO TaskSetManager: Starting task 0.0 in stage 13480.0 (TID 113, localhost, PROCESS_LOCAL, 4297 bytes)
15/06/03 18:34:29 INFO Executor: Running task 0.0 in stage 13480.0 (TID 113)
15/06/03 18:34:29 INFO TaskSetManager: Finished task 0.0 in stage 13395.0 (TID 112) in 32 ms on localhost (1/1)
15/06/03 18:34:29 INFO TaskSchedulerImpl: Removed TaskSet 13395.0, whose tasks have all completed, from pool
15/06/03 18:34:29 INFO DAGScheduler: Stage 13395 (mapPartitions at VertexRDDImpl.scala:218) finished in 0.035 s
15/06/03 18:34:29 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:29 INFO DAGScheduler: running: Set(Stage 13480)
15/06/03 18:34:29 INFO DAGScheduler: waiting: Set(Stage 13482, Stage 13481)
15/06/03 18:34:29 INFO DAGScheduler: failed: Set()
15/06/03 18:34:29 INFO DAGScheduler: Missing parents for Stage 13482: List(Stage 13481)
15/06/03 18:34:29 INFO DAGScheduler: Missing parents for Stage 13481: List(Stage 13480)
15/06/03 18:34:29 INFO BlockManager: Found block rdd_540_0 locally
15/06/03 18:34:29 INFO Executor: Finished task 0.0 in stage 13480.0 (TID 113). 2003 bytes result sent to driver
15/06/03 18:34:29 INFO TaskSetManager: Finished task 0.0 in stage 13480.0 (TID 113) in 19 ms on localhost (1/1)
15/06/03 18:34:29 INFO TaskSchedulerImpl: Removed TaskSet 13480.0, whose tasks have all completed, from pool
15/06/03 18:34:29 INFO DAGScheduler: Stage 13480 (mapPartitions at VertexRDDImpl.scala:222) finished in 0.038 s
15/06/03 18:34:29 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:29 INFO DAGScheduler: running: Set()
15/06/03 18:34:29 INFO DAGScheduler: waiting: Set(Stage 13482, Stage 13481)
15/06/03 18:34:29 INFO DAGScheduler: failed: Set()
15/06/03 18:34:29 INFO DAGScheduler: Missing parents for Stage 13482: List(Stage 13481)
15/06/03 18:34:29 INFO DAGScheduler: Missing parents for Stage 13481: List()
15/06/03 18:34:29 INFO DAGScheduler: Submitting Stage 13481 (GraphImpl.aggregateMessages - preAgg MapPartitionsRDD[554] at mapPartitions at GraphImpl.scala:235), which is now runnable
15/06/03 18:34:29 INFO MemoryStore: ensureFreeSpace(13008) called with curMem=438622, maxMem=559153152
15/06/03 18:34:29 INFO MemoryStore: Block broadcast_115 stored as values in memory (estimated size 12.7 KB, free 532.8 MB)
15/06/03 18:34:29 INFO MemoryStore: ensureFreeSpace(6863) called with curMem=451630, maxMem=559153152
15/06/03 18:34:29 INFO MemoryStore: Block broadcast_115_piece0 stored as bytes in memory (estimated size 6.7 KB, free 532.8 MB)
15/06/03 18:34:29 INFO BlockManagerInfo: Added broadcast_115_piece0 in memory on localhost:60836 (size: 6.7 KB, free: 533.1 MB)
15/06/03 18:34:29 INFO BlockManagerMaster: Updated info of block broadcast_115_piece0
15/06/03 18:34:29 INFO SparkContext: Created broadcast 115 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:29 INFO DAGScheduler: Submitting 1 missing tasks from Stage 13481 (GraphImpl.aggregateMessages - preAgg MapPartitionsRDD[554] at mapPartitions at GraphImpl.scala:235)
15/06/03 18:34:29 INFO TaskSchedulerImpl: Adding task set 13481.0 with 1 tasks
15/06/03 18:34:29 INFO TaskSetManager: Starting task 0.0 in stage 13481.0 (TID 114, localhost, PROCESS_LOCAL, 3042 bytes)
15/06/03 18:34:29 INFO Executor: Running task 0.0 in stage 13481.0 (TID 114)
15/06/03 18:34:29 INFO CacheManager: Partition rdd_548_0 not found, computing it
15/06/03 18:34:29 INFO BlockManager: Found block rdd_529_0 locally
15/06/03 18:34:29 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:29 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:29 INFO MemoryStore: ensureFreeSpace(2976) called with curMem=458493, maxMem=559153152
15/06/03 18:34:29 INFO MemoryStore: Block rdd_548_0 stored as values in memory (estimated size 2.9 KB, free 532.8 MB)
15/06/03 18:34:29 INFO BlockManagerInfo: Added rdd_548_0 in memory on localhost:60836 (size: 2.9 KB, free: 533.1 MB)
15/06/03 18:34:29 INFO BlockManagerMaster: Updated info of block rdd_548_0
15/06/03 18:34:29 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:29 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:29 INFO Executor: Finished task 0.0 in stage 13481.0 (TID 114). 2788 bytes result sent to driver
15/06/03 18:34:29 INFO TaskSetManager: Finished task 0.0 in stage 13481.0 (TID 114) in 17 ms on localhost (1/1)
15/06/03 18:34:29 INFO TaskSchedulerImpl: Removed TaskSet 13481.0, whose tasks have all completed, from pool
15/06/03 18:34:29 INFO DAGScheduler: Stage 13481 (mapPartitions at GraphImpl.scala:235) finished in 0.018 s
15/06/03 18:34:29 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:29 INFO DAGScheduler: running: Set()
15/06/03 18:34:29 INFO DAGScheduler: waiting: Set(Stage 13482)
15/06/03 18:34:29 INFO DAGScheduler: failed: Set()
15/06/03 18:34:29 INFO DAGScheduler: Missing parents for Stage 13482: List()
15/06/03 18:34:29 INFO DAGScheduler: Submitting Stage 13482 (MapPartitionsRDD[558] at map at VertexRDDImpl.scala:90), which is now runnable
15/06/03 18:34:29 INFO MemoryStore: ensureFreeSpace(20688) called with curMem=461469, maxMem=559153152
15/06/03 18:34:29 INFO MemoryStore: Block broadcast_116 stored as values in memory (estimated size 20.2 KB, free 532.8 MB)
15/06/03 18:34:29 INFO MemoryStore: ensureFreeSpace(9779) called with curMem=482157, maxMem=559153152
15/06/03 18:34:29 INFO MemoryStore: Block broadcast_116_piece0 stored as bytes in memory (estimated size 9.5 KB, free 532.8 MB)
15/06/03 18:34:29 INFO BlockManagerInfo: Added broadcast_116_piece0 in memory on localhost:60836 (size: 9.5 KB, free: 533.1 MB)
15/06/03 18:34:29 INFO BlockManagerMaster: Updated info of block broadcast_116_piece0
15/06/03 18:34:29 INFO SparkContext: Created broadcast 116 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:29 INFO DAGScheduler: Submitting 1 missing tasks from Stage 13482 (MapPartitionsRDD[558] at map at VertexRDDImpl.scala:90)
15/06/03 18:34:29 INFO TaskSchedulerImpl: Adding task set 13482.0 with 1 tasks
15/06/03 18:34:29 INFO TaskSetManager: Starting task 0.0 in stage 13482.0 (TID 115, localhost, PROCESS_LOCAL, 4381 bytes)
15/06/03 18:34:29 INFO Executor: Running task 0.0 in stage 13482.0 (TID 115)
15/06/03 18:34:29 INFO CacheManager: Partition rdd_556_0 not found, computing it
15/06/03 18:34:29 INFO BlockManager: Found block rdd_542_0 locally
15/06/03 18:34:29 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:29 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:29 INFO MemoryStore: ensureFreeSpace(1664) called with curMem=491936, maxMem=559153152
15/06/03 18:34:29 INFO MemoryStore: Block rdd_556_0 stored as values in memory (estimated size 1664.0 B, free 532.8 MB)
15/06/03 18:34:29 INFO BlockManagerInfo: Added rdd_556_0 in memory on localhost:60836 (size: 1664.0 B, free: 533.1 MB)
15/06/03 18:34:29 INFO BlockManagerMaster: Updated info of block rdd_556_0
15/06/03 18:34:29 INFO Executor: Finished task 0.0 in stage 13482.0 (TID 115). 2693 bytes result sent to driver
15/06/03 18:34:29 INFO TaskSetManager: Finished task 0.0 in stage 13482.0 (TID 115) in 9 ms on localhost (1/1)
15/06/03 18:34:29 INFO TaskSchedulerImpl: Removed TaskSet 13482.0, whose tasks have all completed, from pool
15/06/03 18:34:29 INFO DAGScheduler: Stage 13482 (reduce at VertexRDDImpl.scala:90) finished in 0.011 s
15/06/03 18:34:29 INFO DAGScheduler: Job 28 finished: reduce at VertexRDDImpl.scala:90, took 0.313461 s
15/06/03 18:34:29 INFO Pregel: Pregel finished iteration 26
15/06/03 18:34:29 INFO ZippedPartitionsRDD2: Removing RDD 537 from persistence list
15/06/03 18:34:29 INFO BlockManager: Removing RDD 537
15/06/03 18:34:29 INFO ZippedPartitionsRDD2: Removing RDD 540 from persistence list
15/06/03 18:34:29 INFO BlockManager: Removing block rdd_537_0
15/06/03 18:34:29 INFO MemoryStore: Block rdd_537_0 of size 1664 dropped from memory (free 558661216)
15/06/03 18:34:29 INFO BlockManager: Removing RDD 540
15/06/03 18:34:29 INFO BlockManager: Removing block rdd_540_0
15/06/03 18:34:29 INFO MemoryStore: Block rdd_540_0 of size 1544 dropped from memory (free 558662760)
15/06/03 18:34:29 INFO ZippedPartitionsRDD2: Removing RDD 523 from persistence list
15/06/03 18:34:29 INFO BlockManager: Removing RDD 523
15/06/03 18:34:29 INFO BlockManager: Removing block rdd_523_0
15/06/03 18:34:29 INFO MemoryStore: Block rdd_523_0 of size 1496 dropped from memory (free 558664256)
15/06/03 18:34:29 INFO ZippedPartitionsRDD2: Removing RDD 529 from persistence list
15/06/03 18:34:29 INFO BlockManager: Removing RDD 529
15/06/03 18:34:29 INFO BlockManager: Removing block rdd_529_0
15/06/03 18:34:29 INFO MemoryStore: Block rdd_529_0 of size 2976 dropped from memory (free 558667232)
15/06/03 18:34:29 INFO SparkContext: Starting job: reduce at VertexRDDImpl.scala:90
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 2 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 3 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 5 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 4 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 1 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 0 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 8 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 7 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 6 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 11 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 10 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 9 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 14 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 13 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 12 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 17 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 16 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 15 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 20 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 19 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 18 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 23 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 22 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 21 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 26 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 25 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 24 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 29 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 28 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 27 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 32 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 31 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 30 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 35 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 34 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 33 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 38 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 37 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 36 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 41 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 40 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 39 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 44 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 43 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 42 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 47 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 46 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 45 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 50 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 49 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 48 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 53 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 52 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 51 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 56 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 55 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 54 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 59 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 58 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 57 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 62 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 61 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 60 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 65 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 64 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 63 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 68 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 67 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 66 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 71 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 70 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 69 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 74 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 73 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 72 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 77 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 76 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 75 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 80 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 79 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 78 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 83 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 82 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 81 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 86 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 85 is 145 bytes
15/06/03 18:34:29 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 84 is 145 bytes
15/06/03 18:34:29 INFO DAGScheduler: Registering RDD 565 (mapPartitions at VertexRDDImpl.scala:218)
15/06/03 18:34:29 INFO DAGScheduler: Registering RDD 569 (mapPartitions at VertexRDDImpl.scala:222)
15/06/03 18:34:29 INFO DAGScheduler: Registering RDD 573 (mapPartitions at GraphImpl.scala:235)
15/06/03 18:34:29 INFO DAGScheduler: Got job 29 (reduce at VertexRDDImpl.scala:90) with 1 output partitions (allowLocal=false)
15/06/03 18:34:29 INFO DAGScheduler: Final stage: Stage 14877(reduce at VertexRDDImpl.scala:90)
15/06/03 18:34:29 INFO DAGScheduler: Parents of final stage: List(Stage 14840, Stage 14832, Stage 14862, Stage 14834, Stage 14856, Stage 14828, Stage 14858, Stage 14850, Stage 14830, Stage 14822, Stage 14852, Stage 14874, Stage 14824, Stage 14854, Stage 14846, Stage 14876, Stage 14868, Stage 14826, Stage 14818, Stage 14848, Stage 14870, Stage 14820, Stage 14842, Stage 14872, Stage 14864, Stage 14844, Stage 14836, Stage 14866, Stage 14838, Stage 14817, Stage 14860)
15/06/03 18:34:29 INFO DAGScheduler: Missing parents: List(Stage 14876)
15/06/03 18:34:29 INFO DAGScheduler: Submitting Stage 14787 (ReplicatedVertexView.updateVertices - shippedVerts true false (broadcast) MapPartitionsRDD[565] at mapPartitions at VertexRDDImpl.scala:218), which has no missing parents
15/06/03 18:34:29 INFO MemoryStore: ensureFreeSpace(20928) called with curMem=485920, maxMem=559153152
15/06/03 18:34:29 INFO MemoryStore: Block broadcast_117 stored as values in memory (estimated size 20.4 KB, free 532.8 MB)
15/06/03 18:34:29 INFO MemoryStore: ensureFreeSpace(9777) called with curMem=506848, maxMem=559153152
15/06/03 18:34:29 INFO MemoryStore: Block broadcast_117_piece0 stored as bytes in memory (estimated size 9.5 KB, free 532.8 MB)
15/06/03 18:34:29 INFO BlockManagerInfo: Added broadcast_117_piece0 in memory on localhost:60836 (size: 9.5 KB, free: 533.1 MB)
15/06/03 18:34:29 INFO BlockManagerMaster: Updated info of block broadcast_117_piece0
15/06/03 18:34:29 INFO SparkContext: Created broadcast 117 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:29 INFO DAGScheduler: Submitting 1 missing tasks from Stage 14787 (ReplicatedVertexView.updateVertices - shippedVerts true false (broadcast) MapPartitionsRDD[565] at mapPartitions at VertexRDDImpl.scala:218)
15/06/03 18:34:29 INFO TaskSchedulerImpl: Adding task set 14787.0 with 1 tasks
15/06/03 18:34:29 INFO DAGScheduler: Submitting Stage 14875 (ReplicatedVertexView.withActiveSet - shippedActives (broadcast) MapPartitionsRDD[569] at mapPartitions at VertexRDDImpl.scala:222), which has no missing parents
15/06/03 18:34:29 INFO TaskSetManager: Starting task 0.0 in stage 14787.0 (TID 116, localhost, PROCESS_LOCAL, 4466 bytes)
15/06/03 18:34:29 INFO Executor: Running task 0.0 in stage 14787.0 (TID 116)
15/06/03 18:34:30 INFO MemoryStore: ensureFreeSpace(20592) called with curMem=516625, maxMem=559153152
15/06/03 18:34:30 INFO MemoryStore: Block broadcast_118 stored as values in memory (estimated size 20.1 KB, free 532.7 MB)
15/06/03 18:34:30 INFO MemoryStore: ensureFreeSpace(9631) called with curMem=537217, maxMem=559153152
15/06/03 18:34:30 INFO MemoryStore: Block broadcast_118_piece0 stored as bytes in memory (estimated size 9.4 KB, free 532.7 MB)
15/06/03 18:34:30 INFO BlockManagerInfo: Added broadcast_118_piece0 in memory on localhost:60836 (size: 9.4 KB, free: 533.1 MB)
15/06/03 18:34:30 INFO BlockManagerMaster: Updated info of block broadcast_118_piece0
15/06/03 18:34:30 INFO SparkContext: Created broadcast 118 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:30 INFO DAGScheduler: Submitting 1 missing tasks from Stage 14875 (ReplicatedVertexView.withActiveSet - shippedActives (broadcast) MapPartitionsRDD[569] at mapPartitions at VertexRDDImpl.scala:222)
15/06/03 18:34:30 INFO TaskSchedulerImpl: Adding task set 14875.0 with 1 tasks
15/06/03 18:34:30 INFO BlockManager: Found block rdd_542_0 locally
15/06/03 18:34:30 INFO CacheManager: Partition rdd_561_0 not found, computing it
15/06/03 18:34:30 INFO BlockManager: Found block rdd_542_0 locally
15/06/03 18:34:30 INFO CacheManager: Partition rdd_559_0 not found, computing it
15/06/03 18:34:30 INFO BlockManager: Found block rdd_542_0 locally
15/06/03 18:34:30 INFO BlockManager: Found block rdd_556_0 locally
15/06/03 18:34:30 INFO MemoryStore: ensureFreeSpace(1544) called with curMem=546848, maxMem=559153152
15/06/03 18:34:30 INFO MemoryStore: Block rdd_559_0 stored as values in memory (estimated size 1544.0 B, free 532.7 MB)
15/06/03 18:34:30 INFO BlockManagerInfo: Added rdd_559_0 in memory on localhost:60836 (size: 1544.0 B, free: 533.1 MB)
15/06/03 18:34:30 INFO BlockManagerMaster: Updated info of block rdd_559_0
15/06/03 18:34:30 INFO MemoryStore: ensureFreeSpace(1496) called with curMem=548392, maxMem=559153152
15/06/03 18:34:30 INFO MemoryStore: Block rdd_561_0 stored as values in memory (estimated size 1496.0 B, free 532.7 MB)
15/06/03 18:34:30 INFO BlockManagerInfo: Added rdd_561_0 in memory on localhost:60836 (size: 1496.0 B, free: 533.1 MB)
15/06/03 18:34:30 INFO BlockManagerMaster: Updated info of block rdd_561_0
15/06/03 18:34:30 INFO Executor: Finished task 0.0 in stage 14787.0 (TID 116). 2627 bytes result sent to driver
15/06/03 18:34:30 INFO TaskSetManager: Starting task 0.0 in stage 14875.0 (TID 117, localhost, PROCESS_LOCAL, 4402 bytes)
15/06/03 18:34:30 INFO Executor: Running task 0.0 in stage 14875.0 (TID 117)
15/06/03 18:34:30 INFO TaskSetManager: Finished task 0.0 in stage 14787.0 (TID 116) in 26 ms on localhost (1/1)
15/06/03 18:34:30 INFO TaskSchedulerImpl: Removed TaskSet 14787.0, whose tasks have all completed, from pool
15/06/03 18:34:30 INFO DAGScheduler: Stage 14787 (mapPartitions at VertexRDDImpl.scala:218) finished in 0.027 s
15/06/03 18:34:30 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:30 INFO DAGScheduler: running: Set(Stage 14875)
15/06/03 18:34:30 INFO DAGScheduler: waiting: Set(Stage 14876, Stage 14877)
15/06/03 18:34:30 INFO DAGScheduler: failed: Set()
15/06/03 18:34:30 INFO DAGScheduler: Missing parents for Stage 14876: List(Stage 14875)
15/06/03 18:34:30 INFO DAGScheduler: Missing parents for Stage 14877: List(Stage 14876)
15/06/03 18:34:30 INFO BlockManager: Found block rdd_559_0 locally
15/06/03 18:34:30 INFO Executor: Finished task 0.0 in stage 14875.0 (TID 117). 2003 bytes result sent to driver
15/06/03 18:34:30 INFO TaskSetManager: Finished task 0.0 in stage 14875.0 (TID 117) in 12 ms on localhost (1/1)
15/06/03 18:34:30 INFO TaskSchedulerImpl: Removed TaskSet 14875.0, whose tasks have all completed, from pool
15/06/03 18:34:30 INFO DAGScheduler: Stage 14875 (mapPartitions at VertexRDDImpl.scala:222) finished in 0.031 s
15/06/03 18:34:30 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:30 INFO DAGScheduler: running: Set()
15/06/03 18:34:30 INFO DAGScheduler: waiting: Set(Stage 14876, Stage 14877)
15/06/03 18:34:30 INFO DAGScheduler: failed: Set()
15/06/03 18:34:30 INFO DAGScheduler: Missing parents for Stage 14876: List()
15/06/03 18:34:30 INFO DAGScheduler: Missing parents for Stage 14877: List(Stage 14876)
15/06/03 18:34:30 INFO DAGScheduler: Submitting Stage 14876 (GraphImpl.aggregateMessages - preAgg MapPartitionsRDD[573] at mapPartitions at GraphImpl.scala:235), which is now runnable
15/06/03 18:34:30 INFO MemoryStore: ensureFreeSpace(13232) called with curMem=549888, maxMem=559153152
15/06/03 18:34:30 INFO MemoryStore: Block broadcast_119 stored as values in memory (estimated size 12.9 KB, free 532.7 MB)
15/06/03 18:34:30 INFO MemoryStore: ensureFreeSpace(6893) called with curMem=563120, maxMem=559153152
15/06/03 18:34:30 INFO MemoryStore: Block broadcast_119_piece0 stored as bytes in memory (estimated size 6.7 KB, free 532.7 MB)
15/06/03 18:34:30 INFO BlockManagerInfo: Added broadcast_119_piece0 in memory on localhost:60836 (size: 6.7 KB, free: 533.1 MB)
15/06/03 18:34:30 INFO BlockManagerMaster: Updated info of block broadcast_119_piece0
15/06/03 18:34:30 INFO SparkContext: Created broadcast 119 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:30 INFO DAGScheduler: Submitting 1 missing tasks from Stage 14876 (GraphImpl.aggregateMessages - preAgg MapPartitionsRDD[573] at mapPartitions at GraphImpl.scala:235)
15/06/03 18:34:30 INFO TaskSchedulerImpl: Adding task set 14876.0 with 1 tasks
15/06/03 18:34:30 INFO TaskSetManager: Starting task 0.0 in stage 14876.0 (TID 118, localhost, PROCESS_LOCAL, 3083 bytes)
15/06/03 18:34:30 INFO Executor: Running task 0.0 in stage 14876.0 (TID 118)
15/06/03 18:34:30 INFO CacheManager: Partition rdd_567_0 not found, computing it
15/06/03 18:34:30 INFO BlockManager: Found block rdd_548_0 locally
15/06/03 18:34:30 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:30 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
15/06/03 18:34:30 INFO MemoryStore: ensureFreeSpace(2976) called with curMem=570013, maxMem=559153152
15/06/03 18:34:30 INFO MemoryStore: Block rdd_567_0 stored as values in memory (estimated size 2.9 KB, free 532.7 MB)
15/06/03 18:34:30 INFO BlockManagerInfo: Added rdd_567_0 in memory on localhost:60836 (size: 2.9 KB, free: 533.1 MB)
15/06/03 18:34:30 INFO BlockManagerMaster: Updated info of block rdd_567_0
15/06/03 18:34:30 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:30 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:30 INFO Executor: Finished task 0.0 in stage 14876.0 (TID 118). 2788 bytes result sent to driver
15/06/03 18:34:30 INFO TaskSetManager: Finished task 0.0 in stage 14876.0 (TID 118) in 14 ms on localhost (1/1)
15/06/03 18:34:30 INFO TaskSchedulerImpl: Removed TaskSet 14876.0, whose tasks have all completed, from pool
15/06/03 18:34:30 INFO DAGScheduler: Stage 14876 (mapPartitions at GraphImpl.scala:235) finished in 0.014 s
15/06/03 18:34:30 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:30 INFO DAGScheduler: running: Set()
15/06/03 18:34:30 INFO DAGScheduler: waiting: Set(Stage 14877)
15/06/03 18:34:30 INFO DAGScheduler: failed: Set()
15/06/03 18:34:30 INFO DAGScheduler: Missing parents for Stage 14877: List()
15/06/03 18:34:30 INFO DAGScheduler: Submitting Stage 14877 (MapPartitionsRDD[577] at map at VertexRDDImpl.scala:90), which is now runnable
15/06/03 18:34:30 INFO MemoryStore: ensureFreeSpace(21216) called with curMem=572989, maxMem=559153152
15/06/03 18:34:30 INFO MemoryStore: Block broadcast_120 stored as values in memory (estimated size 20.7 KB, free 532.7 MB)
15/06/03 18:34:30 INFO MemoryStore: ensureFreeSpace(9845) called with curMem=594205, maxMem=559153152
15/06/03 18:34:30 INFO MemoryStore: Block broadcast_120_piece0 stored as bytes in memory (estimated size 9.6 KB, free 532.7 MB)
15/06/03 18:34:30 INFO BlockManagerInfo: Added broadcast_120_piece0 in memory on localhost:60836 (size: 9.6 KB, free: 533.1 MB)
15/06/03 18:34:30 INFO BlockManagerMaster: Updated info of block broadcast_120_piece0
15/06/03 18:34:30 INFO SparkContext: Created broadcast 120 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:30 INFO DAGScheduler: Submitting 1 missing tasks from Stage 14877 (MapPartitionsRDD[577] at map at VertexRDDImpl.scala:90)
15/06/03 18:34:30 INFO TaskSchedulerImpl: Adding task set 14877.0 with 1 tasks
15/06/03 18:34:30 INFO TaskSetManager: Starting task 0.0 in stage 14877.0 (TID 119, localhost, PROCESS_LOCAL, 4486 bytes)
15/06/03 18:34:30 INFO Executor: Running task 0.0 in stage 14877.0 (TID 119)
15/06/03 18:34:30 INFO CacheManager: Partition rdd_575_0 not found, computing it
15/06/03 18:34:30 INFO BlockManager: Found block rdd_561_0 locally
15/06/03 18:34:30 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:30 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:30 INFO MemoryStore: ensureFreeSpace(1664) called with curMem=604050, maxMem=559153152
15/06/03 18:34:30 INFO MemoryStore: Block rdd_575_0 stored as values in memory (estimated size 1664.0 B, free 532.7 MB)
15/06/03 18:34:30 INFO BlockManagerInfo: Added rdd_575_0 in memory on localhost:60836 (size: 1664.0 B, free: 533.1 MB)
15/06/03 18:34:30 INFO BlockManagerMaster: Updated info of block rdd_575_0
15/06/03 18:34:30 INFO Executor: Finished task 0.0 in stage 14877.0 (TID 119). 2693 bytes result sent to driver
15/06/03 18:34:30 INFO TaskSetManager: Finished task 0.0 in stage 14877.0 (TID 119) in 9 ms on localhost (1/1)
15/06/03 18:34:30 INFO TaskSchedulerImpl: Removed TaskSet 14877.0, whose tasks have all completed, from pool
15/06/03 18:34:30 INFO DAGScheduler: Stage 14877 (reduce at VertexRDDImpl.scala:90) finished in 0.010 s
15/06/03 18:34:30 INFO DAGScheduler: Job 29 finished: reduce at VertexRDDImpl.scala:90, took 0.224721 s
15/06/03 18:34:30 INFO Pregel: Pregel finished iteration 27
15/06/03 18:34:30 INFO ZippedPartitionsRDD2: Removing RDD 556 from persistence list
15/06/03 18:34:30 INFO BlockManager: Removing RDD 556
15/06/03 18:34:30 INFO ZippedPartitionsRDD2: Removing RDD 559 from persistence list
15/06/03 18:34:30 INFO BlockManager: Removing block rdd_556_0
15/06/03 18:34:30 INFO MemoryStore: Block rdd_556_0 of size 1664 dropped from memory (free 558549102)
15/06/03 18:34:30 INFO BlockManager: Removing RDD 559
15/06/03 18:34:30 INFO BlockManager: Removing block rdd_559_0
15/06/03 18:34:30 INFO MemoryStore: Block rdd_559_0 of size 1544 dropped from memory (free 558550646)
15/06/03 18:34:30 INFO ZippedPartitionsRDD2: Removing RDD 542 from persistence list
15/06/03 18:34:30 INFO BlockManager: Removing RDD 542
15/06/03 18:34:30 INFO ZippedPartitionsRDD2: Removing RDD 548 from persistence list
15/06/03 18:34:30 INFO BlockManager: Removing block rdd_542_0
15/06/03 18:34:30 INFO MemoryStore: Block rdd_542_0 of size 1496 dropped from memory (free 558552142)
15/06/03 18:34:30 INFO BlockManager: Removing RDD 548
15/06/03 18:34:30 INFO BlockManager: Removing block rdd_548_0
15/06/03 18:34:30 INFO MemoryStore: Block rdd_548_0 of size 2976 dropped from memory (free 558555118)
15/06/03 18:34:30 INFO SparkContext: Starting job: reduce at VertexRDDImpl.scala:90
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 2 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 3 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 5 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 4 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 1 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 0 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 8 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 7 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 6 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 11 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 10 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 9 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 14 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 13 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 12 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 17 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 16 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 15 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 20 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 19 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 18 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 23 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 22 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 21 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 26 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 25 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 24 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 29 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 28 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 27 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 32 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 31 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 30 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 35 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 34 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 33 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 38 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 37 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 36 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 41 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 40 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 39 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 44 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 43 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 42 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 47 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 46 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 45 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 50 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 49 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 48 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 53 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 52 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 51 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 56 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 55 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 54 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 59 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 58 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 57 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 62 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 61 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 60 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 65 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 64 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 63 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 68 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 67 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 66 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 71 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 70 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 69 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 74 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 73 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 72 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 77 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 76 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 75 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 80 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 79 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 78 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 83 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 82 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 81 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 86 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 85 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 84 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 89 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 88 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 87 is 145 bytes
15/06/03 18:34:30 INFO DAGScheduler: Registering RDD 584 (mapPartitions at VertexRDDImpl.scala:218)
15/06/03 18:34:30 INFO DAGScheduler: Registering RDD 588 (mapPartitions at VertexRDDImpl.scala:222)
15/06/03 18:34:30 INFO DAGScheduler: Registering RDD 592 (mapPartitions at GraphImpl.scala:235)
15/06/03 18:34:30 INFO DAGScheduler: Got job 30 (reduce at VertexRDDImpl.scala:90) with 1 output partitions (allowLocal=false)
15/06/03 18:34:30 INFO DAGScheduler: Final stage: Stage 16365(reduce at VertexRDDImpl.scala:90)
15/06/03 18:34:30 INFO DAGScheduler: Parents of final stage: List(Stage 16356, Stage 16306, Stage 16328, Stage 16358, Stage 16350, Stage 16308, Stage 16330, Stage 16322, Stage 16352, Stage 16324, Stage 16303, Stage 16354, Stage 16346, Stage 16326, Stage 16318, Stage 16348, Stage 16320, Stage 16342, Stage 16364, Stage 16314, Stage 16344, Stage 16336, Stage 16316, Stage 16338, Stage 16360, Stage 16310, Stage 16340, Stage 16332, Stage 16362, Stage 16312, Stage 16304, Stage 16334)
15/06/03 18:34:30 INFO DAGScheduler: Missing parents: List(Stage 16364)
15/06/03 18:34:30 INFO DAGScheduler: Submitting Stage 16272 (ReplicatedVertexView.updateVertices - shippedVerts true false (broadcast) MapPartitionsRDD[584] at mapPartitions at VertexRDDImpl.scala:218), which has no missing parents
15/06/03 18:34:30 INFO MemoryStore: ensureFreeSpace(21456) called with curMem=598034, maxMem=559153152
15/06/03 18:34:30 INFO MemoryStore: Block broadcast_121 stored as values in memory (estimated size 21.0 KB, free 532.7 MB)
15/06/03 18:34:30 INFO MemoryStore: ensureFreeSpace(9927) called with curMem=619490, maxMem=559153152
15/06/03 18:34:30 INFO MemoryStore: Block broadcast_121_piece0 stored as bytes in memory (estimated size 9.7 KB, free 532.6 MB)
15/06/03 18:34:30 INFO BlockManagerInfo: Added broadcast_121_piece0 in memory on localhost:60836 (size: 9.7 KB, free: 533.1 MB)
15/06/03 18:34:30 INFO BlockManagerMaster: Updated info of block broadcast_121_piece0
15/06/03 18:34:30 INFO SparkContext: Created broadcast 121 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:30 INFO DAGScheduler: Submitting 1 missing tasks from Stage 16272 (ReplicatedVertexView.updateVertices - shippedVerts true false (broadcast) MapPartitionsRDD[584] at mapPartitions at VertexRDDImpl.scala:218)
15/06/03 18:34:30 INFO TaskSchedulerImpl: Adding task set 16272.0 with 1 tasks
15/06/03 18:34:30 INFO DAGScheduler: Submitting Stage 16363 (ReplicatedVertexView.withActiveSet - shippedActives (broadcast) MapPartitionsRDD[588] at mapPartitions at VertexRDDImpl.scala:222), which has no missing parents
15/06/03 18:34:30 INFO TaskSetManager: Starting task 0.0 in stage 16272.0 (TID 120, localhost, PROCESS_LOCAL, 4571 bytes)
15/06/03 18:34:30 INFO Executor: Running task 0.0 in stage 16272.0 (TID 120)
15/06/03 18:34:30 INFO MemoryStore: ensureFreeSpace(21112) called with curMem=629417, maxMem=559153152
15/06/03 18:34:30 INFO MemoryStore: Block broadcast_122 stored as values in memory (estimated size 20.6 KB, free 532.6 MB)
15/06/03 18:34:30 INFO BlockManager: Found block rdd_561_0 locally
15/06/03 18:34:30 INFO MemoryStore: ensureFreeSpace(9815) called with curMem=650529, maxMem=559153152
15/06/03 18:34:30 INFO MemoryStore: Block broadcast_122_piece0 stored as bytes in memory (estimated size 9.6 KB, free 532.6 MB)
15/06/03 18:34:30 INFO CacheManager: Partition rdd_580_0 not found, computing it
15/06/03 18:34:30 INFO BlockManager: Found block rdd_561_0 locally
15/06/03 18:34:30 INFO BlockManagerInfo: Added broadcast_122_piece0 in memory on localhost:60836 (size: 9.6 KB, free: 533.1 MB)
15/06/03 18:34:30 INFO BlockManagerMaster: Updated info of block broadcast_122_piece0
15/06/03 18:34:30 INFO CacheManager: Partition rdd_578_0 not found, computing it
15/06/03 18:34:30 INFO BlockManager: Found block rdd_561_0 locally
15/06/03 18:34:30 INFO BlockManager: Found block rdd_575_0 locally
15/06/03 18:34:30 INFO SparkContext: Created broadcast 122 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:30 INFO DAGScheduler: Submitting 1 missing tasks from Stage 16363 (ReplicatedVertexView.withActiveSet - shippedActives (broadcast) MapPartitionsRDD[588] at mapPartitions at VertexRDDImpl.scala:222)
15/06/03 18:34:30 INFO TaskSchedulerImpl: Adding task set 16363.0 with 1 tasks
15/06/03 18:34:30 INFO MemoryStore: ensureFreeSpace(1544) called with curMem=660344, maxMem=559153152
15/06/03 18:34:30 INFO MemoryStore: Block rdd_578_0 stored as values in memory (estimated size 1544.0 B, free 532.6 MB)
15/06/03 18:34:30 INFO BlockManagerInfo: Added rdd_578_0 in memory on localhost:60836 (size: 1544.0 B, free: 533.1 MB)
15/06/03 18:34:30 INFO BlockManagerMaster: Updated info of block rdd_578_0
15/06/03 18:34:30 INFO MemoryStore: ensureFreeSpace(1496) called with curMem=661888, maxMem=559153152
15/06/03 18:34:30 INFO MemoryStore: Block rdd_580_0 stored as values in memory (estimated size 1496.0 B, free 532.6 MB)
15/06/03 18:34:30 INFO BlockManagerInfo: Added rdd_580_0 in memory on localhost:60836 (size: 1496.0 B, free: 533.1 MB)
15/06/03 18:34:30 INFO BlockManagerMaster: Updated info of block rdd_580_0
15/06/03 18:34:30 INFO Executor: Finished task 0.0 in stage 16272.0 (TID 120). 2627 bytes result sent to driver
15/06/03 18:34:30 INFO TaskSetManager: Starting task 0.0 in stage 16363.0 (TID 121, localhost, PROCESS_LOCAL, 4507 bytes)
15/06/03 18:34:30 INFO TaskSetManager: Finished task 0.0 in stage 16272.0 (TID 120) in 30 ms on localhost (1/1)
15/06/03 18:34:30 INFO Executor: Running task 0.0 in stage 16363.0 (TID 121)
15/06/03 18:34:30 INFO TaskSchedulerImpl: Removed TaskSet 16272.0, whose tasks have all completed, from pool
15/06/03 18:34:30 INFO DAGScheduler: Stage 16272 (mapPartitions at VertexRDDImpl.scala:218) finished in 0.031 s
15/06/03 18:34:30 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:30 INFO DAGScheduler: running: Set(Stage 16363)
15/06/03 18:34:30 INFO DAGScheduler: waiting: Set(Stage 16364, Stage 16365)
15/06/03 18:34:30 INFO DAGScheduler: failed: Set()
15/06/03 18:34:30 INFO DAGScheduler: Missing parents for Stage 16364: List(Stage 16363)
15/06/03 18:34:30 INFO DAGScheduler: Missing parents for Stage 16365: List(Stage 16364)
15/06/03 18:34:30 INFO BlockManager: Found block rdd_578_0 locally
15/06/03 18:34:30 INFO Executor: Finished task 0.0 in stage 16363.0 (TID 121). 2003 bytes result sent to driver
15/06/03 18:34:30 INFO TaskSetManager: Finished task 0.0 in stage 16363.0 (TID 121) in 16 ms on localhost (1/1)
15/06/03 18:34:30 INFO TaskSchedulerImpl: Removed TaskSet 16363.0, whose tasks have all completed, from pool
15/06/03 18:34:30 INFO DAGScheduler: Stage 16363 (mapPartitions at VertexRDDImpl.scala:222) finished in 0.036 s
15/06/03 18:34:30 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:30 INFO DAGScheduler: running: Set()
15/06/03 18:34:30 INFO DAGScheduler: waiting: Set(Stage 16364, Stage 16365)
15/06/03 18:34:30 INFO DAGScheduler: failed: Set()
15/06/03 18:34:30 INFO DAGScheduler: Missing parents for Stage 16364: List()
15/06/03 18:34:30 INFO DAGScheduler: Missing parents for Stage 16365: List(Stage 16364)
15/06/03 18:34:30 INFO DAGScheduler: Submitting Stage 16364 (GraphImpl.aggregateMessages - preAgg MapPartitionsRDD[592] at mapPartitions at GraphImpl.scala:235), which is now runnable
15/06/03 18:34:30 INFO MemoryStore: ensureFreeSpace(13464) called with curMem=663384, maxMem=559153152
15/06/03 18:34:30 INFO MemoryStore: Block broadcast_123 stored as values in memory (estimated size 13.1 KB, free 532.6 MB)
15/06/03 18:34:30 INFO MemoryStore: ensureFreeSpace(6932) called with curMem=676848, maxMem=559153152
15/06/03 18:34:30 INFO MemoryStore: Block broadcast_123_piece0 stored as bytes in memory (estimated size 6.8 KB, free 532.6 MB)
15/06/03 18:34:30 INFO BlockManagerInfo: Added broadcast_123_piece0 in memory on localhost:60836 (size: 6.8 KB, free: 533.1 MB)
15/06/03 18:34:30 INFO BlockManagerMaster: Updated info of block broadcast_123_piece0
15/06/03 18:34:30 INFO SparkContext: Created broadcast 123 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:30 INFO DAGScheduler: Submitting 1 missing tasks from Stage 16364 (GraphImpl.aggregateMessages - preAgg MapPartitionsRDD[592] at mapPartitions at GraphImpl.scala:235)
15/06/03 18:34:30 INFO TaskSchedulerImpl: Adding task set 16364.0 with 1 tasks
15/06/03 18:34:30 INFO TaskSetManager: Starting task 0.0 in stage 16364.0 (TID 122, localhost, PROCESS_LOCAL, 3124 bytes)
15/06/03 18:34:30 INFO Executor: Running task 0.0 in stage 16364.0 (TID 122)
15/06/03 18:34:30 INFO CacheManager: Partition rdd_586_0 not found, computing it
15/06/03 18:34:30 INFO BlockManager: Found block rdd_567_0 locally
15/06/03 18:34:30 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:30 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
15/06/03 18:34:30 INFO MemoryStore: ensureFreeSpace(2976) called with curMem=683780, maxMem=559153152
15/06/03 18:34:30 INFO MemoryStore: Block rdd_586_0 stored as values in memory (estimated size 2.9 KB, free 532.6 MB)
15/06/03 18:34:30 INFO BlockManagerInfo: Added rdd_586_0 in memory on localhost:60836 (size: 2.9 KB, free: 533.1 MB)
15/06/03 18:34:30 INFO BlockManagerMaster: Updated info of block rdd_586_0
15/06/03 18:34:30 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:30 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
15/06/03 18:34:30 INFO Executor: Finished task 0.0 in stage 16364.0 (TID 122). 2788 bytes result sent to driver
15/06/03 18:34:30 INFO TaskSetManager: Finished task 0.0 in stage 16364.0 (TID 122) in 14 ms on localhost (1/1)
15/06/03 18:34:30 INFO TaskSchedulerImpl: Removed TaskSet 16364.0, whose tasks have all completed, from pool
15/06/03 18:34:30 INFO DAGScheduler: Stage 16364 (mapPartitions at GraphImpl.scala:235) finished in 0.015 s
15/06/03 18:34:30 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:30 INFO DAGScheduler: running: Set()
15/06/03 18:34:30 INFO DAGScheduler: waiting: Set(Stage 16365)
15/06/03 18:34:30 INFO DAGScheduler: failed: Set()
15/06/03 18:34:30 INFO DAGScheduler: Missing parents for Stage 16365: List()
15/06/03 18:34:30 INFO DAGScheduler: Submitting Stage 16365 (MapPartitionsRDD[596] at map at VertexRDDImpl.scala:90), which is now runnable
15/06/03 18:34:30 INFO MemoryStore: ensureFreeSpace(21744) called with curMem=686756, maxMem=559153152
15/06/03 18:34:30 INFO MemoryStore: Block broadcast_124 stored as values in memory (estimated size 21.2 KB, free 532.6 MB)
15/06/03 18:34:30 INFO MemoryStore: ensureFreeSpace(10114) called with curMem=708500, maxMem=559153152
15/06/03 18:34:30 INFO MemoryStore: Block broadcast_124_piece0 stored as bytes in memory (estimated size 9.9 KB, free 532.6 MB)
15/06/03 18:34:30 INFO BlockManager: Removing broadcast 122
15/06/03 18:34:30 INFO BlockManager: Removing block broadcast_122
15/06/03 18:34:30 INFO MemoryStore: Block broadcast_122 of size 21112 dropped from memory (free 558455650)
15/06/03 18:34:30 INFO BlockManager: Removing block broadcast_122_piece0
15/06/03 18:34:30 INFO MemoryStore: Block broadcast_122_piece0 of size 9815 dropped from memory (free 558465465)
15/06/03 18:34:30 INFO BlockManagerInfo: Added broadcast_124_piece0 in memory on localhost:60836 (size: 9.9 KB, free: 533.0 MB)
15/06/03 18:34:30 INFO BlockManagerMaster: Updated info of block broadcast_124_piece0
15/06/03 18:34:30 INFO BlockManagerInfo: Removed broadcast_122_piece0 on localhost:60836 in memory (size: 9.6 KB, free: 533.1 MB)
15/06/03 18:34:30 INFO BlockManagerMaster: Updated info of block broadcast_122_piece0
15/06/03 18:34:30 INFO ContextCleaner: Cleaned broadcast 122
15/06/03 18:34:30 INFO SparkContext: Created broadcast 124 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:30 INFO DAGScheduler: Submitting 1 missing tasks from Stage 16365 (MapPartitionsRDD[596] at map at VertexRDDImpl.scala:90)
15/06/03 18:34:30 INFO TaskSchedulerImpl: Adding task set 16365.0 with 1 tasks
15/06/03 18:34:30 INFO BlockManager: Removing broadcast 121
15/06/03 18:34:30 INFO BlockManager: Removing block broadcast_121
15/06/03 18:34:30 INFO MemoryStore: Block broadcast_121 of size 21456 dropped from memory (free 558486921)
15/06/03 18:34:30 INFO BlockManager: Removing block broadcast_121_piece0
15/06/03 18:34:30 INFO MemoryStore: Block broadcast_121_piece0 of size 9927 dropped from memory (free 558496848)
15/06/03 18:34:30 INFO BlockManagerInfo: Removed broadcast_121_piece0 on localhost:60836 in memory (size: 9.7 KB, free: 533.1 MB)
15/06/03 18:34:30 INFO BlockManagerMaster: Updated info of block broadcast_121_piece0
15/06/03 18:34:30 INFO ContextCleaner: Cleaned broadcast 121
15/06/03 18:34:30 INFO TaskSetManager: Starting task 0.0 in stage 16365.0 (TID 123, localhost, PROCESS_LOCAL, 4591 bytes)
15/06/03 18:34:30 INFO BlockManager: Removing broadcast 120
15/06/03 18:34:30 INFO BlockManager: Removing block broadcast_120
15/06/03 18:34:30 INFO Executor: Running task 0.0 in stage 16365.0 (TID 123)
15/06/03 18:34:30 INFO MemoryStore: Block broadcast_120 of size 21216 dropped from memory (free 558518064)
15/06/03 18:34:30 INFO BlockManager: Removing block broadcast_120_piece0
15/06/03 18:34:30 INFO MemoryStore: Block broadcast_120_piece0 of size 9845 dropped from memory (free 558527909)
15/06/03 18:34:30 INFO BlockManagerInfo: Removed broadcast_120_piece0 on localhost:60836 in memory (size: 9.6 KB, free: 533.1 MB)
15/06/03 18:34:30 INFO BlockManagerMaster: Updated info of block broadcast_120_piece0
15/06/03 18:34:30 INFO ContextCleaner: Cleaned broadcast 120
15/06/03 18:34:30 INFO BlockManager: Removing broadcast 119
15/06/03 18:34:30 INFO BlockManager: Removing block broadcast_119_piece0
15/06/03 18:34:30 INFO MemoryStore: Block broadcast_119_piece0 of size 6893 dropped from memory (free 558534802)
15/06/03 18:34:30 INFO BlockManagerInfo: Removed broadcast_119_piece0 on localhost:60836 in memory (size: 6.7 KB, free: 533.1 MB)
15/06/03 18:34:30 INFO BlockManagerMaster: Updated info of block broadcast_119_piece0
15/06/03 18:34:30 INFO BlockManager: Removing block broadcast_119
15/06/03 18:34:30 INFO MemoryStore: Block broadcast_119 of size 13232 dropped from memory (free 558548034)
15/06/03 18:34:30 INFO ContextCleaner: Cleaned broadcast 119
15/06/03 18:34:30 INFO BlockManager: Removing broadcast 112
15/06/03 18:34:30 INFO BlockManager: Removing block broadcast_112
15/06/03 18:34:30 INFO MemoryStore: Block broadcast_112 of size 20168 dropped from memory (free 558568202)
15/06/03 18:34:30 INFO BlockManager: Removing block broadcast_112_piece0
15/06/03 18:34:30 INFO MemoryStore: Block broadcast_112_piece0 of size 9404 dropped from memory (free 558577606)
15/06/03 18:34:30 INFO BlockManagerInfo: Removed broadcast_112_piece0 on localhost:60836 in memory (size: 9.2 KB, free: 533.1 MB)
15/06/03 18:34:30 INFO BlockManagerMaster: Updated info of block broadcast_112_piece0
15/06/03 18:34:30 INFO ContextCleaner: Cleaned broadcast 112
15/06/03 18:34:30 INFO BlockManager: Removing broadcast 111
15/06/03 18:34:30 INFO BlockManager: Removing block broadcast_111
15/06/03 18:34:30 INFO MemoryStore: Block broadcast_111 of size 12776 dropped from memory (free 558590382)
15/06/03 18:34:30 INFO BlockManager: Removing block broadcast_111_piece0
15/06/03 18:34:30 INFO MemoryStore: Block broadcast_111_piece0 of size 6791 dropped from memory (free 558597173)
15/06/03 18:34:30 INFO BlockManagerInfo: Removed broadcast_111_piece0 on localhost:60836 in memory (size: 6.6 KB, free: 533.1 MB)
15/06/03 18:34:30 INFO BlockManagerMaster: Updated info of block broadcast_111_piece0
15/06/03 18:34:30 INFO ContextCleaner: Cleaned broadcast 111
15/06/03 18:34:30 INFO BlockManager: Removing broadcast 110
15/06/03 18:34:30 INFO BlockManager: Removing block broadcast_110_piece0
15/06/03 18:34:30 INFO MemoryStore: Block broadcast_110_piece0 of size 9166 dropped from memory (free 558606339)
15/06/03 18:34:30 INFO BlockManagerInfo: Removed broadcast_110_piece0 on localhost:60836 in memory (size: 9.0 KB, free: 533.1 MB)
15/06/03 18:34:30 INFO BlockManagerMaster: Updated info of block broadcast_110_piece0
15/06/03 18:34:30 INFO BlockManager: Removing block broadcast_110
15/06/03 18:34:30 INFO MemoryStore: Block broadcast_110 of size 19536 dropped from memory (free 558625875)
15/06/03 18:34:30 INFO ContextCleaner: Cleaned broadcast 110
15/06/03 18:34:30 INFO BlockManager: Removing broadcast 109
15/06/03 18:34:30 INFO BlockManager: Removing block broadcast_109
15/06/03 18:34:30 INFO MemoryStore: Block broadcast_109 of size 19880 dropped from memory (free 558645755)
15/06/03 18:34:30 INFO BlockManager: Removing block broadcast_109_piece0
15/06/03 18:34:30 INFO MemoryStore: Block broadcast_109_piece0 of size 9320 dropped from memory (free 558655075)
15/06/03 18:34:30 INFO BlockManagerInfo: Removed broadcast_109_piece0 on localhost:60836 in memory (size: 9.1 KB, free: 533.1 MB)
15/06/03 18:34:30 INFO BlockManagerMaster: Updated info of block broadcast_109_piece0
15/06/03 18:34:30 INFO CacheManager: Partition rdd_594_0 not found, computing it
15/06/03 18:34:30 INFO BlockManager: Found block rdd_580_0 locally
15/06/03 18:34:30 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:30 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:30 INFO ContextCleaner: Cleaned broadcast 109
15/06/03 18:34:30 INFO BlockManager: Removing broadcast 116
15/06/03 18:34:30 INFO BlockManager: Removing block broadcast_116_piece0
15/06/03 18:34:30 INFO MemoryStore: Block broadcast_116_piece0 of size 9779 dropped from memory (free 558664854)
15/06/03 18:34:30 INFO BlockManagerInfo: Removed broadcast_116_piece0 on localhost:60836 in memory (size: 9.5 KB, free: 533.1 MB)
15/06/03 18:34:30 INFO BlockManagerMaster: Updated info of block broadcast_116_piece0
15/06/03 18:34:30 INFO BlockManager: Removing block broadcast_116
15/06/03 18:34:30 INFO MemoryStore: Block broadcast_116 of size 20688 dropped from memory (free 558685542)
15/06/03 18:34:30 INFO ContextCleaner: Cleaned broadcast 116
15/06/03 18:34:30 INFO BlockManager: Removing broadcast 115
15/06/03 18:34:30 INFO BlockManager: Removing block broadcast_115
15/06/03 18:34:30 INFO MemoryStore: Block broadcast_115 of size 13008 dropped from memory (free 558698550)
15/06/03 18:34:30 INFO BlockManager: Removing block broadcast_115_piece0
15/06/03 18:34:30 INFO MemoryStore: Block broadcast_115_piece0 of size 6863 dropped from memory (free 558705413)
15/06/03 18:34:30 INFO BlockManagerInfo: Removed broadcast_115_piece0 on localhost:60836 in memory (size: 6.7 KB, free: 533.1 MB)
15/06/03 18:34:30 INFO MemoryStore: ensureFreeSpace(1664) called with curMem=447739, maxMem=559153152
15/06/03 18:34:30 INFO MemoryStore: Block rdd_594_0 stored as values in memory (estimated size 1664.0 B, free 532.8 MB)
15/06/03 18:34:30 INFO BlockManagerMaster: Updated info of block broadcast_115_piece0
15/06/03 18:34:30 INFO BlockManagerInfo: Added rdd_594_0 in memory on localhost:60836 (size: 1664.0 B, free: 533.1 MB)
15/06/03 18:34:30 INFO ContextCleaner: Cleaned broadcast 115
15/06/03 18:34:30 INFO BlockManagerMaster: Updated info of block rdd_594_0
15/06/03 18:34:30 INFO Executor: Finished task 0.0 in stage 16365.0 (TID 123). 2693 bytes result sent to driver
15/06/03 18:34:30 INFO BlockManager: Removing broadcast 114
15/06/03 18:34:30 INFO BlockManager: Removing block broadcast_114
15/06/03 18:34:30 INFO MemoryStore: Block broadcast_114 of size 20064 dropped from memory (free 558723813)
15/06/03 18:34:30 INFO BlockManager: Removing block broadcast_114_piece0
15/06/03 18:34:30 INFO MemoryStore: Block broadcast_114_piece0 of size 9370 dropped from memory (free 558733183)
15/06/03 18:34:30 INFO TaskSetManager: Finished task 0.0 in stage 16365.0 (TID 123) in 20 ms on localhost (1/1)
15/06/03 18:34:30 INFO TaskSchedulerImpl: Removed TaskSet 16365.0, whose tasks have all completed, from pool
15/06/03 18:34:30 INFO BlockManagerInfo: Removed broadcast_114_piece0 on localhost:60836 in memory (size: 9.2 KB, free: 533.1 MB)
15/06/03 18:34:30 INFO BlockManagerMaster: Updated info of block broadcast_114_piece0
15/06/03 18:34:30 INFO ContextCleaner: Cleaned broadcast 114
15/06/03 18:34:30 INFO DAGScheduler: Stage 16365 (reduce at VertexRDDImpl.scala:90) finished in 0.023 s
15/06/03 18:34:30 INFO BlockManager: Removing broadcast 113
15/06/03 18:34:30 INFO BlockManager: Removing block broadcast_113_piece0
15/06/03 18:34:30 INFO MemoryStore: Block broadcast_113_piece0 of size 9481 dropped from memory (free 558742664)
15/06/03 18:34:30 INFO BlockManagerInfo: Removed broadcast_113_piece0 on localhost:60836 in memory (size: 9.3 KB, free: 533.1 MB)
15/06/03 18:34:30 INFO BlockManagerMaster: Updated info of block broadcast_113_piece0
15/06/03 18:34:30 INFO BlockManager: Removing block broadcast_113
15/06/03 18:34:30 INFO MemoryStore: Block broadcast_113 of size 20408 dropped from memory (free 558763072)
15/06/03 18:34:30 INFO ContextCleaner: Cleaned broadcast 113
15/06/03 18:34:30 INFO BlockManager: Removing broadcast 118
15/06/03 18:34:30 INFO BlockManager: Removing block broadcast_118_piece0
15/06/03 18:34:30 INFO MemoryStore: Block broadcast_118_piece0 of size 9631 dropped from memory (free 558772703)
15/06/03 18:34:30 INFO BlockManagerInfo: Removed broadcast_118_piece0 on localhost:60836 in memory (size: 9.4 KB, free: 533.2 MB)
15/06/03 18:34:30 INFO BlockManagerMaster: Updated info of block broadcast_118_piece0
15/06/03 18:34:30 INFO BlockManager: Removing block broadcast_118
15/06/03 18:34:30 INFO MemoryStore: Block broadcast_118 of size 20592 dropped from memory (free 558793295)
15/06/03 18:34:30 INFO ContextCleaner: Cleaned broadcast 118
15/06/03 18:34:30 INFO BlockManager: Removing broadcast 117
15/06/03 18:34:30 INFO BlockManager: Removing block broadcast_117_piece0
15/06/03 18:34:30 INFO MemoryStore: Block broadcast_117_piece0 of size 9777 dropped from memory (free 558803072)
15/06/03 18:34:30 INFO BlockManagerInfo: Removed broadcast_117_piece0 on localhost:60836 in memory (size: 9.5 KB, free: 533.2 MB)
15/06/03 18:34:30 INFO BlockManagerMaster: Updated info of block broadcast_117_piece0
15/06/03 18:34:30 INFO BlockManager: Removing block broadcast_117
15/06/03 18:34:30 INFO MemoryStore: Block broadcast_117 of size 20928 dropped from memory (free 558824000)
15/06/03 18:34:30 INFO ContextCleaner: Cleaned broadcast 117
15/06/03 18:34:30 INFO DAGScheduler: Job 30 finished: reduce at VertexRDDImpl.scala:90, took 0.336311 s
15/06/03 18:34:30 INFO Pregel: Pregel finished iteration 28
15/06/03 18:34:30 INFO ZippedPartitionsRDD2: Removing RDD 575 from persistence list
15/06/03 18:34:30 INFO BlockManager: Removing RDD 575
15/06/03 18:34:30 INFO BlockManager: Removing block rdd_575_0
15/06/03 18:34:30 INFO MemoryStore: Block rdd_575_0 of size 1664 dropped from memory (free 558825664)
15/06/03 18:34:30 INFO ZippedPartitionsRDD2: Removing RDD 578 from persistence list
15/06/03 18:34:30 INFO BlockManager: Removing RDD 578
15/06/03 18:34:30 INFO BlockManager: Removing block rdd_578_0
15/06/03 18:34:30 INFO MemoryStore: Block rdd_578_0 of size 1544 dropped from memory (free 558827208)
15/06/03 18:34:30 INFO ZippedPartitionsRDD2: Removing RDD 561 from persistence list
15/06/03 18:34:30 INFO BlockManager: Removing RDD 561
15/06/03 18:34:30 INFO ZippedPartitionsRDD2: Removing RDD 567 from persistence list
15/06/03 18:34:30 INFO BlockManager: Removing block rdd_561_0
15/06/03 18:34:30 INFO MemoryStore: Block rdd_561_0 of size 1496 dropped from memory (free 558828704)
15/06/03 18:34:30 INFO BlockManager: Removing RDD 567
15/06/03 18:34:30 INFO BlockManager: Removing block rdd_567_0
15/06/03 18:34:30 INFO MemoryStore: Block rdd_567_0 of size 2976 dropped from memory (free 558831680)
15/06/03 18:34:30 INFO SparkContext: Starting job: reduce at VertexRDDImpl.scala:90
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 2 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 3 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 5 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 4 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 1 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 0 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 8 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 7 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 6 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 11 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 10 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 9 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 14 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 13 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 12 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 17 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 16 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 15 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 20 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 19 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 18 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 23 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 22 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 21 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 26 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 25 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 24 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 29 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 28 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 27 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 32 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 31 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 30 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 35 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 34 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 33 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 38 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 37 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 36 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 41 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 40 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 39 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 44 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 43 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 42 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 47 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 46 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 45 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 50 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 49 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 48 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 53 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 52 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 51 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 56 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 55 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 54 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 59 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 58 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 57 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 62 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 61 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 60 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 65 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 64 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 63 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 68 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 67 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 66 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 71 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 70 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 69 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 74 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 73 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 72 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 77 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 76 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 75 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 80 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 79 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 78 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 83 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 82 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 81 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 86 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 85 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 84 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 89 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 88 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 87 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 92 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 91 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 90 is 145 bytes
15/06/03 18:34:30 INFO DAGScheduler: Registering RDD 603 (mapPartitions at VertexRDDImpl.scala:218)
15/06/03 18:34:30 INFO DAGScheduler: Registering RDD 607 (mapPartitions at VertexRDDImpl.scala:222)
15/06/03 18:34:30 INFO DAGScheduler: Registering RDD 611 (mapPartitions at GraphImpl.scala:235)
15/06/03 18:34:30 INFO DAGScheduler: Got job 31 (reduce at VertexRDDImpl.scala:90) with 1 output partitions (allowLocal=false)
15/06/03 18:34:30 INFO DAGScheduler: Final stage: Stage 17949(reduce at VertexRDDImpl.scala:90)
15/06/03 18:34:30 INFO DAGScheduler: Parents of final stage: List(Stage 17902, Stage 17932, Stage 17924, Stage 17904, Stage 17896, Stage 17926, Stage 17948, Stage 17898, Stage 17928, Stage 17920, Stage 17900, Stage 17892, Stage 17922, Stage 17944, Stage 17894, Stage 17916, Stage 17946, Stage 17938, Stage 17888, Stage 17918, Stage 17910, Stage 17940, Stage 17890, Stage 17912, Stage 17942, Stage 17934, Stage 17914, Stage 17906, Stage 17885, Stage 17936, Stage 17886, Stage 17908, Stage 17930)
15/06/03 18:34:30 INFO DAGScheduler: Missing parents: List(Stage 17948)
15/06/03 18:34:30 INFO DAGScheduler: Submitting Stage 17853 (ReplicatedVertexView.updateVertices - shippedVerts true false (broadcast) MapPartitionsRDD[603] at mapPartitions at VertexRDDImpl.scala:218), which has no missing parents
15/06/03 18:34:30 INFO MemoryStore: ensureFreeSpace(21984) called with curMem=321472, maxMem=559153152
15/06/03 18:34:30 INFO MemoryStore: Block broadcast_125 stored as values in memory (estimated size 21.5 KB, free 532.9 MB)
15/06/03 18:34:30 INFO MemoryStore: ensureFreeSpace(10151) called with curMem=343456, maxMem=559153152
15/06/03 18:34:30 INFO MemoryStore: Block broadcast_125_piece0 stored as bytes in memory (estimated size 9.9 KB, free 532.9 MB)
15/06/03 18:34:30 INFO BlockManagerInfo: Added broadcast_125_piece0 in memory on localhost:60836 (size: 9.9 KB, free: 533.2 MB)
15/06/03 18:34:30 INFO BlockManagerMaster: Updated info of block broadcast_125_piece0
15/06/03 18:34:30 INFO SparkContext: Created broadcast 125 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:30 INFO DAGScheduler: Submitting 1 missing tasks from Stage 17853 (ReplicatedVertexView.updateVertices - shippedVerts true false (broadcast) MapPartitionsRDD[603] at mapPartitions at VertexRDDImpl.scala:218)
15/06/03 18:34:30 INFO TaskSchedulerImpl: Adding task set 17853.0 with 1 tasks
15/06/03 18:34:30 INFO DAGScheduler: Submitting Stage 17947 (ReplicatedVertexView.withActiveSet - shippedActives (broadcast) MapPartitionsRDD[607] at mapPartitions at VertexRDDImpl.scala:222), which has no missing parents
15/06/03 18:34:30 INFO TaskSetManager: Starting task 0.0 in stage 17853.0 (TID 124, localhost, PROCESS_LOCAL, 4676 bytes)
15/06/03 18:34:30 INFO Executor: Running task 0.0 in stage 17853.0 (TID 124)
15/06/03 18:34:30 INFO MemoryStore: ensureFreeSpace(21640) called with curMem=353607, maxMem=559153152
15/06/03 18:34:30 INFO MemoryStore: Block broadcast_126 stored as values in memory (estimated size 21.1 KB, free 532.9 MB)
15/06/03 18:34:30 INFO MemoryStore: ensureFreeSpace(9978) called with curMem=375247, maxMem=559153152
15/06/03 18:34:30 INFO MemoryStore: Block broadcast_126_piece0 stored as bytes in memory (estimated size 9.7 KB, free 532.9 MB)
15/06/03 18:34:30 INFO BlockManagerInfo: Added broadcast_126_piece0 in memory on localhost:60836 (size: 9.7 KB, free: 533.2 MB)
15/06/03 18:34:30 INFO BlockManagerMaster: Updated info of block broadcast_126_piece0
15/06/03 18:34:30 INFO SparkContext: Created broadcast 126 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:30 INFO DAGScheduler: Submitting 1 missing tasks from Stage 17947 (ReplicatedVertexView.withActiveSet - shippedActives (broadcast) MapPartitionsRDD[607] at mapPartitions at VertexRDDImpl.scala:222)
15/06/03 18:34:30 INFO TaskSchedulerImpl: Adding task set 17947.0 with 1 tasks
15/06/03 18:34:30 INFO BlockManager: Found block rdd_580_0 locally
15/06/03 18:34:30 INFO CacheManager: Partition rdd_599_0 not found, computing it
15/06/03 18:34:30 INFO BlockManager: Found block rdd_580_0 locally
15/06/03 18:34:30 INFO CacheManager: Partition rdd_597_0 not found, computing it
15/06/03 18:34:30 INFO BlockManager: Found block rdd_580_0 locally
15/06/03 18:34:30 INFO BlockManager: Found block rdd_594_0 locally
15/06/03 18:34:30 INFO MemoryStore: ensureFreeSpace(1544) called with curMem=385225, maxMem=559153152
15/06/03 18:34:30 INFO MemoryStore: Block rdd_597_0 stored as values in memory (estimated size 1544.0 B, free 532.9 MB)
15/06/03 18:34:30 INFO BlockManagerInfo: Added rdd_597_0 in memory on localhost:60836 (size: 1544.0 B, free: 533.1 MB)
15/06/03 18:34:30 INFO BlockManagerMaster: Updated info of block rdd_597_0
15/06/03 18:34:30 INFO MemoryStore: ensureFreeSpace(1496) called with curMem=386769, maxMem=559153152
15/06/03 18:34:30 INFO MemoryStore: Block rdd_599_0 stored as values in memory (estimated size 1496.0 B, free 532.9 MB)
15/06/03 18:34:30 INFO BlockManagerInfo: Added rdd_599_0 in memory on localhost:60836 (size: 1496.0 B, free: 533.1 MB)
15/06/03 18:34:30 INFO BlockManagerMaster: Updated info of block rdd_599_0
15/06/03 18:34:30 INFO Executor: Finished task 0.0 in stage 17853.0 (TID 124). 2627 bytes result sent to driver
15/06/03 18:34:30 INFO TaskSetManager: Starting task 0.0 in stage 17947.0 (TID 125, localhost, PROCESS_LOCAL, 4612 bytes)
15/06/03 18:34:30 INFO TaskSetManager: Finished task 0.0 in stage 17853.0 (TID 124) in 27 ms on localhost (1/1)
15/06/03 18:34:30 INFO Executor: Running task 0.0 in stage 17947.0 (TID 125)
15/06/03 18:34:30 INFO TaskSchedulerImpl: Removed TaskSet 17853.0, whose tasks have all completed, from pool
15/06/03 18:34:30 INFO DAGScheduler: Stage 17853 (mapPartitions at VertexRDDImpl.scala:218) finished in 0.027 s
15/06/03 18:34:30 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:30 INFO DAGScheduler: running: Set(Stage 17947)
15/06/03 18:34:30 INFO DAGScheduler: waiting: Set(Stage 17948, Stage 17949)
15/06/03 18:34:30 INFO DAGScheduler: failed: Set()
15/06/03 18:34:30 INFO DAGScheduler: Missing parents for Stage 17948: List(Stage 17947)
15/06/03 18:34:30 INFO DAGScheduler: Missing parents for Stage 17949: List(Stage 17948)
15/06/03 18:34:30 INFO BlockManager: Found block rdd_597_0 locally
15/06/03 18:34:30 INFO Executor: Finished task 0.0 in stage 17947.0 (TID 125). 2003 bytes result sent to driver
15/06/03 18:34:30 INFO TaskSetManager: Finished task 0.0 in stage 17947.0 (TID 125) in 12 ms on localhost (1/1)
15/06/03 18:34:30 INFO TaskSchedulerImpl: Removed TaskSet 17947.0, whose tasks have all completed, from pool
15/06/03 18:34:30 INFO DAGScheduler: Stage 17947 (mapPartitions at VertexRDDImpl.scala:222) finished in 0.033 s
15/06/03 18:34:30 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:30 INFO DAGScheduler: running: Set()
15/06/03 18:34:30 INFO DAGScheduler: waiting: Set(Stage 17948, Stage 17949)
15/06/03 18:34:30 INFO DAGScheduler: failed: Set()
15/06/03 18:34:30 INFO DAGScheduler: Missing parents for Stage 17948: List()
15/06/03 18:34:30 INFO DAGScheduler: Missing parents for Stage 17949: List(Stage 17948)
15/06/03 18:34:30 INFO DAGScheduler: Submitting Stage 17948 (GraphImpl.aggregateMessages - preAgg MapPartitionsRDD[611] at mapPartitions at GraphImpl.scala:235), which is now runnable
15/06/03 18:34:30 INFO MemoryStore: ensureFreeSpace(13688) called with curMem=388265, maxMem=559153152
15/06/03 18:34:30 INFO MemoryStore: Block broadcast_127 stored as values in memory (estimated size 13.4 KB, free 532.9 MB)
15/06/03 18:34:30 INFO MemoryStore: ensureFreeSpace(7128) called with curMem=401953, maxMem=559153152
15/06/03 18:34:30 INFO MemoryStore: Block broadcast_127_piece0 stored as bytes in memory (estimated size 7.0 KB, free 532.9 MB)
15/06/03 18:34:30 INFO BlockManagerInfo: Added broadcast_127_piece0 in memory on localhost:60836 (size: 7.0 KB, free: 533.1 MB)
15/06/03 18:34:30 INFO BlockManagerMaster: Updated info of block broadcast_127_piece0
15/06/03 18:34:30 INFO SparkContext: Created broadcast 127 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:30 INFO DAGScheduler: Submitting 1 missing tasks from Stage 17948 (GraphImpl.aggregateMessages - preAgg MapPartitionsRDD[611] at mapPartitions at GraphImpl.scala:235)
15/06/03 18:34:30 INFO TaskSchedulerImpl: Adding task set 17948.0 with 1 tasks
15/06/03 18:34:30 INFO TaskSetManager: Starting task 0.0 in stage 17948.0 (TID 126, localhost, PROCESS_LOCAL, 3165 bytes)
15/06/03 18:34:30 INFO Executor: Running task 0.0 in stage 17948.0 (TID 126)
15/06/03 18:34:30 INFO CacheManager: Partition rdd_605_0 not found, computing it
15/06/03 18:34:30 INFO BlockManager: Found block rdd_586_0 locally
15/06/03 18:34:30 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:30 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:30 INFO MemoryStore: ensureFreeSpace(2976) called with curMem=409081, maxMem=559153152
15/06/03 18:34:30 INFO MemoryStore: Block rdd_605_0 stored as values in memory (estimated size 2.9 KB, free 532.9 MB)
15/06/03 18:34:30 INFO BlockManagerInfo: Added rdd_605_0 in memory on localhost:60836 (size: 2.9 KB, free: 533.1 MB)
15/06/03 18:34:30 INFO BlockManagerMaster: Updated info of block rdd_605_0
15/06/03 18:34:30 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:30 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:30 INFO Executor: Finished task 0.0 in stage 17948.0 (TID 126). 2788 bytes result sent to driver
15/06/03 18:34:30 INFO TaskSetManager: Finished task 0.0 in stage 17948.0 (TID 126) in 17 ms on localhost (1/1)
15/06/03 18:34:30 INFO TaskSchedulerImpl: Removed TaskSet 17948.0, whose tasks have all completed, from pool
15/06/03 18:34:30 INFO DAGScheduler: Stage 17948 (mapPartitions at GraphImpl.scala:235) finished in 0.017 s
15/06/03 18:34:30 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:30 INFO DAGScheduler: running: Set()
15/06/03 18:34:30 INFO DAGScheduler: waiting: Set(Stage 17949)
15/06/03 18:34:30 INFO DAGScheduler: failed: Set()
15/06/03 18:34:30 INFO DAGScheduler: Missing parents for Stage 17949: List()
15/06/03 18:34:30 INFO DAGScheduler: Submitting Stage 17949 (MapPartitionsRDD[615] at map at VertexRDDImpl.scala:90), which is now runnable
15/06/03 18:34:30 INFO MemoryStore: ensureFreeSpace(22272) called with curMem=412057, maxMem=559153152
15/06/03 18:34:30 INFO MemoryStore: Block broadcast_128 stored as values in memory (estimated size 21.8 KB, free 532.8 MB)
15/06/03 18:34:30 INFO MemoryStore: ensureFreeSpace(10219) called with curMem=434329, maxMem=559153152
15/06/03 18:34:30 INFO MemoryStore: Block broadcast_128_piece0 stored as bytes in memory (estimated size 10.0 KB, free 532.8 MB)
15/06/03 18:34:30 INFO BlockManagerInfo: Added broadcast_128_piece0 in memory on localhost:60836 (size: 10.0 KB, free: 533.1 MB)
15/06/03 18:34:30 INFO BlockManagerMaster: Updated info of block broadcast_128_piece0
15/06/03 18:34:30 INFO SparkContext: Created broadcast 128 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:30 INFO DAGScheduler: Submitting 1 missing tasks from Stage 17949 (MapPartitionsRDD[615] at map at VertexRDDImpl.scala:90)
15/06/03 18:34:30 INFO TaskSchedulerImpl: Adding task set 17949.0 with 1 tasks
15/06/03 18:34:30 INFO TaskSetManager: Starting task 0.0 in stage 17949.0 (TID 127, localhost, PROCESS_LOCAL, 4696 bytes)
15/06/03 18:34:30 INFO Executor: Running task 0.0 in stage 17949.0 (TID 127)
15/06/03 18:34:30 INFO CacheManager: Partition rdd_613_0 not found, computing it
15/06/03 18:34:30 INFO BlockManager: Found block rdd_599_0 locally
15/06/03 18:34:30 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:30 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:30 INFO MemoryStore: ensureFreeSpace(1664) called with curMem=444548, maxMem=559153152
15/06/03 18:34:30 INFO MemoryStore: Block rdd_613_0 stored as values in memory (estimated size 1664.0 B, free 532.8 MB)
15/06/03 18:34:30 INFO BlockManagerInfo: Added rdd_613_0 in memory on localhost:60836 (size: 1664.0 B, free: 533.1 MB)
15/06/03 18:34:30 INFO BlockManagerMaster: Updated info of block rdd_613_0
15/06/03 18:34:30 INFO Executor: Finished task 0.0 in stage 17949.0 (TID 127). 2693 bytes result sent to driver
15/06/03 18:34:30 INFO TaskSetManager: Finished task 0.0 in stage 17949.0 (TID 127) in 8 ms on localhost (1/1)
15/06/03 18:34:30 INFO TaskSchedulerImpl: Removed TaskSet 17949.0, whose tasks have all completed, from pool
15/06/03 18:34:30 INFO DAGScheduler: Stage 17949 (reduce at VertexRDDImpl.scala:90) finished in 0.008 s
15/06/03 18:34:30 INFO DAGScheduler: Job 31 finished: reduce at VertexRDDImpl.scala:90, took 0.238681 s
15/06/03 18:34:30 INFO Pregel: Pregel finished iteration 29
15/06/03 18:34:30 INFO ZippedPartitionsRDD2: Removing RDD 594 from persistence list
15/06/03 18:34:30 INFO BlockManager: Removing RDD 594
15/06/03 18:34:30 INFO ZippedPartitionsRDD2: Removing RDD 597 from persistence list
15/06/03 18:34:30 INFO BlockManager: Removing block rdd_594_0
15/06/03 18:34:30 INFO MemoryStore: Block rdd_594_0 of size 1664 dropped from memory (free 558708604)
15/06/03 18:34:30 INFO BlockManager: Removing RDD 597
15/06/03 18:34:30 INFO BlockManager: Removing block rdd_597_0
15/06/03 18:34:30 INFO MemoryStore: Block rdd_597_0 of size 1544 dropped from memory (free 558710148)
15/06/03 18:34:30 INFO ZippedPartitionsRDD2: Removing RDD 580 from persistence list
15/06/03 18:34:30 INFO BlockManager: Removing RDD 580
15/06/03 18:34:30 INFO BlockManager: Removing block rdd_580_0
15/06/03 18:34:30 INFO MemoryStore: Block rdd_580_0 of size 1496 dropped from memory (free 558711644)
15/06/03 18:34:30 INFO ZippedPartitionsRDD2: Removing RDD 586 from persistence list
15/06/03 18:34:30 INFO BlockManager: Removing RDD 586
15/06/03 18:34:30 INFO BlockManager: Removing block rdd_586_0
15/06/03 18:34:30 INFO MemoryStore: Block rdd_586_0 of size 2976 dropped from memory (free 558714620)
15/06/03 18:34:30 INFO SparkContext: Starting job: reduce at VertexRDDImpl.scala:90
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 2 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 3 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 5 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 4 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 1 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 0 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 8 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 7 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 6 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 11 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 10 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 9 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 14 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 13 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 12 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 17 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 16 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 15 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 20 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 19 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 18 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 23 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 22 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 21 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 26 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 25 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 24 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 29 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 28 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 27 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 32 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 31 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 30 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 35 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 34 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 33 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 38 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 37 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 36 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 41 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 40 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 39 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 44 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 43 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 42 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 47 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 46 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 45 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 50 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 49 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 48 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 53 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 52 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 51 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 56 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 55 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 54 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 59 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 58 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 57 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 62 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 61 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 60 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 65 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 64 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 63 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 68 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 67 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 66 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 71 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 70 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 69 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 74 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 73 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 72 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 77 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 76 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 75 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 80 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 79 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 78 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 83 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 82 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 81 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 86 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 85 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 84 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 89 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 88 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 87 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 92 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 91 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 90 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 95 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 94 is 145 bytes
15/06/03 18:34:30 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 93 is 145 bytes
15/06/03 18:34:30 INFO DAGScheduler: Registering RDD 622 (mapPartitions at VertexRDDImpl.scala:218)
15/06/03 18:34:30 INFO DAGScheduler: Registering RDD 626 (mapPartitions at VertexRDDImpl.scala:222)
15/06/03 18:34:30 INFO DAGScheduler: Registering RDD 630 (mapPartitions at GraphImpl.scala:235)
15/06/03 18:34:30 INFO DAGScheduler: Got job 32 (reduce at VertexRDDImpl.scala:90) with 1 output partitions (allowLocal=false)
15/06/03 18:34:30 INFO DAGScheduler: Final stage: Stage 19632(reduce at VertexRDDImpl.scala:90)
15/06/03 18:34:30 INFO DAGScheduler: Parents of final stage: List(Stage 19601, Stage 19631, Stage 19623, Stage 19573, Stage 19603, Stage 19595, Stage 19625, Stage 19575, Stage 19567, Stage 19597, Stage 19619, Stage 19569, Stage 19599, Stage 19591, Stage 19621, Stage 19571, Stage 19593, Stage 19615, Stage 19587, Stage 19566, Stage 19617, Stage 19609, Stage 19589, Stage 19581, Stage 19611, Stage 19583, Stage 19613, Stage 19605, Stage 19627, Stage 19585, Stage 19577, Stage 19607, Stage 19629, Stage 19579)
15/06/03 18:34:30 INFO DAGScheduler: Missing parents: List(Stage 19631)
15/06/03 18:34:30 INFO DAGScheduler: Submitting Stage 19533 (ReplicatedVertexView.updateVertices - shippedVerts true false (broadcast) MapPartitionsRDD[622] at mapPartitions at VertexRDDImpl.scala:218), which has no missing parents
15/06/03 18:34:30 INFO MemoryStore: ensureFreeSpace(22512) called with curMem=438532, maxMem=559153152
15/06/03 18:34:30 INFO MemoryStore: Block broadcast_129 stored as values in memory (estimated size 22.0 KB, free 532.8 MB)
15/06/03 18:34:30 INFO MemoryStore: ensureFreeSpace(10272) called with curMem=461044, maxMem=559153152
15/06/03 18:34:30 INFO MemoryStore: Block broadcast_129_piece0 stored as bytes in memory (estimated size 10.0 KB, free 532.8 MB)
15/06/03 18:34:30 INFO BlockManagerInfo: Added broadcast_129_piece0 in memory on localhost:60836 (size: 10.0 KB, free: 533.1 MB)
15/06/03 18:34:30 INFO BlockManagerMaster: Updated info of block broadcast_129_piece0
15/06/03 18:34:30 INFO SparkContext: Created broadcast 129 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:30 INFO DAGScheduler: Submitting 1 missing tasks from Stage 19533 (ReplicatedVertexView.updateVertices - shippedVerts true false (broadcast) MapPartitionsRDD[622] at mapPartitions at VertexRDDImpl.scala:218)
15/06/03 18:34:30 INFO TaskSchedulerImpl: Adding task set 19533.0 with 1 tasks
15/06/03 18:34:30 INFO DAGScheduler: Submitting Stage 19630 (ReplicatedVertexView.withActiveSet - shippedActives (broadcast) MapPartitionsRDD[626] at mapPartitions at VertexRDDImpl.scala:222), which has no missing parents
15/06/03 18:34:30 INFO TaskSetManager: Starting task 0.0 in stage 19533.0 (TID 128, localhost, PROCESS_LOCAL, 4781 bytes)
15/06/03 18:34:30 INFO Executor: Running task 0.0 in stage 19533.0 (TID 128)
15/06/03 18:34:30 INFO MemoryStore: ensureFreeSpace(22168) called with curMem=471316, maxMem=559153152
15/06/03 18:34:30 INFO MemoryStore: Block broadcast_130 stored as values in memory (estimated size 21.6 KB, free 532.8 MB)
15/06/03 18:34:30 INFO MemoryStore: ensureFreeSpace(10164) called with curMem=493484, maxMem=559153152
15/06/03 18:34:30 INFO MemoryStore: Block broadcast_130_piece0 stored as bytes in memory (estimated size 9.9 KB, free 532.8 MB)
15/06/03 18:34:30 INFO BlockManagerInfo: Added broadcast_130_piece0 in memory on localhost:60836 (size: 9.9 KB, free: 533.1 MB)
15/06/03 18:34:30 INFO BlockManagerMaster: Updated info of block broadcast_130_piece0
15/06/03 18:34:30 INFO SparkContext: Created broadcast 130 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:30 INFO DAGScheduler: Submitting 1 missing tasks from Stage 19630 (ReplicatedVertexView.withActiveSet - shippedActives (broadcast) MapPartitionsRDD[626] at mapPartitions at VertexRDDImpl.scala:222)
15/06/03 18:34:30 INFO TaskSchedulerImpl: Adding task set 19630.0 with 1 tasks
15/06/03 18:34:30 INFO BlockManager: Found block rdd_599_0 locally
15/06/03 18:34:30 INFO CacheManager: Partition rdd_618_0 not found, computing it
15/06/03 18:34:30 INFO BlockManager: Found block rdd_599_0 locally
15/06/03 18:34:30 INFO CacheManager: Partition rdd_616_0 not found, computing it
15/06/03 18:34:30 INFO BlockManager: Found block rdd_599_0 locally
15/06/03 18:34:30 INFO BlockManager: Found block rdd_613_0 locally
15/06/03 18:34:30 INFO MemoryStore: ensureFreeSpace(1544) called with curMem=503648, maxMem=559153152
15/06/03 18:34:30 INFO MemoryStore: Block rdd_616_0 stored as values in memory (estimated size 1544.0 B, free 532.8 MB)
15/06/03 18:34:30 INFO BlockManagerInfo: Added rdd_616_0 in memory on localhost:60836 (size: 1544.0 B, free: 533.1 MB)
15/06/03 18:34:30 INFO BlockManagerMaster: Updated info of block rdd_616_0
15/06/03 18:34:30 INFO MemoryStore: ensureFreeSpace(1496) called with curMem=505192, maxMem=559153152
15/06/03 18:34:30 INFO MemoryStore: Block rdd_618_0 stored as values in memory (estimated size 1496.0 B, free 532.8 MB)
15/06/03 18:34:30 INFO BlockManagerInfo: Added rdd_618_0 in memory on localhost:60836 (size: 1496.0 B, free: 533.1 MB)
15/06/03 18:34:30 INFO BlockManagerMaster: Updated info of block rdd_618_0
15/06/03 18:34:30 INFO Executor: Finished task 0.0 in stage 19533.0 (TID 128). 2627 bytes result sent to driver
15/06/03 18:34:30 INFO TaskSetManager: Starting task 0.0 in stage 19630.0 (TID 129, localhost, PROCESS_LOCAL, 4717 bytes)
15/06/03 18:34:30 INFO TaskSetManager: Finished task 0.0 in stage 19533.0 (TID 128) in 34 ms on localhost (1/1)
15/06/03 18:34:30 INFO TaskSchedulerImpl: Removed TaskSet 19533.0, whose tasks have all completed, from pool
15/06/03 18:34:30 INFO Executor: Running task 0.0 in stage 19630.0 (TID 129)
15/06/03 18:34:30 INFO DAGScheduler: Stage 19533 (mapPartitions at VertexRDDImpl.scala:218) finished in 0.035 s
15/06/03 18:34:30 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:30 INFO DAGScheduler: running: Set(Stage 19630)
15/06/03 18:34:30 INFO DAGScheduler: waiting: Set(Stage 19631, Stage 19632)
15/06/03 18:34:30 INFO DAGScheduler: failed: Set()
15/06/03 18:34:30 INFO DAGScheduler: Missing parents for Stage 19631: List(Stage 19630)
15/06/03 18:34:30 INFO DAGScheduler: Missing parents for Stage 19632: List(Stage 19631)
15/06/03 18:34:30 INFO BlockManager: Found block rdd_616_0 locally
15/06/03 18:34:30 INFO Executor: Finished task 0.0 in stage 19630.0 (TID 129). 2003 bytes result sent to driver
15/06/03 18:34:30 INFO TaskSetManager: Finished task 0.0 in stage 19630.0 (TID 129) in 16 ms on localhost (1/1)
15/06/03 18:34:30 INFO TaskSchedulerImpl: Removed TaskSet 19630.0, whose tasks have all completed, from pool
15/06/03 18:34:30 INFO DAGScheduler: Stage 19630 (mapPartitions at VertexRDDImpl.scala:222) finished in 0.038 s
15/06/03 18:34:30 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:30 INFO DAGScheduler: running: Set()
15/06/03 18:34:30 INFO DAGScheduler: waiting: Set(Stage 19631, Stage 19632)
15/06/03 18:34:30 INFO DAGScheduler: failed: Set()
15/06/03 18:34:30 INFO DAGScheduler: Missing parents for Stage 19631: List()
15/06/03 18:34:30 INFO DAGScheduler: Missing parents for Stage 19632: List(Stage 19631)
15/06/03 18:34:30 INFO DAGScheduler: Submitting Stage 19631 (GraphImpl.aggregateMessages - preAgg MapPartitionsRDD[630] at mapPartitions at GraphImpl.scala:235), which is now runnable
15/06/03 18:34:30 INFO MemoryStore: ensureFreeSpace(13920) called with curMem=506688, maxMem=559153152
15/06/03 18:34:30 INFO MemoryStore: Block broadcast_131 stored as values in memory (estimated size 13.6 KB, free 532.8 MB)
15/06/03 18:34:30 INFO MemoryStore: ensureFreeSpace(7226) called with curMem=520608, maxMem=559153152
15/06/03 18:34:30 INFO MemoryStore: Block broadcast_131_piece0 stored as bytes in memory (estimated size 7.1 KB, free 532.7 MB)
15/06/03 18:34:30 INFO BlockManagerInfo: Added broadcast_131_piece0 in memory on localhost:60836 (size: 7.1 KB, free: 533.1 MB)
15/06/03 18:34:30 INFO BlockManagerMaster: Updated info of block broadcast_131_piece0
15/06/03 18:34:30 INFO SparkContext: Created broadcast 131 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:30 INFO DAGScheduler: Submitting 1 missing tasks from Stage 19631 (GraphImpl.aggregateMessages - preAgg MapPartitionsRDD[630] at mapPartitions at GraphImpl.scala:235)
15/06/03 18:34:30 INFO TaskSchedulerImpl: Adding task set 19631.0 with 1 tasks
15/06/03 18:34:30 INFO TaskSetManager: Starting task 0.0 in stage 19631.0 (TID 130, localhost, PROCESS_LOCAL, 3206 bytes)
15/06/03 18:34:30 INFO Executor: Running task 0.0 in stage 19631.0 (TID 130)
15/06/03 18:34:30 INFO CacheManager: Partition rdd_624_0 not found, computing it
15/06/03 18:34:30 INFO BlockManager: Found block rdd_605_0 locally
15/06/03 18:34:30 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:30 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:30 INFO MemoryStore: ensureFreeSpace(2976) called with curMem=527834, maxMem=559153152
15/06/03 18:34:30 INFO MemoryStore: Block rdd_624_0 stored as values in memory (estimated size 2.9 KB, free 532.7 MB)
15/06/03 18:34:30 INFO BlockManagerInfo: Added rdd_624_0 in memory on localhost:60836 (size: 2.9 KB, free: 533.1 MB)
15/06/03 18:34:30 INFO BlockManagerMaster: Updated info of block rdd_624_0
15/06/03 18:34:30 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:30 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:30 INFO Executor: Finished task 0.0 in stage 19631.0 (TID 130). 2788 bytes result sent to driver
15/06/03 18:34:30 INFO TaskSetManager: Finished task 0.0 in stage 19631.0 (TID 130) in 13 ms on localhost (1/1)
15/06/03 18:34:30 INFO TaskSchedulerImpl: Removed TaskSet 19631.0, whose tasks have all completed, from pool
15/06/03 18:34:30 INFO DAGScheduler: Stage 19631 (mapPartitions at GraphImpl.scala:235) finished in 0.013 s
15/06/03 18:34:30 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:30 INFO DAGScheduler: running: Set()
15/06/03 18:34:30 INFO DAGScheduler: waiting: Set(Stage 19632)
15/06/03 18:34:30 INFO DAGScheduler: failed: Set()
15/06/03 18:34:30 INFO DAGScheduler: Missing parents for Stage 19632: List()
15/06/03 18:34:30 INFO DAGScheduler: Submitting Stage 19632 (MapPartitionsRDD[634] at map at VertexRDDImpl.scala:90), which is now runnable
15/06/03 18:34:30 INFO MemoryStore: ensureFreeSpace(22800) called with curMem=530810, maxMem=559153152
15/06/03 18:34:30 INFO MemoryStore: Block broadcast_132 stored as values in memory (estimated size 22.3 KB, free 532.7 MB)
15/06/03 18:34:30 INFO MemoryStore: ensureFreeSpace(10447) called with curMem=553610, maxMem=559153152
15/06/03 18:34:30 INFO MemoryStore: Block broadcast_132_piece0 stored as bytes in memory (estimated size 10.2 KB, free 532.7 MB)
15/06/03 18:34:30 INFO BlockManagerInfo: Added broadcast_132_piece0 in memory on localhost:60836 (size: 10.2 KB, free: 533.1 MB)
15/06/03 18:34:30 INFO BlockManagerMaster: Updated info of block broadcast_132_piece0
15/06/03 18:34:30 INFO SparkContext: Created broadcast 132 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:30 INFO DAGScheduler: Submitting 1 missing tasks from Stage 19632 (MapPartitionsRDD[634] at map at VertexRDDImpl.scala:90)
15/06/03 18:34:30 INFO TaskSchedulerImpl: Adding task set 19632.0 with 1 tasks
15/06/03 18:34:30 INFO TaskSetManager: Starting task 0.0 in stage 19632.0 (TID 131, localhost, PROCESS_LOCAL, 4801 bytes)
15/06/03 18:34:30 INFO Executor: Running task 0.0 in stage 19632.0 (TID 131)
15/06/03 18:34:31 INFO CacheManager: Partition rdd_632_0 not found, computing it
15/06/03 18:34:31 INFO BlockManager: Found block rdd_618_0 locally
15/06/03 18:34:31 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:31 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:31 INFO MemoryStore: ensureFreeSpace(1664) called with curMem=564057, maxMem=559153152
15/06/03 18:34:31 INFO MemoryStore: Block rdd_632_0 stored as values in memory (estimated size 1664.0 B, free 532.7 MB)
15/06/03 18:34:31 INFO BlockManagerInfo: Added rdd_632_0 in memory on localhost:60836 (size: 1664.0 B, free: 533.1 MB)
15/06/03 18:34:31 INFO BlockManagerMaster: Updated info of block rdd_632_0
15/06/03 18:34:31 INFO Executor: Finished task 0.0 in stage 19632.0 (TID 131). 2693 bytes result sent to driver
15/06/03 18:34:31 INFO TaskSetManager: Finished task 0.0 in stage 19632.0 (TID 131) in 15 ms on localhost (1/1)
15/06/03 18:34:31 INFO TaskSchedulerImpl: Removed TaskSet 19632.0, whose tasks have all completed, from pool
15/06/03 18:34:31 INFO DAGScheduler: Stage 19632 (reduce at VertexRDDImpl.scala:90) finished in 0.016 s
15/06/03 18:34:31 INFO DAGScheduler: Job 32 finished: reduce at VertexRDDImpl.scala:90, took 0.280265 s
15/06/03 18:34:31 INFO Pregel: Pregel finished iteration 30
15/06/03 18:34:31 INFO ZippedPartitionsRDD2: Removing RDD 613 from persistence list
15/06/03 18:34:31 INFO BlockManager: Removing RDD 613
15/06/03 18:34:31 INFO ZippedPartitionsRDD2: Removing RDD 616 from persistence list
15/06/03 18:34:31 INFO BlockManager: Removing block rdd_613_0
15/06/03 18:34:31 INFO MemoryStore: Block rdd_613_0 of size 1664 dropped from memory (free 558589095)
15/06/03 18:34:31 INFO BlockManager: Removing RDD 616
15/06/03 18:34:31 INFO BlockManager: Removing block rdd_616_0
15/06/03 18:34:31 INFO MemoryStore: Block rdd_616_0 of size 1544 dropped from memory (free 558590639)
15/06/03 18:34:31 INFO ZippedPartitionsRDD2: Removing RDD 599 from persistence list
15/06/03 18:34:31 INFO BlockManager: Removing RDD 599
15/06/03 18:34:31 INFO ZippedPartitionsRDD2: Removing RDD 605 from persistence list
15/06/03 18:34:31 INFO BlockManager: Removing block rdd_599_0
15/06/03 18:34:31 INFO MemoryStore: Block rdd_599_0 of size 1496 dropped from memory (free 558592135)
15/06/03 18:34:31 INFO BlockManager: Removing RDD 605
15/06/03 18:34:31 INFO BlockManager: Removing block rdd_605_0
15/06/03 18:34:31 INFO MemoryStore: Block rdd_605_0 of size 2976 dropped from memory (free 558595111)
15/06/03 18:34:31 INFO SparkContext: Starting job: reduce at VertexRDDImpl.scala:90
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 2 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 3 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 5 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 4 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 1 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 0 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 8 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 7 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 6 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 11 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 10 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 9 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 14 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 13 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 12 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 17 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 16 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 15 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 20 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 19 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 18 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 23 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 22 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 21 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 26 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 25 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 24 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 29 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 28 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 27 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 32 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 31 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 30 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 35 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 34 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 33 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 38 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 37 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 36 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 41 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 40 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 39 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 44 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 43 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 42 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 47 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 46 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 45 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 50 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 49 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 48 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 53 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 52 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 51 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 56 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 55 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 54 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 59 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 58 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 57 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 62 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 61 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 60 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 65 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 64 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 63 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 68 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 67 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 66 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 71 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 70 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 69 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 74 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 73 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 72 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 77 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 76 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 75 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 80 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 79 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 78 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 83 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 82 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 81 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 86 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 85 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 84 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 89 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 88 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 87 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 92 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 91 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 90 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 95 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 94 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 93 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 98 is 145 bytes
15/06/03 18:34:31 INFO BlockManager: Removing broadcast 124
15/06/03 18:34:31 INFO BlockManager: Removing block broadcast_124
15/06/03 18:34:31 INFO MemoryStore: Block broadcast_124 of size 21744 dropped from memory (free 558616855)
15/06/03 18:34:31 INFO BlockManager: Removing block broadcast_124_piece0
15/06/03 18:34:31 INFO MemoryStore: Block broadcast_124_piece0 of size 10114 dropped from memory (free 558626969)
15/06/03 18:34:31 INFO BlockManagerInfo: Removed broadcast_124_piece0 on localhost:60836 in memory (size: 9.9 KB, free: 533.1 MB)
15/06/03 18:34:31 INFO BlockManagerMaster: Updated info of block broadcast_124_piece0
15/06/03 18:34:31 INFO ContextCleaner: Cleaned broadcast 124
15/06/03 18:34:31 INFO BlockManager: Removing broadcast 132
15/06/03 18:34:31 INFO BlockManager: Removing block broadcast_132
15/06/03 18:34:31 INFO MemoryStore: Block broadcast_132 of size 22800 dropped from memory (free 558649769)
15/06/03 18:34:31 INFO BlockManager: Removing block broadcast_132_piece0
15/06/03 18:34:31 INFO MemoryStore: Block broadcast_132_piece0 of size 10447 dropped from memory (free 558660216)
15/06/03 18:34:31 INFO BlockManagerInfo: Removed broadcast_132_piece0 on localhost:60836 in memory (size: 10.2 KB, free: 533.1 MB)
15/06/03 18:34:31 INFO BlockManagerMaster: Updated info of block broadcast_132_piece0
15/06/03 18:34:31 INFO ContextCleaner: Cleaned broadcast 132
15/06/03 18:34:31 INFO BlockManager: Removing broadcast 131
15/06/03 18:34:31 INFO BlockManager: Removing block broadcast_131_piece0
15/06/03 18:34:31 INFO MemoryStore: Block broadcast_131_piece0 of size 7226 dropped from memory (free 558667442)
15/06/03 18:34:31 INFO BlockManagerInfo: Removed broadcast_131_piece0 on localhost:60836 in memory (size: 7.1 KB, free: 533.1 MB)
15/06/03 18:34:31 INFO BlockManagerMaster: Updated info of block broadcast_131_piece0
15/06/03 18:34:31 INFO BlockManager: Removing block broadcast_131
15/06/03 18:34:31 INFO MemoryStore: Block broadcast_131 of size 13920 dropped from memory (free 558681362)
15/06/03 18:34:31 INFO ContextCleaner: Cleaned broadcast 131
15/06/03 18:34:31 INFO BlockManager: Removing broadcast 130
15/06/03 18:34:31 INFO BlockManager: Removing block broadcast_130
15/06/03 18:34:31 INFO MemoryStore: Block broadcast_130 of size 22168 dropped from memory (free 558703530)
15/06/03 18:34:31 INFO BlockManager: Removing block broadcast_130_piece0
15/06/03 18:34:31 INFO MemoryStore: Block broadcast_130_piece0 of size 10164 dropped from memory (free 558713694)
15/06/03 18:34:31 INFO BlockManagerInfo: Removed broadcast_130_piece0 on localhost:60836 in memory (size: 9.9 KB, free: 533.1 MB)
15/06/03 18:34:31 INFO BlockManagerMaster: Updated info of block broadcast_130_piece0
15/06/03 18:34:31 INFO ContextCleaner: Cleaned broadcast 130
15/06/03 18:34:31 INFO BlockManager: Removing broadcast 128
15/06/03 18:34:31 INFO BlockManager: Removing block broadcast_128_piece0
15/06/03 18:34:31 INFO MemoryStore: Block broadcast_128_piece0 of size 10219 dropped from memory (free 558723913)
15/06/03 18:34:31 INFO BlockManagerInfo: Removed broadcast_128_piece0 on localhost:60836 in memory (size: 10.0 KB, free: 533.1 MB)
15/06/03 18:34:31 INFO BlockManagerMaster: Updated info of block broadcast_128_piece0
15/06/03 18:34:31 INFO BlockManager: Removing block broadcast_128
15/06/03 18:34:31 INFO MemoryStore: Block broadcast_128 of size 22272 dropped from memory (free 558746185)
15/06/03 18:34:31 INFO ContextCleaner: Cleaned broadcast 128
15/06/03 18:34:31 INFO BlockManager: Removing broadcast 127
15/06/03 18:34:31 INFO BlockManager: Removing block broadcast_127
15/06/03 18:34:31 INFO MemoryStore: Block broadcast_127 of size 13688 dropped from memory (free 558759873)
15/06/03 18:34:31 INFO BlockManager: Removing block broadcast_127_piece0
15/06/03 18:34:31 INFO MemoryStore: Block broadcast_127_piece0 of size 7128 dropped from memory (free 558767001)
15/06/03 18:34:31 INFO BlockManagerInfo: Removed broadcast_127_piece0 on localhost:60836 in memory (size: 7.0 KB, free: 533.2 MB)
15/06/03 18:34:31 INFO BlockManagerMaster: Updated info of block broadcast_127_piece0
15/06/03 18:34:31 INFO ContextCleaner: Cleaned broadcast 127
15/06/03 18:34:31 INFO BlockManager: Removing broadcast 126
15/06/03 18:34:31 INFO BlockManager: Removing block broadcast_126_piece0
15/06/03 18:34:31 INFO MemoryStore: Block broadcast_126_piece0 of size 9978 dropped from memory (free 558776979)
15/06/03 18:34:31 INFO BlockManagerInfo: Removed broadcast_126_piece0 on localhost:60836 in memory (size: 9.7 KB, free: 533.2 MB)
15/06/03 18:34:31 INFO BlockManagerMaster: Updated info of block broadcast_126_piece0
15/06/03 18:34:31 INFO BlockManager: Removing block broadcast_126
15/06/03 18:34:31 INFO MemoryStore: Block broadcast_126 of size 21640 dropped from memory (free 558798619)
15/06/03 18:34:31 INFO ContextCleaner: Cleaned broadcast 126
15/06/03 18:34:31 INFO BlockManager: Removing broadcast 125
15/06/03 18:34:31 INFO BlockManager: Removing block broadcast_125
15/06/03 18:34:31 INFO MemoryStore: Block broadcast_125 of size 21984 dropped from memory (free 558820603)
15/06/03 18:34:31 INFO BlockManager: Removing block broadcast_125_piece0
15/06/03 18:34:31 INFO MemoryStore: Block broadcast_125_piece0 of size 10151 dropped from memory (free 558830754)
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 97 is 145 bytes
15/06/03 18:34:31 INFO BlockManagerInfo: Removed broadcast_125_piece0 on localhost:60836 in memory (size: 9.9 KB, free: 533.2 MB)
15/06/03 18:34:31 INFO BlockManagerMaster: Updated info of block broadcast_125_piece0
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 96 is 145 bytes
15/06/03 18:34:31 INFO ContextCleaner: Cleaned broadcast 125
15/06/03 18:34:31 INFO DAGScheduler: Registering RDD 641 (mapPartitions at VertexRDDImpl.scala:218)
15/06/03 18:34:31 INFO BlockManager: Removing broadcast 129
15/06/03 18:34:31 INFO BlockManager: Removing block broadcast_129_piece0
15/06/03 18:34:31 INFO MemoryStore: Block broadcast_129_piece0 of size 10272 dropped from memory (free 558841026)
15/06/03 18:34:31 INFO BlockManagerInfo: Removed broadcast_129_piece0 on localhost:60836 in memory (size: 10.0 KB, free: 533.2 MB)
15/06/03 18:34:31 INFO BlockManagerMaster: Updated info of block broadcast_129_piece0
15/06/03 18:34:31 INFO BlockManager: Removing block broadcast_129
15/06/03 18:34:31 INFO MemoryStore: Block broadcast_129 of size 22512 dropped from memory (free 558863538)
15/06/03 18:34:31 INFO ContextCleaner: Cleaned broadcast 129
15/06/03 18:34:31 INFO DAGScheduler: Registering RDD 645 (mapPartitions at VertexRDDImpl.scala:222)
15/06/03 18:34:31 INFO DAGScheduler: Registering RDD 649 (mapPartitions at GraphImpl.scala:235)
15/06/03 18:34:31 INFO DAGScheduler: Got job 33 (reduce at VertexRDDImpl.scala:90) with 1 output partitions (allowLocal=false)
15/06/03 18:34:31 INFO DAGScheduler: Final stage: Stage 21417(reduce at VertexRDDImpl.scala:90)
15/06/03 18:34:31 INFO DAGScheduler: Parents of final stage: List(Stage 21350, Stage 21380, Stage 21372, Stage 21402, Stage 21352, Stage 21374, Stage 21404, Stage 21396, Stage 21376, Stage 21368, Stage 21398, Stage 21370, Stage 21392, Stage 21414, Stage 21364, Stage 21394, Stage 21386, Stage 21416, Stage 21366, Stage 21358, Stage 21388, Stage 21410, Stage 21360, Stage 21390, Stage 21382, Stage 21412, Stage 21362, Stage 21354, Stage 21384, Stage 21406, Stage 21356, Stage 21378, Stage 21408, Stage 21349, Stage 21400)
15/06/03 18:34:31 INFO DAGScheduler: Missing parents: List(Stage 21416)
15/06/03 18:34:31 INFO DAGScheduler: Submitting Stage 21315 (ReplicatedVertexView.updateVertices - shippedVerts true false (broadcast) MapPartitionsRDD[641] at mapPartitions at VertexRDDImpl.scala:218), which has no missing parents
15/06/03 18:34:31 INFO MemoryStore: ensureFreeSpace(23040) called with curMem=289614, maxMem=559153152
15/06/03 18:34:31 INFO MemoryStore: Block broadcast_133 stored as values in memory (estimated size 22.5 KB, free 533.0 MB)
15/06/03 18:34:31 INFO MemoryStore: ensureFreeSpace(10474) called with curMem=312654, maxMem=559153152
15/06/03 18:34:31 INFO MemoryStore: Block broadcast_133_piece0 stored as bytes in memory (estimated size 10.2 KB, free 532.9 MB)
15/06/03 18:34:31 INFO BlockManagerInfo: Added broadcast_133_piece0 in memory on localhost:60836 (size: 10.2 KB, free: 533.2 MB)
15/06/03 18:34:31 INFO BlockManagerMaster: Updated info of block broadcast_133_piece0
15/06/03 18:34:31 INFO SparkContext: Created broadcast 133 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:31 INFO DAGScheduler: Submitting 1 missing tasks from Stage 21315 (ReplicatedVertexView.updateVertices - shippedVerts true false (broadcast) MapPartitionsRDD[641] at mapPartitions at VertexRDDImpl.scala:218)
15/06/03 18:34:31 INFO TaskSchedulerImpl: Adding task set 21315.0 with 1 tasks
15/06/03 18:34:31 INFO DAGScheduler: Submitting Stage 21415 (ReplicatedVertexView.withActiveSet - shippedActives (broadcast) MapPartitionsRDD[645] at mapPartitions at VertexRDDImpl.scala:222), which has no missing parents
15/06/03 18:34:31 INFO TaskSetManager: Starting task 0.0 in stage 21315.0 (TID 132, localhost, PROCESS_LOCAL, 4886 bytes)
15/06/03 18:34:31 INFO Executor: Running task 0.0 in stage 21315.0 (TID 132)
15/06/03 18:34:31 INFO MemoryStore: ensureFreeSpace(22696) called with curMem=323128, maxMem=559153152
15/06/03 18:34:31 INFO MemoryStore: Block broadcast_134 stored as values in memory (estimated size 22.2 KB, free 532.9 MB)
15/06/03 18:34:31 INFO MemoryStore: ensureFreeSpace(10312) called with curMem=345824, maxMem=559153152
15/06/03 18:34:31 INFO MemoryStore: Block broadcast_134_piece0 stored as bytes in memory (estimated size 10.1 KB, free 532.9 MB)
15/06/03 18:34:31 INFO BlockManagerInfo: Added broadcast_134_piece0 in memory on localhost:60836 (size: 10.1 KB, free: 533.2 MB)
15/06/03 18:34:31 INFO BlockManagerMaster: Updated info of block broadcast_134_piece0
15/06/03 18:34:31 INFO SparkContext: Created broadcast 134 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:31 INFO DAGScheduler: Submitting 1 missing tasks from Stage 21415 (ReplicatedVertexView.withActiveSet - shippedActives (broadcast) MapPartitionsRDD[645] at mapPartitions at VertexRDDImpl.scala:222)
15/06/03 18:34:31 INFO TaskSchedulerImpl: Adding task set 21415.0 with 1 tasks
15/06/03 18:34:31 INFO BlockManager: Found block rdd_618_0 locally
15/06/03 18:34:31 INFO CacheManager: Partition rdd_637_0 not found, computing it
15/06/03 18:34:31 INFO BlockManager: Found block rdd_618_0 locally
15/06/03 18:34:31 INFO CacheManager: Partition rdd_635_0 not found, computing it
15/06/03 18:34:31 INFO BlockManager: Found block rdd_618_0 locally
15/06/03 18:34:31 INFO BlockManager: Found block rdd_632_0 locally
15/06/03 18:34:31 INFO MemoryStore: ensureFreeSpace(1544) called with curMem=356136, maxMem=559153152
15/06/03 18:34:31 INFO MemoryStore: Block rdd_635_0 stored as values in memory (estimated size 1544.0 B, free 532.9 MB)
15/06/03 18:34:31 INFO BlockManagerInfo: Added rdd_635_0 in memory on localhost:60836 (size: 1544.0 B, free: 533.2 MB)
15/06/03 18:34:31 INFO BlockManagerMaster: Updated info of block rdd_635_0
15/06/03 18:34:31 INFO MemoryStore: ensureFreeSpace(1496) called with curMem=357680, maxMem=559153152
15/06/03 18:34:31 INFO MemoryStore: Block rdd_637_0 stored as values in memory (estimated size 1496.0 B, free 532.9 MB)
15/06/03 18:34:31 INFO BlockManagerInfo: Added rdd_637_0 in memory on localhost:60836 (size: 1496.0 B, free: 533.2 MB)
15/06/03 18:34:31 INFO BlockManagerMaster: Updated info of block rdd_637_0
15/06/03 18:34:31 INFO Executor: Finished task 0.0 in stage 21315.0 (TID 132). 2627 bytes result sent to driver
15/06/03 18:34:31 INFO TaskSetManager: Starting task 0.0 in stage 21415.0 (TID 133, localhost, PROCESS_LOCAL, 4822 bytes)
15/06/03 18:34:31 INFO TaskSetManager: Finished task 0.0 in stage 21315.0 (TID 132) in 30 ms on localhost (1/1)
15/06/03 18:34:31 INFO Executor: Running task 0.0 in stage 21415.0 (TID 133)
15/06/03 18:34:31 INFO TaskSchedulerImpl: Removed TaskSet 21315.0, whose tasks have all completed, from pool
15/06/03 18:34:31 INFO DAGScheduler: Stage 21315 (mapPartitions at VertexRDDImpl.scala:218) finished in 0.030 s
15/06/03 18:34:31 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:31 INFO DAGScheduler: running: Set(Stage 21415)
15/06/03 18:34:31 INFO DAGScheduler: waiting: Set(Stage 21416, Stage 21417)
15/06/03 18:34:31 INFO DAGScheduler: failed: Set()
15/06/03 18:34:31 INFO DAGScheduler: Missing parents for Stage 21416: List(Stage 21415)
15/06/03 18:34:31 INFO DAGScheduler: Missing parents for Stage 21417: List(Stage 21416)
15/06/03 18:34:31 INFO BlockManager: Found block rdd_635_0 locally
15/06/03 18:34:31 INFO Executor: Finished task 0.0 in stage 21415.0 (TID 133). 2003 bytes result sent to driver
15/06/03 18:34:31 INFO TaskSetManager: Finished task 0.0 in stage 21415.0 (TID 133) in 14 ms on localhost (1/1)
15/06/03 18:34:31 INFO TaskSchedulerImpl: Removed TaskSet 21415.0, whose tasks have all completed, from pool
15/06/03 18:34:31 INFO DAGScheduler: Stage 21415 (mapPartitions at VertexRDDImpl.scala:222) finished in 0.035 s
15/06/03 18:34:31 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:31 INFO DAGScheduler: running: Set()
15/06/03 18:34:31 INFO DAGScheduler: waiting: Set(Stage 21416, Stage 21417)
15/06/03 18:34:31 INFO DAGScheduler: failed: Set()
15/06/03 18:34:31 INFO DAGScheduler: Missing parents for Stage 21416: List()
15/06/03 18:34:31 INFO DAGScheduler: Missing parents for Stage 21417: List(Stage 21416)
15/06/03 18:34:31 INFO DAGScheduler: Submitting Stage 21416 (GraphImpl.aggregateMessages - preAgg MapPartitionsRDD[649] at mapPartitions at GraphImpl.scala:235), which is now runnable
15/06/03 18:34:31 INFO MemoryStore: ensureFreeSpace(14144) called with curMem=359176, maxMem=559153152
15/06/03 18:34:31 INFO MemoryStore: Block broadcast_135 stored as values in memory (estimated size 13.8 KB, free 532.9 MB)
15/06/03 18:34:31 INFO MemoryStore: ensureFreeSpace(7279) called with curMem=373320, maxMem=559153152
15/06/03 18:34:31 INFO MemoryStore: Block broadcast_135_piece0 stored as bytes in memory (estimated size 7.1 KB, free 532.9 MB)
15/06/03 18:34:31 INFO BlockManagerInfo: Added broadcast_135_piece0 in memory on localhost:60836 (size: 7.1 KB, free: 533.1 MB)
15/06/03 18:34:31 INFO BlockManagerMaster: Updated info of block broadcast_135_piece0
15/06/03 18:34:31 INFO SparkContext: Created broadcast 135 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:31 INFO DAGScheduler: Submitting 1 missing tasks from Stage 21416 (GraphImpl.aggregateMessages - preAgg MapPartitionsRDD[649] at mapPartitions at GraphImpl.scala:235)
15/06/03 18:34:31 INFO TaskSchedulerImpl: Adding task set 21416.0 with 1 tasks
15/06/03 18:34:31 INFO TaskSetManager: Starting task 0.0 in stage 21416.0 (TID 134, localhost, PROCESS_LOCAL, 3247 bytes)
15/06/03 18:34:31 INFO Executor: Running task 0.0 in stage 21416.0 (TID 134)
15/06/03 18:34:31 INFO CacheManager: Partition rdd_643_0 not found, computing it
15/06/03 18:34:31 INFO BlockManager: Found block rdd_624_0 locally
15/06/03 18:34:31 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:31 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:31 INFO MemoryStore: ensureFreeSpace(2976) called with curMem=380599, maxMem=559153152
15/06/03 18:34:31 INFO MemoryStore: Block rdd_643_0 stored as values in memory (estimated size 2.9 KB, free 532.9 MB)
15/06/03 18:34:31 INFO BlockManagerInfo: Added rdd_643_0 in memory on localhost:60836 (size: 2.9 KB, free: 533.1 MB)
15/06/03 18:34:31 INFO BlockManagerMaster: Updated info of block rdd_643_0
15/06/03 18:34:31 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:31 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:31 INFO Executor: Finished task 0.0 in stage 21416.0 (TID 134). 2788 bytes result sent to driver
15/06/03 18:34:31 INFO TaskSetManager: Finished task 0.0 in stage 21416.0 (TID 134) in 15 ms on localhost (1/1)
15/06/03 18:34:31 INFO TaskSchedulerImpl: Removed TaskSet 21416.0, whose tasks have all completed, from pool
15/06/03 18:34:31 INFO DAGScheduler: Stage 21416 (mapPartitions at GraphImpl.scala:235) finished in 0.015 s
15/06/03 18:34:31 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:31 INFO DAGScheduler: running: Set()
15/06/03 18:34:31 INFO DAGScheduler: waiting: Set(Stage 21417)
15/06/03 18:34:31 INFO DAGScheduler: failed: Set()
15/06/03 18:34:31 INFO DAGScheduler: Missing parents for Stage 21417: List()
15/06/03 18:34:31 INFO DAGScheduler: Submitting Stage 21417 (MapPartitionsRDD[653] at map at VertexRDDImpl.scala:90), which is now runnable
15/06/03 18:34:31 INFO MemoryStore: ensureFreeSpace(23328) called with curMem=383575, maxMem=559153152
15/06/03 18:34:31 INFO MemoryStore: Block broadcast_136 stored as values in memory (estimated size 22.8 KB, free 532.9 MB)
15/06/03 18:34:31 INFO MemoryStore: ensureFreeSpace(10687) called with curMem=406903, maxMem=559153152
15/06/03 18:34:31 INFO MemoryStore: Block broadcast_136_piece0 stored as bytes in memory (estimated size 10.4 KB, free 532.9 MB)
15/06/03 18:34:31 INFO BlockManagerInfo: Added broadcast_136_piece0 in memory on localhost:60836 (size: 10.4 KB, free: 533.1 MB)
15/06/03 18:34:31 INFO BlockManagerMaster: Updated info of block broadcast_136_piece0
15/06/03 18:34:31 INFO SparkContext: Created broadcast 136 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:31 INFO DAGScheduler: Submitting 1 missing tasks from Stage 21417 (MapPartitionsRDD[653] at map at VertexRDDImpl.scala:90)
15/06/03 18:34:31 INFO TaskSchedulerImpl: Adding task set 21417.0 with 1 tasks
15/06/03 18:34:31 INFO TaskSetManager: Starting task 0.0 in stage 21417.0 (TID 135, localhost, PROCESS_LOCAL, 4906 bytes)
15/06/03 18:34:31 INFO Executor: Running task 0.0 in stage 21417.0 (TID 135)
15/06/03 18:34:31 INFO CacheManager: Partition rdd_651_0 not found, computing it
15/06/03 18:34:31 INFO BlockManager: Found block rdd_637_0 locally
15/06/03 18:34:31 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:31 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:31 INFO MemoryStore: ensureFreeSpace(1664) called with curMem=417590, maxMem=559153152
15/06/03 18:34:31 INFO MemoryStore: Block rdd_651_0 stored as values in memory (estimated size 1664.0 B, free 532.9 MB)
15/06/03 18:34:31 INFO BlockManagerInfo: Added rdd_651_0 in memory on localhost:60836 (size: 1664.0 B, free: 533.1 MB)
15/06/03 18:34:31 INFO BlockManagerMaster: Updated info of block rdd_651_0
15/06/03 18:34:31 INFO Executor: Finished task 0.0 in stage 21417.0 (TID 135). 2693 bytes result sent to driver
15/06/03 18:34:31 INFO TaskSetManager: Finished task 0.0 in stage 21417.0 (TID 135) in 9 ms on localhost (1/1)
15/06/03 18:34:31 INFO TaskSchedulerImpl: Removed TaskSet 21417.0, whose tasks have all completed, from pool
15/06/03 18:34:31 INFO DAGScheduler: Stage 21417 (reduce at VertexRDDImpl.scala:90) finished in 0.009 s
15/06/03 18:34:31 INFO DAGScheduler: Job 33 finished: reduce at VertexRDDImpl.scala:90, took 0.344010 s
15/06/03 18:34:31 INFO Pregel: Pregel finished iteration 31
15/06/03 18:34:31 INFO ZippedPartitionsRDD2: Removing RDD 632 from persistence list
15/06/03 18:34:31 INFO BlockManager: Removing RDD 632
15/06/03 18:34:31 INFO BlockManager: Removing block rdd_632_0
15/06/03 18:34:31 INFO ZippedPartitionsRDD2: Removing RDD 635 from persistence list
15/06/03 18:34:31 INFO MemoryStore: Block rdd_632_0 of size 1664 dropped from memory (free 558735562)
15/06/03 18:34:31 INFO BlockManager: Removing RDD 635
15/06/03 18:34:31 INFO BlockManager: Removing block rdd_635_0
15/06/03 18:34:31 INFO MemoryStore: Block rdd_635_0 of size 1544 dropped from memory (free 558737106)
15/06/03 18:34:31 INFO ZippedPartitionsRDD2: Removing RDD 618 from persistence list
15/06/03 18:34:31 INFO BlockManager: Removing RDD 618
15/06/03 18:34:31 INFO BlockManager: Removing block rdd_618_0
15/06/03 18:34:31 INFO MemoryStore: Block rdd_618_0 of size 1496 dropped from memory (free 558738602)
15/06/03 18:34:31 INFO ZippedPartitionsRDD2: Removing RDD 624 from persistence list
15/06/03 18:34:31 INFO BlockManager: Removing RDD 624
15/06/03 18:34:31 INFO BlockManager: Removing block rdd_624_0
15/06/03 18:34:31 INFO MemoryStore: Block rdd_624_0 of size 2976 dropped from memory (free 558741578)
15/06/03 18:34:31 INFO SparkContext: Starting job: reduce at VertexRDDImpl.scala:90
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 2 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 3 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 5 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 4 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 1 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 0 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 8 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 7 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 6 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 11 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 10 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 9 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 14 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 13 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 12 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 17 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 16 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 15 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 20 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 19 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 18 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 23 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 22 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 21 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 26 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 25 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 24 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 29 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 28 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 27 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 32 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 31 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 30 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 35 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 34 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 33 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 38 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 37 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 36 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 41 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 40 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 39 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 44 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 43 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 42 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 47 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 46 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 45 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 50 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 49 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 48 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 53 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 52 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 51 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 56 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 55 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 54 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 59 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 58 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 57 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 62 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 61 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 60 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 65 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 64 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 63 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 68 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 67 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 66 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 71 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 70 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 69 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 74 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 73 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 72 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 77 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 76 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 75 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 80 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 79 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 78 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 83 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 82 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 81 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 86 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 85 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 84 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 89 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 88 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 87 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 92 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 91 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 90 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 95 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 94 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 93 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 98 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 97 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 96 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 101 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 100 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 99 is 145 bytes
15/06/03 18:34:31 INFO DAGScheduler: Registering RDD 660 (mapPartitions at VertexRDDImpl.scala:218)
15/06/03 18:34:31 INFO DAGScheduler: Registering RDD 664 (mapPartitions at VertexRDDImpl.scala:222)
15/06/03 18:34:31 INFO DAGScheduler: Registering RDD 668 (mapPartitions at GraphImpl.scala:235)
15/06/03 18:34:31 INFO DAGScheduler: Got job 34 (reduce at VertexRDDImpl.scala:90) with 1 output partitions (allowLocal=false)
15/06/03 18:34:31 INFO DAGScheduler: Final stage: Stage 23307(reduce at VertexRDDImpl.scala:90)
15/06/03 18:34:31 INFO DAGScheduler: Parents of final stage: List(Stage 23252, Stage 23282, Stage 23274, Stage 23304, Stage 23254, Stage 23246, Stage 23276, Stage 23298, Stage 23248, Stage 23278, Stage 23270, Stage 23300, Stage 23292, Stage 23250, Stage 23242, Stage 23272, Stage 23294, Stage 23244, Stage 23266, Stage 23296, Stage 23237, Stage 23288, Stage 23238, Stage 23268, Stage 23260, Stage 23290, Stage 23240, Stage 23262, Stage 23284, Stage 23306, Stage 23264, Stage 23256, Stage 23286, Stage 23258, Stage 23280, Stage 23302)
15/06/03 18:34:31 INFO DAGScheduler: Missing parents: List(Stage 23306)
15/06/03 18:34:31 INFO DAGScheduler: Submitting Stage 23202 (ReplicatedVertexView.updateVertices - shippedVerts true false (broadcast) MapPartitionsRDD[660] at mapPartitions at VertexRDDImpl.scala:218), which has no missing parents
15/06/03 18:34:31 INFO MemoryStore: ensureFreeSpace(23568) called with curMem=411574, maxMem=559153152
15/06/03 18:34:31 INFO MemoryStore: Block broadcast_137 stored as values in memory (estimated size 23.0 KB, free 532.8 MB)
15/06/03 18:34:31 INFO MemoryStore: ensureFreeSpace(10748) called with curMem=435142, maxMem=559153152
15/06/03 18:34:31 INFO MemoryStore: Block broadcast_137_piece0 stored as bytes in memory (estimated size 10.5 KB, free 532.8 MB)
15/06/03 18:34:31 INFO BlockManagerInfo: Added broadcast_137_piece0 in memory on localhost:60836 (size: 10.5 KB, free: 533.1 MB)
15/06/03 18:34:31 INFO BlockManagerMaster: Updated info of block broadcast_137_piece0
15/06/03 18:34:31 INFO SparkContext: Created broadcast 137 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:31 INFO DAGScheduler: Submitting 1 missing tasks from Stage 23202 (ReplicatedVertexView.updateVertices - shippedVerts true false (broadcast) MapPartitionsRDD[660] at mapPartitions at VertexRDDImpl.scala:218)
15/06/03 18:34:31 INFO TaskSchedulerImpl: Adding task set 23202.0 with 1 tasks
15/06/03 18:34:31 INFO DAGScheduler: Submitting Stage 23305 (ReplicatedVertexView.withActiveSet - shippedActives (broadcast) MapPartitionsRDD[664] at mapPartitions at VertexRDDImpl.scala:222), which has no missing parents
15/06/03 18:34:31 INFO TaskSetManager: Starting task 0.0 in stage 23202.0 (TID 136, localhost, PROCESS_LOCAL, 4991 bytes)
15/06/03 18:34:31 INFO Executor: Running task 0.0 in stage 23202.0 (TID 136)
15/06/03 18:34:31 INFO MemoryStore: ensureFreeSpace(23224) called with curMem=445890, maxMem=559153152
15/06/03 18:34:31 INFO MemoryStore: Block broadcast_138 stored as values in memory (estimated size 22.7 KB, free 532.8 MB)
15/06/03 18:34:31 INFO MemoryStore: ensureFreeSpace(10636) called with curMem=469114, maxMem=559153152
15/06/03 18:34:31 INFO MemoryStore: Block broadcast_138_piece0 stored as bytes in memory (estimated size 10.4 KB, free 532.8 MB)
15/06/03 18:34:31 INFO BlockManagerInfo: Added broadcast_138_piece0 in memory on localhost:60836 (size: 10.4 KB, free: 533.1 MB)
15/06/03 18:34:31 INFO BlockManagerMaster: Updated info of block broadcast_138_piece0
15/06/03 18:34:31 INFO SparkContext: Created broadcast 138 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:31 INFO DAGScheduler: Submitting 1 missing tasks from Stage 23305 (ReplicatedVertexView.withActiveSet - shippedActives (broadcast) MapPartitionsRDD[664] at mapPartitions at VertexRDDImpl.scala:222)
15/06/03 18:34:31 INFO TaskSchedulerImpl: Adding task set 23305.0 with 1 tasks
15/06/03 18:34:31 INFO BlockManager: Found block rdd_637_0 locally
15/06/03 18:34:31 INFO CacheManager: Partition rdd_656_0 not found, computing it
15/06/03 18:34:31 INFO BlockManager: Found block rdd_637_0 locally
15/06/03 18:34:31 INFO CacheManager: Partition rdd_654_0 not found, computing it
15/06/03 18:34:31 INFO BlockManager: Found block rdd_637_0 locally
15/06/03 18:34:31 INFO BlockManager: Found block rdd_651_0 locally
15/06/03 18:34:31 INFO MemoryStore: ensureFreeSpace(1544) called with curMem=479750, maxMem=559153152
15/06/03 18:34:31 INFO MemoryStore: Block rdd_654_0 stored as values in memory (estimated size 1544.0 B, free 532.8 MB)
15/06/03 18:34:31 INFO BlockManagerInfo: Added rdd_654_0 in memory on localhost:60836 (size: 1544.0 B, free: 533.1 MB)
15/06/03 18:34:31 INFO BlockManagerMaster: Updated info of block rdd_654_0
15/06/03 18:34:31 INFO MemoryStore: ensureFreeSpace(1496) called with curMem=481294, maxMem=559153152
15/06/03 18:34:31 INFO MemoryStore: Block rdd_656_0 stored as values in memory (estimated size 1496.0 B, free 532.8 MB)
15/06/03 18:34:31 INFO BlockManagerInfo: Added rdd_656_0 in memory on localhost:60836 (size: 1496.0 B, free: 533.1 MB)
15/06/03 18:34:31 INFO BlockManagerMaster: Updated info of block rdd_656_0
15/06/03 18:34:31 INFO Executor: Finished task 0.0 in stage 23202.0 (TID 136). 2627 bytes result sent to driver
15/06/03 18:34:31 INFO TaskSetManager: Starting task 0.0 in stage 23305.0 (TID 137, localhost, PROCESS_LOCAL, 4927 bytes)
15/06/03 18:34:31 INFO TaskSetManager: Finished task 0.0 in stage 23202.0 (TID 136) in 26 ms on localhost (1/1)
15/06/03 18:34:31 INFO Executor: Running task 0.0 in stage 23305.0 (TID 137)
15/06/03 18:34:31 INFO TaskSchedulerImpl: Removed TaskSet 23202.0, whose tasks have all completed, from pool
15/06/03 18:34:31 INFO DAGScheduler: Stage 23202 (mapPartitions at VertexRDDImpl.scala:218) finished in 0.026 s
15/06/03 18:34:31 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:31 INFO DAGScheduler: running: Set(Stage 23305)
15/06/03 18:34:31 INFO DAGScheduler: waiting: Set(Stage 23306, Stage 23307)
15/06/03 18:34:31 INFO DAGScheduler: failed: Set()
15/06/03 18:34:31 INFO DAGScheduler: Missing parents for Stage 23306: List(Stage 23305)
15/06/03 18:34:31 INFO DAGScheduler: Missing parents for Stage 23307: List(Stage 23306)
15/06/03 18:34:31 INFO BlockManager: Found block rdd_654_0 locally
15/06/03 18:34:31 INFO Executor: Finished task 0.0 in stage 23305.0 (TID 137). 2003 bytes result sent to driver
15/06/03 18:34:31 INFO TaskSetManager: Finished task 0.0 in stage 23305.0 (TID 137) in 14 ms on localhost (1/1)
15/06/03 18:34:31 INFO TaskSchedulerImpl: Removed TaskSet 23305.0, whose tasks have all completed, from pool
15/06/03 18:34:31 INFO DAGScheduler: Stage 23305 (mapPartitions at VertexRDDImpl.scala:222) finished in 0.031 s
15/06/03 18:34:31 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:31 INFO DAGScheduler: running: Set()
15/06/03 18:34:31 INFO DAGScheduler: waiting: Set(Stage 23306, Stage 23307)
15/06/03 18:34:31 INFO DAGScheduler: failed: Set()
15/06/03 18:34:31 INFO DAGScheduler: Missing parents for Stage 23306: List()
15/06/03 18:34:31 INFO DAGScheduler: Missing parents for Stage 23307: List(Stage 23306)
15/06/03 18:34:31 INFO DAGScheduler: Submitting Stage 23306 (GraphImpl.aggregateMessages - preAgg MapPartitionsRDD[668] at mapPartitions at GraphImpl.scala:235), which is now runnable
15/06/03 18:34:31 INFO MemoryStore: ensureFreeSpace(14376) called with curMem=482790, maxMem=559153152
15/06/03 18:34:31 INFO MemoryStore: Block broadcast_139 stored as values in memory (estimated size 14.0 KB, free 532.8 MB)
15/06/03 18:34:31 INFO MemoryStore: ensureFreeSpace(7379) called with curMem=497166, maxMem=559153152
15/06/03 18:34:31 INFO MemoryStore: Block broadcast_139_piece0 stored as bytes in memory (estimated size 7.2 KB, free 532.8 MB)
15/06/03 18:34:31 INFO BlockManagerInfo: Added broadcast_139_piece0 in memory on localhost:60836 (size: 7.2 KB, free: 533.1 MB)
15/06/03 18:34:31 INFO BlockManagerMaster: Updated info of block broadcast_139_piece0
15/06/03 18:34:31 INFO SparkContext: Created broadcast 139 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:31 INFO DAGScheduler: Submitting 1 missing tasks from Stage 23306 (GraphImpl.aggregateMessages - preAgg MapPartitionsRDD[668] at mapPartitions at GraphImpl.scala:235)
15/06/03 18:34:31 INFO TaskSchedulerImpl: Adding task set 23306.0 with 1 tasks
15/06/03 18:34:31 INFO TaskSetManager: Starting task 0.0 in stage 23306.0 (TID 138, localhost, PROCESS_LOCAL, 3288 bytes)
15/06/03 18:34:31 INFO Executor: Running task 0.0 in stage 23306.0 (TID 138)
15/06/03 18:34:31 INFO CacheManager: Partition rdd_662_0 not found, computing it
15/06/03 18:34:31 INFO BlockManager: Found block rdd_643_0 locally
15/06/03 18:34:31 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:31 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:31 INFO MemoryStore: ensureFreeSpace(2976) called with curMem=504545, maxMem=559153152
15/06/03 18:34:31 INFO MemoryStore: Block rdd_662_0 stored as values in memory (estimated size 2.9 KB, free 532.8 MB)
15/06/03 18:34:31 INFO BlockManagerInfo: Added rdd_662_0 in memory on localhost:60836 (size: 2.9 KB, free: 533.1 MB)
15/06/03 18:34:31 INFO BlockManagerMaster: Updated info of block rdd_662_0
15/06/03 18:34:31 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:31 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
15/06/03 18:34:31 INFO Executor: Finished task 0.0 in stage 23306.0 (TID 138). 2788 bytes result sent to driver
15/06/03 18:34:31 INFO TaskSetManager: Finished task 0.0 in stage 23306.0 (TID 138) in 18 ms on localhost (1/1)
15/06/03 18:34:31 INFO TaskSchedulerImpl: Removed TaskSet 23306.0, whose tasks have all completed, from pool
15/06/03 18:34:31 INFO DAGScheduler: Stage 23306 (mapPartitions at GraphImpl.scala:235) finished in 0.018 s
15/06/03 18:34:31 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:31 INFO DAGScheduler: running: Set()
15/06/03 18:34:31 INFO DAGScheduler: waiting: Set(Stage 23307)
15/06/03 18:34:31 INFO DAGScheduler: failed: Set()
15/06/03 18:34:31 INFO DAGScheduler: Missing parents for Stage 23307: List()
15/06/03 18:34:31 INFO DAGScheduler: Submitting Stage 23307 (MapPartitionsRDD[672] at map at VertexRDDImpl.scala:90), which is now runnable
15/06/03 18:34:31 INFO MemoryStore: ensureFreeSpace(23856) called with curMem=507521, maxMem=559153152
15/06/03 18:34:31 INFO MemoryStore: Block broadcast_140 stored as values in memory (estimated size 23.3 KB, free 532.7 MB)
15/06/03 18:34:31 INFO MemoryStore: ensureFreeSpace(10836) called with curMem=531377, maxMem=559153152
15/06/03 18:34:31 INFO MemoryStore: Block broadcast_140_piece0 stored as bytes in memory (estimated size 10.6 KB, free 532.7 MB)
15/06/03 18:34:31 INFO BlockManagerInfo: Added broadcast_140_piece0 in memory on localhost:60836 (size: 10.6 KB, free: 533.1 MB)
15/06/03 18:34:31 INFO BlockManagerMaster: Updated info of block broadcast_140_piece0
15/06/03 18:34:31 INFO SparkContext: Created broadcast 140 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:31 INFO DAGScheduler: Submitting 1 missing tasks from Stage 23307 (MapPartitionsRDD[672] at map at VertexRDDImpl.scala:90)
15/06/03 18:34:31 INFO TaskSchedulerImpl: Adding task set 23307.0 with 1 tasks
15/06/03 18:34:31 INFO TaskSetManager: Starting task 0.0 in stage 23307.0 (TID 139, localhost, PROCESS_LOCAL, 5011 bytes)
15/06/03 18:34:31 INFO Executor: Running task 0.0 in stage 23307.0 (TID 139)
15/06/03 18:34:31 INFO CacheManager: Partition rdd_670_0 not found, computing it
15/06/03 18:34:31 INFO BlockManager: Found block rdd_656_0 locally
15/06/03 18:34:31 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:31 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:31 INFO MemoryStore: ensureFreeSpace(1664) called with curMem=542213, maxMem=559153152
15/06/03 18:34:31 INFO MemoryStore: Block rdd_670_0 stored as values in memory (estimated size 1664.0 B, free 532.7 MB)
15/06/03 18:34:31 INFO BlockManagerInfo: Added rdd_670_0 in memory on localhost:60836 (size: 1664.0 B, free: 533.1 MB)
15/06/03 18:34:31 INFO BlockManagerMaster: Updated info of block rdd_670_0
15/06/03 18:34:31 INFO Executor: Finished task 0.0 in stage 23307.0 (TID 139). 2693 bytes result sent to driver
15/06/03 18:34:31 INFO TaskSetManager: Finished task 0.0 in stage 23307.0 (TID 139) in 9 ms on localhost (1/1)
15/06/03 18:34:31 INFO TaskSchedulerImpl: Removed TaskSet 23307.0, whose tasks have all completed, from pool
15/06/03 18:34:31 INFO DAGScheduler: Stage 23307 (reduce at VertexRDDImpl.scala:90) finished in 0.009 s
15/06/03 18:34:31 INFO DAGScheduler: Job 34 finished: reduce at VertexRDDImpl.scala:90, took 0.285799 s
15/06/03 18:34:31 INFO Pregel: Pregel finished iteration 32
15/06/03 18:34:31 INFO ZippedPartitionsRDD2: Removing RDD 651 from persistence list
15/06/03 18:34:31 INFO BlockManager: Removing RDD 651
15/06/03 18:34:31 INFO ZippedPartitionsRDD2: Removing RDD 654 from persistence list
15/06/03 18:34:31 INFO BlockManager: Removing block rdd_651_0
15/06/03 18:34:31 INFO MemoryStore: Block rdd_651_0 of size 1664 dropped from memory (free 558610939)
15/06/03 18:34:31 INFO BlockManager: Removing RDD 654
15/06/03 18:34:31 INFO ZippedPartitionsRDD2: Removing RDD 637 from persistence list
15/06/03 18:34:31 INFO BlockManager: Removing block rdd_654_0
15/06/03 18:34:31 INFO MemoryStore: Block rdd_654_0 of size 1544 dropped from memory (free 558612483)
15/06/03 18:34:31 INFO BlockManager: Removing RDD 637
15/06/03 18:34:31 INFO ZippedPartitionsRDD2: Removing RDD 643 from persistence list
15/06/03 18:34:31 INFO BlockManager: Removing block rdd_637_0
15/06/03 18:34:31 INFO MemoryStore: Block rdd_637_0 of size 1496 dropped from memory (free 558613979)
15/06/03 18:34:31 INFO BlockManager: Removing RDD 643
15/06/03 18:34:31 INFO BlockManager: Removing block rdd_643_0
15/06/03 18:34:31 INFO MemoryStore: Block rdd_643_0 of size 2976 dropped from memory (free 558616955)
15/06/03 18:34:31 INFO SparkContext: Starting job: reduce at VertexRDDImpl.scala:90
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 2 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 3 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 5 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 4 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 1 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 0 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 8 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 7 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 6 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 11 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 10 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 9 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 14 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 13 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 12 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 17 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 16 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 15 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 20 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 19 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 18 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 23 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 22 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 21 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 26 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 25 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 24 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 29 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 28 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 27 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 32 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 31 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 30 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 35 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 34 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 33 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 38 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 37 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 36 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 41 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 40 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 39 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 44 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 43 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 42 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 47 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 46 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 45 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 50 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 49 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 48 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 53 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 52 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 51 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 56 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 55 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 54 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 59 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 58 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 57 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 62 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 61 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 60 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 65 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 64 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 63 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 68 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 67 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 66 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 71 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 70 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 69 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 74 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 73 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 72 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 77 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 76 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 75 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 80 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 79 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 78 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 83 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 82 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 81 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 86 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 85 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 84 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 89 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 88 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 87 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 92 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 91 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 90 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 95 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 94 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 93 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 98 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 97 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 96 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 101 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 100 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 99 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 104 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 103 is 145 bytes
15/06/03 18:34:31 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 102 is 145 bytes
15/06/03 18:34:31 INFO DAGScheduler: Registering RDD 679 (mapPartitions at VertexRDDImpl.scala:218)
15/06/03 18:34:31 INFO DAGScheduler: Registering RDD 683 (mapPartitions at VertexRDDImpl.scala:222)
15/06/03 18:34:31 INFO DAGScheduler: Registering RDD 687 (mapPartitions at GraphImpl.scala:235)
15/06/03 18:34:31 INFO DAGScheduler: Got job 35 (reduce at VertexRDDImpl.scala:90) with 1 output partitions (allowLocal=false)
15/06/03 18:34:31 INFO DAGScheduler: Final stage: Stage 25305(reduce at VertexRDDImpl.scala:90)
15/06/03 18:34:31 INFO DAGScheduler: Parents of final stage: List(Stage 25256, Stage 25286, Stage 25278, Stage 25300, Stage 25258, Stage 25250, Stage 25280, Stage 25302, Stage 25252, Stage 25274, Stage 25304, Stage 25296, Stage 25246, Stage 25276, Stage 25268, Stage 25298, Stage 25248, Stage 25240, Stage 25270, Stage 25292, Stage 25242, Stage 25272, Stage 25264, Stage 25294, Stage 25244, Stage 25236, Stage 25266, Stage 25288, Stage 25238, Stage 25260, Stage 25290, Stage 25282, Stage 25262, Stage 25254, Stage 25233, Stage 25284, Stage 25234)
15/06/03 18:34:31 INFO DAGScheduler: Missing parents: List(Stage 25304)
15/06/03 18:34:31 INFO DAGScheduler: Submitting Stage 25197 (ReplicatedVertexView.updateVertices - shippedVerts true false (broadcast) MapPartitionsRDD[679] at mapPartitions at VertexRDDImpl.scala:218), which has no missing parents
15/06/03 18:34:31 INFO MemoryStore: ensureFreeSpace(24096) called with curMem=536197, maxMem=559153152
15/06/03 18:34:31 INFO MemoryStore: Block broadcast_141 stored as values in memory (estimated size 23.5 KB, free 532.7 MB)
15/06/03 18:34:31 INFO MemoryStore: ensureFreeSpace(10882) called with curMem=560293, maxMem=559153152
15/06/03 18:34:31 INFO MemoryStore: Block broadcast_141_piece0 stored as bytes in memory (estimated size 10.6 KB, free 532.7 MB)
15/06/03 18:34:31 INFO BlockManagerInfo: Added broadcast_141_piece0 in memory on localhost:60836 (size: 10.6 KB, free: 533.1 MB)
15/06/03 18:34:31 INFO BlockManagerMaster: Updated info of block broadcast_141_piece0
15/06/03 18:34:31 INFO SparkContext: Created broadcast 141 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:31 INFO DAGScheduler: Submitting 1 missing tasks from Stage 25197 (ReplicatedVertexView.updateVertices - shippedVerts true false (broadcast) MapPartitionsRDD[679] at mapPartitions at VertexRDDImpl.scala:218)
15/06/03 18:34:31 INFO TaskSchedulerImpl: Adding task set 25197.0 with 1 tasks
15/06/03 18:34:31 INFO DAGScheduler: Submitting Stage 25303 (ReplicatedVertexView.withActiveSet - shippedActives (broadcast) MapPartitionsRDD[683] at mapPartitions at VertexRDDImpl.scala:222), which has no missing parents
15/06/03 18:34:31 INFO TaskSetManager: Starting task 0.0 in stage 25197.0 (TID 140, localhost, PROCESS_LOCAL, 5096 bytes)
15/06/03 18:34:31 INFO Executor: Running task 0.0 in stage 25197.0 (TID 140)
15/06/03 18:34:31 INFO MemoryStore: ensureFreeSpace(23752) called with curMem=571175, maxMem=559153152
15/06/03 18:34:31 INFO MemoryStore: Block broadcast_142 stored as values in memory (estimated size 23.2 KB, free 532.7 MB)
15/06/03 18:34:31 INFO MemoryStore: ensureFreeSpace(10718) called with curMem=594927, maxMem=559153152
15/06/03 18:34:31 INFO MemoryStore: Block broadcast_142_piece0 stored as bytes in memory (estimated size 10.5 KB, free 532.7 MB)
15/06/03 18:34:31 INFO BlockManagerInfo: Added broadcast_142_piece0 in memory on localhost:60836 (size: 10.5 KB, free: 533.1 MB)
15/06/03 18:34:31 INFO BlockManagerMaster: Updated info of block broadcast_142_piece0
15/06/03 18:34:31 INFO SparkContext: Created broadcast 142 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:31 INFO DAGScheduler: Submitting 1 missing tasks from Stage 25303 (ReplicatedVertexView.withActiveSet - shippedActives (broadcast) MapPartitionsRDD[683] at mapPartitions at VertexRDDImpl.scala:222)
15/06/03 18:34:31 INFO BlockManager: Found block rdd_656_0 locally
15/06/03 18:34:31 INFO TaskSchedulerImpl: Adding task set 25303.0 with 1 tasks
15/06/03 18:34:31 INFO CacheManager: Partition rdd_675_0 not found, computing it
15/06/03 18:34:31 INFO BlockManager: Found block rdd_656_0 locally
15/06/03 18:34:31 INFO CacheManager: Partition rdd_673_0 not found, computing it
15/06/03 18:34:31 INFO BlockManager: Found block rdd_656_0 locally
15/06/03 18:34:31 INFO BlockManager: Found block rdd_670_0 locally
15/06/03 18:34:31 INFO MemoryStore: ensureFreeSpace(1544) called with curMem=605645, maxMem=559153152
15/06/03 18:34:31 INFO MemoryStore: Block rdd_673_0 stored as values in memory (estimated size 1544.0 B, free 532.7 MB)
15/06/03 18:34:31 INFO BlockManagerInfo: Added rdd_673_0 in memory on localhost:60836 (size: 1544.0 B, free: 533.1 MB)
15/06/03 18:34:31 INFO BlockManagerMaster: Updated info of block rdd_673_0
15/06/03 18:34:31 INFO MemoryStore: ensureFreeSpace(1496) called with curMem=607189, maxMem=559153152
15/06/03 18:34:31 INFO MemoryStore: Block rdd_675_0 stored as values in memory (estimated size 1496.0 B, free 532.7 MB)
15/06/03 18:34:31 INFO BlockManagerInfo: Added rdd_675_0 in memory on localhost:60836 (size: 1496.0 B, free: 533.1 MB)
15/06/03 18:34:31 INFO BlockManagerMaster: Updated info of block rdd_675_0
15/06/03 18:34:31 INFO Executor: Finished task 0.0 in stage 25197.0 (TID 140). 2627 bytes result sent to driver
15/06/03 18:34:31 INFO TaskSetManager: Starting task 0.0 in stage 25303.0 (TID 141, localhost, PROCESS_LOCAL, 5032 bytes)
15/06/03 18:34:31 INFO TaskSetManager: Finished task 0.0 in stage 25197.0 (TID 140) in 25 ms on localhost (1/1)
15/06/03 18:34:31 INFO TaskSchedulerImpl: Removed TaskSet 25197.0, whose tasks have all completed, from pool
15/06/03 18:34:31 INFO Executor: Running task 0.0 in stage 25303.0 (TID 141)
15/06/03 18:34:31 INFO DAGScheduler: Stage 25197 (mapPartitions at VertexRDDImpl.scala:218) finished in 0.025 s
15/06/03 18:34:31 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:31 INFO DAGScheduler: running: Set(Stage 25303)
15/06/03 18:34:31 INFO DAGScheduler: waiting: Set(Stage 25304, Stage 25305)
15/06/03 18:34:31 INFO DAGScheduler: failed: Set()
15/06/03 18:34:31 INFO DAGScheduler: Missing parents for Stage 25304: List(Stage 25303)
15/06/03 18:34:31 INFO DAGScheduler: Missing parents for Stage 25305: List(Stage 25304)
15/06/03 18:34:31 INFO BlockManager: Found block rdd_673_0 locally
15/06/03 18:34:31 INFO Executor: Finished task 0.0 in stage 25303.0 (TID 141). 2003 bytes result sent to driver
15/06/03 18:34:31 INFO TaskSetManager: Finished task 0.0 in stage 25303.0 (TID 141) in 18 ms on localhost (1/1)
15/06/03 18:34:31 INFO TaskSchedulerImpl: Removed TaskSet 25303.0, whose tasks have all completed, from pool
15/06/03 18:34:31 INFO DAGScheduler: Stage 25303 (mapPartitions at VertexRDDImpl.scala:222) finished in 0.037 s
15/06/03 18:34:31 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:31 INFO DAGScheduler: running: Set()
15/06/03 18:34:31 INFO DAGScheduler: waiting: Set(Stage 25304, Stage 25305)
15/06/03 18:34:31 INFO DAGScheduler: failed: Set()
15/06/03 18:34:31 INFO DAGScheduler: Missing parents for Stage 25304: List()
15/06/03 18:34:31 INFO DAGScheduler: Missing parents for Stage 25305: List(Stage 25304)
15/06/03 18:34:31 INFO DAGScheduler: Submitting Stage 25304 (GraphImpl.aggregateMessages - preAgg MapPartitionsRDD[687] at mapPartitions at GraphImpl.scala:235), which is now runnable
15/06/03 18:34:31 INFO MemoryStore: ensureFreeSpace(14600) called with curMem=608685, maxMem=559153152
15/06/03 18:34:31 INFO MemoryStore: Block broadcast_143 stored as values in memory (estimated size 14.3 KB, free 532.7 MB)
15/06/03 18:34:32 INFO MemoryStore: ensureFreeSpace(7545) called with curMem=623285, maxMem=559153152
15/06/03 18:34:32 INFO MemoryStore: Block broadcast_143_piece0 stored as bytes in memory (estimated size 7.4 KB, free 532.6 MB)
15/06/03 18:34:32 INFO BlockManagerInfo: Added broadcast_143_piece0 in memory on localhost:60836 (size: 7.4 KB, free: 533.1 MB)
15/06/03 18:34:32 INFO BlockManagerMaster: Updated info of block broadcast_143_piece0
15/06/03 18:34:32 INFO SparkContext: Created broadcast 143 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:32 INFO DAGScheduler: Submitting 1 missing tasks from Stage 25304 (GraphImpl.aggregateMessages - preAgg MapPartitionsRDD[687] at mapPartitions at GraphImpl.scala:235)
15/06/03 18:34:32 INFO TaskSchedulerImpl: Adding task set 25304.0 with 1 tasks
15/06/03 18:34:32 INFO TaskSetManager: Starting task 0.0 in stage 25304.0 (TID 142, localhost, PROCESS_LOCAL, 3329 bytes)
15/06/03 18:34:32 INFO Executor: Running task 0.0 in stage 25304.0 (TID 142)
15/06/03 18:34:32 INFO CacheManager: Partition rdd_681_0 not found, computing it
15/06/03 18:34:32 INFO BlockManager: Found block rdd_662_0 locally
15/06/03 18:34:32 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:32 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:32 INFO MemoryStore: ensureFreeSpace(2976) called with curMem=630830, maxMem=559153152
15/06/03 18:34:32 INFO MemoryStore: Block rdd_681_0 stored as values in memory (estimated size 2.9 KB, free 532.6 MB)
15/06/03 18:34:32 INFO BlockManagerInfo: Added rdd_681_0 in memory on localhost:60836 (size: 2.9 KB, free: 533.1 MB)
15/06/03 18:34:32 INFO BlockManagerMaster: Updated info of block rdd_681_0
15/06/03 18:34:32 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:32 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:32 INFO Executor: Finished task 0.0 in stage 25304.0 (TID 142). 2788 bytes result sent to driver
15/06/03 18:34:32 INFO TaskSetManager: Finished task 0.0 in stage 25304.0 (TID 142) in 15 ms on localhost (1/1)
15/06/03 18:34:32 INFO TaskSchedulerImpl: Removed TaskSet 25304.0, whose tasks have all completed, from pool
15/06/03 18:34:32 INFO DAGScheduler: Stage 25304 (mapPartitions at GraphImpl.scala:235) finished in 0.015 s
15/06/03 18:34:32 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:32 INFO DAGScheduler: running: Set()
15/06/03 18:34:32 INFO DAGScheduler: waiting: Set(Stage 25305)
15/06/03 18:34:32 INFO DAGScheduler: failed: Set()
15/06/03 18:34:32 INFO DAGScheduler: Missing parents for Stage 25305: List()
15/06/03 18:34:32 INFO DAGScheduler: Submitting Stage 25305 (MapPartitionsRDD[691] at map at VertexRDDImpl.scala:90), which is now runnable
15/06/03 18:34:32 INFO MemoryStore: ensureFreeSpace(24384) called with curMem=633806, maxMem=559153152
15/06/03 18:34:32 INFO MemoryStore: Block broadcast_144 stored as values in memory (estimated size 23.8 KB, free 532.6 MB)
15/06/03 18:34:32 INFO MemoryStore: ensureFreeSpace(11026) called with curMem=658190, maxMem=559153152
15/06/03 18:34:32 INFO MemoryStore: Block broadcast_144_piece0 stored as bytes in memory (estimated size 10.8 KB, free 532.6 MB)
15/06/03 18:34:32 INFO BlockManagerInfo: Added broadcast_144_piece0 in memory on localhost:60836 (size: 10.8 KB, free: 533.1 MB)
15/06/03 18:34:32 INFO BlockManagerMaster: Updated info of block broadcast_144_piece0
15/06/03 18:34:32 INFO SparkContext: Created broadcast 144 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:32 INFO DAGScheduler: Submitting 1 missing tasks from Stage 25305 (MapPartitionsRDD[691] at map at VertexRDDImpl.scala:90)
15/06/03 18:34:32 INFO TaskSchedulerImpl: Adding task set 25305.0 with 1 tasks
15/06/03 18:34:32 INFO TaskSetManager: Starting task 0.0 in stage 25305.0 (TID 143, localhost, PROCESS_LOCAL, 5116 bytes)
15/06/03 18:34:32 INFO Executor: Running task 0.0 in stage 25305.0 (TID 143)
15/06/03 18:34:32 INFO CacheManager: Partition rdd_689_0 not found, computing it
15/06/03 18:34:32 INFO BlockManager: Found block rdd_675_0 locally
15/06/03 18:34:32 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:32 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:32 INFO MemoryStore: ensureFreeSpace(1664) called with curMem=669216, maxMem=559153152
15/06/03 18:34:32 INFO MemoryStore: Block rdd_689_0 stored as values in memory (estimated size 1664.0 B, free 532.6 MB)
15/06/03 18:34:32 INFO BlockManagerInfo: Added rdd_689_0 in memory on localhost:60836 (size: 1664.0 B, free: 533.1 MB)
15/06/03 18:34:32 INFO BlockManagerMaster: Updated info of block rdd_689_0
15/06/03 18:34:32 INFO Executor: Finished task 0.0 in stage 25305.0 (TID 143). 2693 bytes result sent to driver
15/06/03 18:34:32 INFO TaskSetManager: Finished task 0.0 in stage 25305.0 (TID 143) in 9 ms on localhost (1/1)
15/06/03 18:34:32 INFO TaskSchedulerImpl: Removed TaskSet 25305.0, whose tasks have all completed, from pool
15/06/03 18:34:32 INFO DAGScheduler: Stage 25305 (reduce at VertexRDDImpl.scala:90) finished in 0.009 s
15/06/03 18:34:32 INFO DAGScheduler: Job 35 finished: reduce at VertexRDDImpl.scala:90, took 0.317524 s
15/06/03 18:34:32 INFO Pregel: Pregel finished iteration 33
15/06/03 18:34:32 INFO ZippedPartitionsRDD2: Removing RDD 670 from persistence list
15/06/03 18:34:32 INFO BlockManager: Removing RDD 670
15/06/03 18:34:32 INFO BlockManager: Removing block rdd_670_0
15/06/03 18:34:32 INFO MemoryStore: Block rdd_670_0 of size 1664 dropped from memory (free 558483936)
15/06/03 18:34:32 INFO ZippedPartitionsRDD2: Removing RDD 673 from persistence list
15/06/03 18:34:32 INFO BlockManager: Removing RDD 673
15/06/03 18:34:32 INFO ZippedPartitionsRDD2: Removing RDD 656 from persistence list
15/06/03 18:34:32 INFO BlockManager: Removing block rdd_673_0
15/06/03 18:34:32 INFO MemoryStore: Block rdd_673_0 of size 1544 dropped from memory (free 558485480)
15/06/03 18:34:32 INFO BlockManager: Removing RDD 656
15/06/03 18:34:32 INFO ZippedPartitionsRDD2: Removing RDD 662 from persistence list
15/06/03 18:34:32 INFO BlockManager: Removing block rdd_656_0
15/06/03 18:34:32 INFO MemoryStore: Block rdd_656_0 of size 1496 dropped from memory (free 558486976)
15/06/03 18:34:32 INFO BlockManager: Removing RDD 662
15/06/03 18:34:32 INFO BlockManager: Removing block rdd_662_0
15/06/03 18:34:32 INFO MemoryStore: Block rdd_662_0 of size 2976 dropped from memory (free 558489952)
15/06/03 18:34:32 INFO BlockManager: Removing broadcast 136
15/06/03 18:34:32 INFO BlockManager: Removing block broadcast_136
15/06/03 18:34:32 INFO MemoryStore: Block broadcast_136 of size 23328 dropped from memory (free 558513280)
15/06/03 18:34:32 INFO BlockManager: Removing block broadcast_136_piece0
15/06/03 18:34:32 INFO MemoryStore: Block broadcast_136_piece0 of size 10687 dropped from memory (free 558523967)
15/06/03 18:34:32 INFO BlockManagerInfo: Removed broadcast_136_piece0 on localhost:60836 in memory (size: 10.4 KB, free: 533.1 MB)
15/06/03 18:34:32 INFO BlockManagerMaster: Updated info of block broadcast_136_piece0
15/06/03 18:34:32 INFO ContextCleaner: Cleaned broadcast 136
15/06/03 18:34:32 INFO BlockManager: Removing broadcast 135
15/06/03 18:34:32 INFO BlockManager: Removing block broadcast_135
15/06/03 18:34:32 INFO MemoryStore: Block broadcast_135 of size 14144 dropped from memory (free 558538111)
15/06/03 18:34:32 INFO BlockManager: Removing block broadcast_135_piece0
15/06/03 18:34:32 INFO MemoryStore: Block broadcast_135_piece0 of size 7279 dropped from memory (free 558545390)
15/06/03 18:34:32 INFO BlockManagerInfo: Removed broadcast_135_piece0 on localhost:60836 in memory (size: 7.1 KB, free: 533.1 MB)
15/06/03 18:34:32 INFO BlockManagerMaster: Updated info of block broadcast_135_piece0
15/06/03 18:34:32 INFO ContextCleaner: Cleaned broadcast 135
15/06/03 18:34:32 INFO BlockManager: Removing broadcast 134
15/06/03 18:34:32 INFO BlockManager: Removing block broadcast_134
15/06/03 18:34:32 INFO MemoryStore: Block broadcast_134 of size 22696 dropped from memory (free 558568086)
15/06/03 18:34:32 INFO BlockManager: Removing block broadcast_134_piece0
15/06/03 18:34:32 INFO MemoryStore: Block broadcast_134_piece0 of size 10312 dropped from memory (free 558578398)
15/06/03 18:34:32 INFO BlockManagerInfo: Removed broadcast_134_piece0 on localhost:60836 in memory (size: 10.1 KB, free: 533.1 MB)
15/06/03 18:34:32 INFO BlockManagerMaster: Updated info of block broadcast_134_piece0
15/06/03 18:34:32 INFO ContextCleaner: Cleaned broadcast 134
15/06/03 18:34:32 INFO BlockManager: Removing broadcast 133
15/06/03 18:34:32 INFO BlockManager: Removing block broadcast_133_piece0
15/06/03 18:34:32 INFO MemoryStore: Block broadcast_133_piece0 of size 10474 dropped from memory (free 558588872)
15/06/03 18:34:32 INFO BlockManagerInfo: Removed broadcast_133_piece0 on localhost:60836 in memory (size: 10.2 KB, free: 533.1 MB)
15/06/03 18:34:32 INFO BlockManagerMaster: Updated info of block broadcast_133_piece0
15/06/03 18:34:32 INFO BlockManager: Removing block broadcast_133
15/06/03 18:34:32 INFO MemoryStore: Block broadcast_133 of size 23040 dropped from memory (free 558611912)
15/06/03 18:34:32 INFO ContextCleaner: Cleaned broadcast 133
15/06/03 18:34:32 INFO BlockManager: Removing broadcast 144
15/06/03 18:34:32 INFO BlockManager: Removing block broadcast_144
15/06/03 18:34:32 INFO MemoryStore: Block broadcast_144 of size 24384 dropped from memory (free 558636296)
15/06/03 18:34:32 INFO BlockManager: Removing block broadcast_144_piece0
15/06/03 18:34:32 INFO MemoryStore: Block broadcast_144_piece0 of size 11026 dropped from memory (free 558647322)
15/06/03 18:34:32 INFO BlockManagerInfo: Removed broadcast_144_piece0 on localhost:60836 in memory (size: 10.8 KB, free: 533.1 MB)
15/06/03 18:34:32 INFO BlockManagerMaster: Updated info of block broadcast_144_piece0
15/06/03 18:34:32 INFO ContextCleaner: Cleaned broadcast 144
15/06/03 18:34:32 INFO BlockManager: Removing broadcast 143
15/06/03 18:34:32 INFO BlockManager: Removing block broadcast_143_piece0
15/06/03 18:34:32 INFO MemoryStore: Block broadcast_143_piece0 of size 7545 dropped from memory (free 558654867)
15/06/03 18:34:32 INFO BlockManagerInfo: Removed broadcast_143_piece0 on localhost:60836 in memory (size: 7.4 KB, free: 533.1 MB)
15/06/03 18:34:32 INFO BlockManagerMaster: Updated info of block broadcast_143_piece0
15/06/03 18:34:32 INFO BlockManager: Removing block broadcast_143
15/06/03 18:34:32 INFO MemoryStore: Block broadcast_143 of size 14600 dropped from memory (free 558669467)
15/06/03 18:34:32 INFO ContextCleaner: Cleaned broadcast 143
15/06/03 18:34:32 INFO BlockManager: Removing broadcast 142
15/06/03 18:34:32 INFO BlockManager: Removing block broadcast_142_piece0
15/06/03 18:34:32 INFO MemoryStore: Block broadcast_142_piece0 of size 10718 dropped from memory (free 558680185)
15/06/03 18:34:32 INFO BlockManagerInfo: Removed broadcast_142_piece0 on localhost:60836 in memory (size: 10.5 KB, free: 533.1 MB)
15/06/03 18:34:32 INFO BlockManagerMaster: Updated info of block broadcast_142_piece0
15/06/03 18:34:32 INFO BlockManager: Removing block broadcast_142
15/06/03 18:34:32 INFO MemoryStore: Block broadcast_142 of size 23752 dropped from memory (free 558703937)
15/06/03 18:34:32 INFO ContextCleaner: Cleaned broadcast 142
15/06/03 18:34:32 INFO BlockManager: Removing broadcast 141
15/06/03 18:34:32 INFO BlockManager: Removing block broadcast_141
15/06/03 18:34:32 INFO MemoryStore: Block broadcast_141 of size 24096 dropped from memory (free 558728033)
15/06/03 18:34:32 INFO BlockManager: Removing block broadcast_141_piece0
15/06/03 18:34:32 INFO MemoryStore: Block broadcast_141_piece0 of size 10882 dropped from memory (free 558738915)
15/06/03 18:34:32 INFO BlockManagerInfo: Removed broadcast_141_piece0 on localhost:60836 in memory (size: 10.6 KB, free: 533.1 MB)
15/06/03 18:34:32 INFO BlockManagerMaster: Updated info of block broadcast_141_piece0
15/06/03 18:34:32 INFO ContextCleaner: Cleaned broadcast 141
15/06/03 18:34:32 INFO BlockManager: Removing broadcast 140
15/06/03 18:34:32 INFO BlockManager: Removing block broadcast_140_piece0
15/06/03 18:34:32 INFO MemoryStore: Block broadcast_140_piece0 of size 10836 dropped from memory (free 558749751)
15/06/03 18:34:32 INFO BlockManagerInfo: Removed broadcast_140_piece0 on localhost:60836 in memory (size: 10.6 KB, free: 533.2 MB)
15/06/03 18:34:32 INFO BlockManagerMaster: Updated info of block broadcast_140_piece0
15/06/03 18:34:32 INFO BlockManager: Removing block broadcast_140
15/06/03 18:34:32 INFO MemoryStore: Block broadcast_140 of size 23856 dropped from memory (free 558773607)
15/06/03 18:34:32 INFO ContextCleaner: Cleaned broadcast 140
15/06/03 18:34:32 INFO BlockManager: Removing broadcast 139
15/06/03 18:34:32 INFO BlockManager: Removing block broadcast_139_piece0
15/06/03 18:34:32 INFO MemoryStore: Block broadcast_139_piece0 of size 7379 dropped from memory (free 558780986)
15/06/03 18:34:32 INFO BlockManagerInfo: Removed broadcast_139_piece0 on localhost:60836 in memory (size: 7.2 KB, free: 533.2 MB)
15/06/03 18:34:32 INFO BlockManagerMaster: Updated info of block broadcast_139_piece0
15/06/03 18:34:32 INFO BlockManager: Removing block broadcast_139
15/06/03 18:34:32 INFO MemoryStore: Block broadcast_139 of size 14376 dropped from memory (free 558795362)
15/06/03 18:34:32 INFO ContextCleaner: Cleaned broadcast 139
15/06/03 18:34:32 INFO BlockManager: Removing broadcast 138
15/06/03 18:34:32 INFO BlockManager: Removing block broadcast_138_piece0
15/06/03 18:34:32 INFO MemoryStore: Block broadcast_138_piece0 of size 10636 dropped from memory (free 558805998)
15/06/03 18:34:32 INFO BlockManagerInfo: Removed broadcast_138_piece0 on localhost:60836 in memory (size: 10.4 KB, free: 533.2 MB)
15/06/03 18:34:32 INFO BlockManagerMaster: Updated info of block broadcast_138_piece0
15/06/03 18:34:32 INFO BlockManager: Removing block broadcast_138
15/06/03 18:34:32 INFO MemoryStore: Block broadcast_138 of size 23224 dropped from memory (free 558829222)
15/06/03 18:34:32 INFO ContextCleaner: Cleaned broadcast 138
15/06/03 18:34:32 INFO BlockManager: Removing broadcast 137
15/06/03 18:34:32 INFO BlockManager: Removing block broadcast_137_piece0
15/06/03 18:34:32 INFO MemoryStore: Block broadcast_137_piece0 of size 10748 dropped from memory (free 558839970)
15/06/03 18:34:32 INFO BlockManagerInfo: Removed broadcast_137_piece0 on localhost:60836 in memory (size: 10.5 KB, free: 533.2 MB)
15/06/03 18:34:32 INFO BlockManagerMaster: Updated info of block broadcast_137_piece0
15/06/03 18:34:32 INFO BlockManager: Removing block broadcast_137
15/06/03 18:34:32 INFO MemoryStore: Block broadcast_137 of size 23568 dropped from memory (free 558863538)
15/06/03 18:34:32 INFO ContextCleaner: Cleaned broadcast 137
15/06/03 18:34:32 INFO SparkContext: Starting job: reduce at VertexRDDImpl.scala:90
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 2 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 3 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 5 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 4 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 1 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 0 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 8 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 7 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 6 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 11 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 10 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 9 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 14 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 13 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 12 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 17 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 16 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 15 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 20 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 19 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 18 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 23 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 22 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 21 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 26 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 25 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 24 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 29 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 28 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 27 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 32 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 31 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 30 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 35 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 34 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 33 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 38 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 37 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 36 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 41 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 40 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 39 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 44 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 43 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 42 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 47 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 46 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 45 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 50 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 49 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 48 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 53 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 52 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 51 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 56 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 55 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 54 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 59 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 58 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 57 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 62 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 61 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 60 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 65 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 64 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 63 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 68 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 67 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 66 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 71 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 70 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 69 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 74 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 73 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 72 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 77 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 76 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 75 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 80 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 79 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 78 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 83 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 82 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 81 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 86 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 85 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 84 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 89 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 88 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 87 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 92 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 91 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 90 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 95 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 94 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 93 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 98 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 97 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 96 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 101 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 100 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 99 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 104 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 103 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 102 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 107 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 106 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 105 is 145 bytes
15/06/03 18:34:32 INFO DAGScheduler: Registering RDD 698 (mapPartitions at VertexRDDImpl.scala:218)
15/06/03 18:34:32 INFO DAGScheduler: Registering RDD 702 (mapPartitions at VertexRDDImpl.scala:222)
15/06/03 18:34:32 INFO DAGScheduler: Registering RDD 706 (mapPartitions at GraphImpl.scala:235)
15/06/03 18:34:32 INFO DAGScheduler: Got job 36 (reduce at VertexRDDImpl.scala:90) with 1 output partitions (allowLocal=false)
15/06/03 18:34:32 INFO DAGScheduler: Final stage: Stage 27414(reduce at VertexRDDImpl.scala:90)
15/06/03 18:34:32 INFO DAGScheduler: Parents of final stage: List(Stage 27361, Stage 27340, Stage 27383, Stage 27413, Stage 27405, Stage 27363, Stage 27355, Stage 27385, Stage 27407, Stage 27357, Stage 27379, Stage 27409, Stage 27401, Stage 27351, Stage 27381, Stage 27373, Stage 27403, Stage 27353, Stage 27345, Stage 27375, Stage 27397, Stage 27347, Stage 27377, Stage 27369, Stage 27399, Stage 27391, Stage 27349, Stage 27341, Stage 27371, Stage 27393, Stage 27343, Stage 27365, Stage 27395, Stage 27387, Stage 27367, Stage 27359, Stage 27389, Stage 27411)
15/06/03 18:34:32 INFO DAGScheduler: Missing parents: List(Stage 27413)
15/06/03 18:34:32 INFO DAGScheduler: Submitting Stage 27303 (ReplicatedVertexView.updateVertices - shippedVerts true false (broadcast) MapPartitionsRDD[698] at mapPartitions at VertexRDDImpl.scala:218), which has no missing parents
15/06/03 18:34:32 INFO MemoryStore: ensureFreeSpace(24624) called with curMem=289614, maxMem=559153152
15/06/03 18:34:32 INFO MemoryStore: Block broadcast_145 stored as values in memory (estimated size 24.0 KB, free 533.0 MB)
15/06/03 18:34:32 INFO MemoryStore: ensureFreeSpace(11079) called with curMem=314238, maxMem=559153152
15/06/03 18:34:32 INFO MemoryStore: Block broadcast_145_piece0 stored as bytes in memory (estimated size 10.8 KB, free 532.9 MB)
15/06/03 18:34:32 INFO BlockManagerInfo: Added broadcast_145_piece0 in memory on localhost:60836 (size: 10.8 KB, free: 533.2 MB)
15/06/03 18:34:32 INFO BlockManagerMaster: Updated info of block broadcast_145_piece0
15/06/03 18:34:32 INFO SparkContext: Created broadcast 145 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:32 INFO DAGScheduler: Submitting 1 missing tasks from Stage 27303 (ReplicatedVertexView.updateVertices - shippedVerts true false (broadcast) MapPartitionsRDD[698] at mapPartitions at VertexRDDImpl.scala:218)
15/06/03 18:34:32 INFO TaskSchedulerImpl: Adding task set 27303.0 with 1 tasks
15/06/03 18:34:32 INFO DAGScheduler: Submitting Stage 27412 (ReplicatedVertexView.withActiveSet - shippedActives (broadcast) MapPartitionsRDD[702] at mapPartitions at VertexRDDImpl.scala:222), which has no missing parents
15/06/03 18:34:32 INFO TaskSetManager: Starting task 0.0 in stage 27303.0 (TID 144, localhost, PROCESS_LOCAL, 5201 bytes)
15/06/03 18:34:32 INFO Executor: Running task 0.0 in stage 27303.0 (TID 144)
15/06/03 18:34:32 INFO MemoryStore: ensureFreeSpace(24280) called with curMem=325317, maxMem=559153152
15/06/03 18:34:32 INFO MemoryStore: Block broadcast_146 stored as values in memory (estimated size 23.7 KB, free 532.9 MB)
15/06/03 18:34:32 INFO MemoryStore: ensureFreeSpace(10965) called with curMem=349597, maxMem=559153152
15/06/03 18:34:32 INFO MemoryStore: Block broadcast_146_piece0 stored as bytes in memory (estimated size 10.7 KB, free 532.9 MB)
15/06/03 18:34:32 INFO BlockManagerInfo: Added broadcast_146_piece0 in memory on localhost:60836 (size: 10.7 KB, free: 533.2 MB)
15/06/03 18:34:32 INFO BlockManagerMaster: Updated info of block broadcast_146_piece0
15/06/03 18:34:32 INFO SparkContext: Created broadcast 146 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:32 INFO DAGScheduler: Submitting 1 missing tasks from Stage 27412 (ReplicatedVertexView.withActiveSet - shippedActives (broadcast) MapPartitionsRDD[702] at mapPartitions at VertexRDDImpl.scala:222)
15/06/03 18:34:32 INFO TaskSchedulerImpl: Adding task set 27412.0 with 1 tasks
15/06/03 18:34:32 INFO BlockManager: Found block rdd_675_0 locally
15/06/03 18:34:32 INFO CacheManager: Partition rdd_694_0 not found, computing it
15/06/03 18:34:32 INFO BlockManager: Found block rdd_675_0 locally
15/06/03 18:34:32 INFO CacheManager: Partition rdd_692_0 not found, computing it
15/06/03 18:34:32 INFO BlockManager: Found block rdd_675_0 locally
15/06/03 18:34:32 INFO BlockManager: Found block rdd_689_0 locally
15/06/03 18:34:32 INFO MemoryStore: ensureFreeSpace(1544) called with curMem=360562, maxMem=559153152
15/06/03 18:34:32 INFO MemoryStore: Block rdd_692_0 stored as values in memory (estimated size 1544.0 B, free 532.9 MB)
15/06/03 18:34:32 INFO BlockManagerInfo: Added rdd_692_0 in memory on localhost:60836 (size: 1544.0 B, free: 533.2 MB)
15/06/03 18:34:32 INFO BlockManagerMaster: Updated info of block rdd_692_0
15/06/03 18:34:32 INFO MemoryStore: ensureFreeSpace(1496) called with curMem=362106, maxMem=559153152
15/06/03 18:34:32 INFO MemoryStore: Block rdd_694_0 stored as values in memory (estimated size 1496.0 B, free 532.9 MB)
15/06/03 18:34:32 INFO BlockManagerInfo: Added rdd_694_0 in memory on localhost:60836 (size: 1496.0 B, free: 533.2 MB)
15/06/03 18:34:32 INFO BlockManagerMaster: Updated info of block rdd_694_0
15/06/03 18:34:32 INFO Executor: Finished task 0.0 in stage 27303.0 (TID 144). 2627 bytes result sent to driver
15/06/03 18:34:32 INFO TaskSetManager: Starting task 0.0 in stage 27412.0 (TID 145, localhost, PROCESS_LOCAL, 5137 bytes)
15/06/03 18:34:32 INFO Executor: Running task 0.0 in stage 27412.0 (TID 145)
15/06/03 18:34:32 INFO TaskSetManager: Finished task 0.0 in stage 27303.0 (TID 144) in 22 ms on localhost (1/1)
15/06/03 18:34:32 INFO TaskSchedulerImpl: Removed TaskSet 27303.0, whose tasks have all completed, from pool
15/06/03 18:34:32 INFO DAGScheduler: Stage 27303 (mapPartitions at VertexRDDImpl.scala:218) finished in 0.022 s
15/06/03 18:34:32 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:32 INFO DAGScheduler: running: Set(Stage 27412)
15/06/03 18:34:32 INFO DAGScheduler: waiting: Set(Stage 27413, Stage 27414)
15/06/03 18:34:32 INFO DAGScheduler: failed: Set()
15/06/03 18:34:32 INFO DAGScheduler: Missing parents for Stage 27413: List(Stage 27412)
15/06/03 18:34:32 INFO DAGScheduler: Missing parents for Stage 27414: List(Stage 27413)
15/06/03 18:34:32 INFO BlockManager: Found block rdd_692_0 locally
15/06/03 18:34:32 INFO Executor: Finished task 0.0 in stage 27412.0 (TID 145). 2003 bytes result sent to driver
15/06/03 18:34:32 INFO TaskSetManager: Finished task 0.0 in stage 27412.0 (TID 145) in 16 ms on localhost (1/1)
15/06/03 18:34:32 INFO TaskSchedulerImpl: Removed TaskSet 27412.0, whose tasks have all completed, from pool
15/06/03 18:34:32 INFO DAGScheduler: Stage 27412 (mapPartitions at VertexRDDImpl.scala:222) finished in 0.029 s
15/06/03 18:34:32 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:32 INFO DAGScheduler: running: Set()
15/06/03 18:34:32 INFO DAGScheduler: waiting: Set(Stage 27413, Stage 27414)
15/06/03 18:34:32 INFO DAGScheduler: failed: Set()
15/06/03 18:34:32 INFO DAGScheduler: Missing parents for Stage 27413: List()
15/06/03 18:34:32 INFO DAGScheduler: Missing parents for Stage 27414: List(Stage 27413)
15/06/03 18:34:32 INFO DAGScheduler: Submitting Stage 27413 (GraphImpl.aggregateMessages - preAgg MapPartitionsRDD[706] at mapPartitions at GraphImpl.scala:235), which is now runnable
15/06/03 18:34:32 INFO MemoryStore: ensureFreeSpace(14832) called with curMem=363602, maxMem=559153152
15/06/03 18:34:32 INFO MemoryStore: Block broadcast_147 stored as values in memory (estimated size 14.5 KB, free 532.9 MB)
15/06/03 18:34:32 INFO MemoryStore: ensureFreeSpace(7704) called with curMem=378434, maxMem=559153152
15/06/03 18:34:32 INFO MemoryStore: Block broadcast_147_piece0 stored as bytes in memory (estimated size 7.5 KB, free 532.9 MB)
15/06/03 18:34:32 INFO BlockManagerInfo: Added broadcast_147_piece0 in memory on localhost:60836 (size: 7.5 KB, free: 533.1 MB)
15/06/03 18:34:32 INFO BlockManagerMaster: Updated info of block broadcast_147_piece0
15/06/03 18:34:32 INFO SparkContext: Created broadcast 147 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:32 INFO DAGScheduler: Submitting 1 missing tasks from Stage 27413 (GraphImpl.aggregateMessages - preAgg MapPartitionsRDD[706] at mapPartitions at GraphImpl.scala:235)
15/06/03 18:34:32 INFO TaskSchedulerImpl: Adding task set 27413.0 with 1 tasks
15/06/03 18:34:32 INFO TaskSetManager: Starting task 0.0 in stage 27413.0 (TID 146, localhost, PROCESS_LOCAL, 3370 bytes)
15/06/03 18:34:32 INFO Executor: Running task 0.0 in stage 27413.0 (TID 146)
15/06/03 18:34:32 INFO CacheManager: Partition rdd_700_0 not found, computing it
15/06/03 18:34:32 INFO BlockManager: Found block rdd_681_0 locally
15/06/03 18:34:32 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:32 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:32 INFO MemoryStore: ensureFreeSpace(2976) called with curMem=386138, maxMem=559153152
15/06/03 18:34:32 INFO MemoryStore: Block rdd_700_0 stored as values in memory (estimated size 2.9 KB, free 532.9 MB)
15/06/03 18:34:32 INFO BlockManagerInfo: Added rdd_700_0 in memory on localhost:60836 (size: 2.9 KB, free: 533.1 MB)
15/06/03 18:34:32 INFO BlockManagerMaster: Updated info of block rdd_700_0
15/06/03 18:34:32 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:32 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
15/06/03 18:34:32 INFO Executor: Finished task 0.0 in stage 27413.0 (TID 146). 2788 bytes result sent to driver
15/06/03 18:34:32 INFO TaskSetManager: Finished task 0.0 in stage 27413.0 (TID 146) in 18 ms on localhost (1/1)
15/06/03 18:34:32 INFO TaskSchedulerImpl: Removed TaskSet 27413.0, whose tasks have all completed, from pool
15/06/03 18:34:32 INFO DAGScheduler: Stage 27413 (mapPartitions at GraphImpl.scala:235) finished in 0.019 s
15/06/03 18:34:32 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:32 INFO DAGScheduler: running: Set()
15/06/03 18:34:32 INFO DAGScheduler: waiting: Set(Stage 27414)
15/06/03 18:34:32 INFO DAGScheduler: failed: Set()
15/06/03 18:34:32 INFO DAGScheduler: Missing parents for Stage 27414: List()
15/06/03 18:34:32 INFO DAGScheduler: Submitting Stage 27414 (MapPartitionsRDD[710] at map at VertexRDDImpl.scala:90), which is now runnable
15/06/03 18:34:32 INFO MemoryStore: ensureFreeSpace(24904) called with curMem=389114, maxMem=559153152
15/06/03 18:34:32 INFO MemoryStore: Block broadcast_148 stored as values in memory (estimated size 24.3 KB, free 532.9 MB)
15/06/03 18:34:32 INFO MemoryStore: ensureFreeSpace(11191) called with curMem=414018, maxMem=559153152
15/06/03 18:34:32 INFO MemoryStore: Block broadcast_148_piece0 stored as bytes in memory (estimated size 10.9 KB, free 532.8 MB)
15/06/03 18:34:32 INFO BlockManagerInfo: Added broadcast_148_piece0 in memory on localhost:60836 (size: 10.9 KB, free: 533.1 MB)
15/06/03 18:34:32 INFO BlockManagerMaster: Updated info of block broadcast_148_piece0
15/06/03 18:34:32 INFO SparkContext: Created broadcast 148 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:32 INFO DAGScheduler: Submitting 1 missing tasks from Stage 27414 (MapPartitionsRDD[710] at map at VertexRDDImpl.scala:90)
15/06/03 18:34:32 INFO TaskSchedulerImpl: Adding task set 27414.0 with 1 tasks
15/06/03 18:34:32 INFO TaskSetManager: Starting task 0.0 in stage 27414.0 (TID 147, localhost, PROCESS_LOCAL, 5221 bytes)
15/06/03 18:34:32 INFO Executor: Running task 0.0 in stage 27414.0 (TID 147)
15/06/03 18:34:32 INFO CacheManager: Partition rdd_708_0 not found, computing it
15/06/03 18:34:32 INFO BlockManager: Found block rdd_694_0 locally
15/06/03 18:34:32 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:32 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:32 INFO MemoryStore: ensureFreeSpace(1664) called with curMem=425209, maxMem=559153152
15/06/03 18:34:32 INFO MemoryStore: Block rdd_708_0 stored as values in memory (estimated size 1664.0 B, free 532.8 MB)
15/06/03 18:34:32 INFO BlockManagerInfo: Added rdd_708_0 in memory on localhost:60836 (size: 1664.0 B, free: 533.1 MB)
15/06/03 18:34:32 INFO BlockManagerMaster: Updated info of block rdd_708_0
15/06/03 18:34:32 INFO Executor: Finished task 0.0 in stage 27414.0 (TID 147). 2693 bytes result sent to driver
15/06/03 18:34:32 INFO TaskSetManager: Finished task 0.0 in stage 27414.0 (TID 147) in 10 ms on localhost (1/1)
15/06/03 18:34:32 INFO TaskSchedulerImpl: Removed TaskSet 27414.0, whose tasks have all completed, from pool
15/06/03 18:34:32 INFO DAGScheduler: Stage 27414 (reduce at VertexRDDImpl.scala:90) finished in 0.010 s
15/06/03 18:34:32 INFO DAGScheduler: Job 36 finished: reduce at VertexRDDImpl.scala:90, took 0.338678 s
15/06/03 18:34:32 INFO Pregel: Pregel finished iteration 34
15/06/03 18:34:32 INFO ZippedPartitionsRDD2: Removing RDD 689 from persistence list
15/06/03 18:34:32 INFO BlockManager: Removing RDD 689
15/06/03 18:34:32 INFO BlockManager: Removing block rdd_689_0
15/06/03 18:34:32 INFO MemoryStore: Block rdd_689_0 of size 1664 dropped from memory (free 558727943)
15/06/03 18:34:32 INFO ZippedPartitionsRDD2: Removing RDD 692 from persistence list
15/06/03 18:34:32 INFO BlockManager: Removing RDD 692
15/06/03 18:34:32 INFO BlockManager: Removing block rdd_692_0
15/06/03 18:34:32 INFO MemoryStore: Block rdd_692_0 of size 1544 dropped from memory (free 558729487)
15/06/03 18:34:32 INFO ZippedPartitionsRDD2: Removing RDD 675 from persistence list
15/06/03 18:34:32 INFO BlockManager: Removing RDD 675
15/06/03 18:34:32 INFO BlockManager: Removing block rdd_675_0
15/06/03 18:34:32 INFO MemoryStore: Block rdd_675_0 of size 1496 dropped from memory (free 558730983)
15/06/03 18:34:32 INFO ZippedPartitionsRDD2: Removing RDD 681 from persistence list
15/06/03 18:34:32 INFO BlockManager: Removing RDD 681
15/06/03 18:34:32 INFO BlockManager: Removing block rdd_681_0
15/06/03 18:34:32 INFO MemoryStore: Block rdd_681_0 of size 2976 dropped from memory (free 558733959)
15/06/03 18:34:32 INFO SparkContext: Starting job: reduce at VertexRDDImpl.scala:90
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 2 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 3 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 5 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 4 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 1 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 0 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 8 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 7 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 6 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 11 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 10 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 9 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 14 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 13 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 12 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 17 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 16 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 15 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 20 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 19 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 18 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 23 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 22 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 21 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 26 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 25 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 24 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 29 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 28 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 27 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 32 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 31 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 30 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 35 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 34 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 33 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 38 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 37 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 36 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 41 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 40 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 39 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 44 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 43 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 42 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 47 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 46 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 45 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 50 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 49 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 48 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 53 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 52 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 51 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 56 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 55 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 54 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 59 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 58 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 57 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 62 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 61 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 60 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 65 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 64 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 63 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 68 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 67 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 66 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 71 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 70 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 69 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 74 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 73 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 72 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 77 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 76 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 75 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 80 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 79 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 78 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 83 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 82 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 81 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 86 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 85 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 84 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 89 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 88 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 87 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 92 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 91 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 90 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 95 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 94 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 93 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 98 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 97 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 96 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 101 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 100 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 99 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 104 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 103 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 102 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 107 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 106 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 105 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 110 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 109 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 108 is 145 bytes
15/06/03 18:34:32 INFO DAGScheduler: Registering RDD 717 (mapPartitions at VertexRDDImpl.scala:218)
15/06/03 18:34:32 INFO DAGScheduler: Registering RDD 721 (mapPartitions at VertexRDDImpl.scala:222)
15/06/03 18:34:32 INFO DAGScheduler: Registering RDD 725 (mapPartitions at GraphImpl.scala:235)
15/06/03 18:34:32 INFO DAGScheduler: Got job 37 (reduce at VertexRDDImpl.scala:90) with 1 output partitions (allowLocal=false)
15/06/03 18:34:32 INFO DAGScheduler: Final stage: Stage 29637(reduce at VertexRDDImpl.scala:90)
15/06/03 18:34:32 INFO DAGScheduler: Parents of final stage: List(Stage 29568, Stage 29590, Stage 29620, Stage 29561, Stage 29612, Stage 29562, Stage 29592, Stage 29584, Stage 29614, Stage 29636, Stage 29564, Stage 29586, Stage 29608, Stage 29630, Stage 29588, Stage 29580, Stage 29610, Stage 29602, Stage 29632, Stage 29582, Stage 29604, Stage 29634, Stage 29626, Stage 29576, Stage 29606, Stage 29598, Stage 29628, Stage 29578, Stage 29570, Stage 29600, Stage 29622, Stage 29572, Stage 29594, Stage 29624, Stage 29616, Stage 29574, Stage 29566, Stage 29596, Stage 29618)
15/06/03 18:34:32 INFO DAGScheduler: Missing parents: List(Stage 29636)
15/06/03 18:34:32 INFO DAGScheduler: Submitting Stage 29523 (ReplicatedVertexView.updateVertices - shippedVerts true false (broadcast) MapPartitionsRDD[717] at mapPartitions at VertexRDDImpl.scala:218), which has no missing parents
15/06/03 18:34:32 INFO MemoryStore: ensureFreeSpace(25144) called with curMem=419193, maxMem=559153152
15/06/03 18:34:32 INFO MemoryStore: Block broadcast_149 stored as values in memory (estimated size 24.6 KB, free 532.8 MB)
15/06/03 18:34:32 INFO MemoryStore: ensureFreeSpace(11230) called with curMem=444337, maxMem=559153152
15/06/03 18:34:32 INFO MemoryStore: Block broadcast_149_piece0 stored as bytes in memory (estimated size 11.0 KB, free 532.8 MB)
15/06/03 18:34:32 INFO BlockManagerInfo: Added broadcast_149_piece0 in memory on localhost:60836 (size: 11.0 KB, free: 533.1 MB)
15/06/03 18:34:32 INFO BlockManagerMaster: Updated info of block broadcast_149_piece0
15/06/03 18:34:32 INFO SparkContext: Created broadcast 149 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:32 INFO DAGScheduler: Submitting 1 missing tasks from Stage 29523 (ReplicatedVertexView.updateVertices - shippedVerts true false (broadcast) MapPartitionsRDD[717] at mapPartitions at VertexRDDImpl.scala:218)
15/06/03 18:34:32 INFO TaskSchedulerImpl: Adding task set 29523.0 with 1 tasks
15/06/03 18:34:32 INFO DAGScheduler: Submitting Stage 29635 (ReplicatedVertexView.withActiveSet - shippedActives (broadcast) MapPartitionsRDD[721] at mapPartitions at VertexRDDImpl.scala:222), which has no missing parents
15/06/03 18:34:32 INFO TaskSetManager: Starting task 0.0 in stage 29523.0 (TID 148, localhost, PROCESS_LOCAL, 5306 bytes)
15/06/03 18:34:32 INFO Executor: Running task 0.0 in stage 29523.0 (TID 148)
15/06/03 18:34:32 INFO MemoryStore: ensureFreeSpace(24808) called with curMem=455567, maxMem=559153152
15/06/03 18:34:32 INFO MemoryStore: Block broadcast_150 stored as values in memory (estimated size 24.2 KB, free 532.8 MB)
15/06/03 18:34:32 INFO MemoryStore: ensureFreeSpace(11091) called with curMem=480375, maxMem=559153152
15/06/03 18:34:32 INFO MemoryStore: Block broadcast_150_piece0 stored as bytes in memory (estimated size 10.8 KB, free 532.8 MB)
15/06/03 18:34:32 INFO BlockManagerInfo: Added broadcast_150_piece0 in memory on localhost:60836 (size: 10.8 KB, free: 533.1 MB)
15/06/03 18:34:32 INFO BlockManagerMaster: Updated info of block broadcast_150_piece0
15/06/03 18:34:32 INFO SparkContext: Created broadcast 150 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:32 INFO DAGScheduler: Submitting 1 missing tasks from Stage 29635 (ReplicatedVertexView.withActiveSet - shippedActives (broadcast) MapPartitionsRDD[721] at mapPartitions at VertexRDDImpl.scala:222)
15/06/03 18:34:32 INFO TaskSchedulerImpl: Adding task set 29635.0 with 1 tasks
15/06/03 18:34:32 INFO BlockManager: Found block rdd_694_0 locally
15/06/03 18:34:32 INFO CacheManager: Partition rdd_713_0 not found, computing it
15/06/03 18:34:32 INFO BlockManager: Found block rdd_694_0 locally
15/06/03 18:34:32 INFO CacheManager: Partition rdd_711_0 not found, computing it
15/06/03 18:34:32 INFO BlockManager: Found block rdd_694_0 locally
15/06/03 18:34:32 INFO BlockManager: Found block rdd_708_0 locally
15/06/03 18:34:32 INFO MemoryStore: ensureFreeSpace(1544) called with curMem=491466, maxMem=559153152
15/06/03 18:34:32 INFO MemoryStore: Block rdd_711_0 stored as values in memory (estimated size 1544.0 B, free 532.8 MB)
15/06/03 18:34:32 INFO BlockManagerInfo: Added rdd_711_0 in memory on localhost:60836 (size: 1544.0 B, free: 533.1 MB)
15/06/03 18:34:32 INFO BlockManagerMaster: Updated info of block rdd_711_0
15/06/03 18:34:32 INFO MemoryStore: ensureFreeSpace(1496) called with curMem=493010, maxMem=559153152
15/06/03 18:34:32 INFO MemoryStore: Block rdd_713_0 stored as values in memory (estimated size 1496.0 B, free 532.8 MB)
15/06/03 18:34:32 INFO BlockManagerInfo: Added rdd_713_0 in memory on localhost:60836 (size: 1496.0 B, free: 533.1 MB)
15/06/03 18:34:32 INFO BlockManagerMaster: Updated info of block rdd_713_0
15/06/03 18:34:32 INFO Executor: Finished task 0.0 in stage 29523.0 (TID 148). 2627 bytes result sent to driver
15/06/03 18:34:32 INFO TaskSetManager: Starting task 0.0 in stage 29635.0 (TID 149, localhost, PROCESS_LOCAL, 5242 bytes)
15/06/03 18:34:32 INFO TaskSetManager: Finished task 0.0 in stage 29523.0 (TID 148) in 28 ms on localhost (1/1)
15/06/03 18:34:32 INFO Executor: Running task 0.0 in stage 29635.0 (TID 149)
15/06/03 18:34:32 INFO TaskSchedulerImpl: Removed TaskSet 29523.0, whose tasks have all completed, from pool
15/06/03 18:34:32 INFO DAGScheduler: Stage 29523 (mapPartitions at VertexRDDImpl.scala:218) finished in 0.029 s
15/06/03 18:34:32 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:32 INFO DAGScheduler: running: Set(Stage 29635)
15/06/03 18:34:32 INFO DAGScheduler: waiting: Set(Stage 29637, Stage 29636)
15/06/03 18:34:32 INFO DAGScheduler: failed: Set()
15/06/03 18:34:32 INFO DAGScheduler: Missing parents for Stage 29637: List(Stage 29636)
15/06/03 18:34:32 INFO DAGScheduler: Missing parents for Stage 29636: List(Stage 29635)
15/06/03 18:34:32 INFO BlockManager: Found block rdd_711_0 locally
15/06/03 18:34:32 INFO Executor: Finished task 0.0 in stage 29635.0 (TID 149). 2003 bytes result sent to driver
15/06/03 18:34:32 INFO TaskSetManager: Finished task 0.0 in stage 29635.0 (TID 149) in 15 ms on localhost (1/1)
15/06/03 18:34:32 INFO TaskSchedulerImpl: Removed TaskSet 29635.0, whose tasks have all completed, from pool
15/06/03 18:34:32 INFO DAGScheduler: Stage 29635 (mapPartitions at VertexRDDImpl.scala:222) finished in 0.035 s
15/06/03 18:34:32 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:32 INFO DAGScheduler: running: Set()
15/06/03 18:34:32 INFO DAGScheduler: waiting: Set(Stage 29637, Stage 29636)
15/06/03 18:34:32 INFO DAGScheduler: failed: Set()
15/06/03 18:34:32 INFO DAGScheduler: Missing parents for Stage 29637: List(Stage 29636)
15/06/03 18:34:32 INFO DAGScheduler: Missing parents for Stage 29636: List()
15/06/03 18:34:32 INFO DAGScheduler: Submitting Stage 29636 (GraphImpl.aggregateMessages - preAgg MapPartitionsRDD[725] at mapPartitions at GraphImpl.scala:235), which is now runnable
15/06/03 18:34:32 INFO MemoryStore: ensureFreeSpace(15056) called with curMem=494506, maxMem=559153152
15/06/03 18:34:32 INFO MemoryStore: Block broadcast_151 stored as values in memory (estimated size 14.7 KB, free 532.8 MB)
15/06/03 18:34:32 INFO MemoryStore: ensureFreeSpace(7737) called with curMem=509562, maxMem=559153152
15/06/03 18:34:32 INFO MemoryStore: Block broadcast_151_piece0 stored as bytes in memory (estimated size 7.6 KB, free 532.8 MB)
15/06/03 18:34:32 INFO BlockManagerInfo: Added broadcast_151_piece0 in memory on localhost:60836 (size: 7.6 KB, free: 533.1 MB)
15/06/03 18:34:32 INFO BlockManagerMaster: Updated info of block broadcast_151_piece0
15/06/03 18:34:32 INFO SparkContext: Created broadcast 151 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:32 INFO DAGScheduler: Submitting 1 missing tasks from Stage 29636 (GraphImpl.aggregateMessages - preAgg MapPartitionsRDD[725] at mapPartitions at GraphImpl.scala:235)
15/06/03 18:34:32 INFO TaskSchedulerImpl: Adding task set 29636.0 with 1 tasks
15/06/03 18:34:32 INFO TaskSetManager: Starting task 0.0 in stage 29636.0 (TID 150, localhost, PROCESS_LOCAL, 3411 bytes)
15/06/03 18:34:32 INFO Executor: Running task 0.0 in stage 29636.0 (TID 150)
15/06/03 18:34:32 INFO CacheManager: Partition rdd_719_0 not found, computing it
15/06/03 18:34:32 INFO BlockManager: Found block rdd_700_0 locally
15/06/03 18:34:32 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:32 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:32 INFO MemoryStore: ensureFreeSpace(2976) called with curMem=517299, maxMem=559153152
15/06/03 18:34:32 INFO MemoryStore: Block rdd_719_0 stored as values in memory (estimated size 2.9 KB, free 532.8 MB)
15/06/03 18:34:32 INFO BlockManagerInfo: Added rdd_719_0 in memory on localhost:60836 (size: 2.9 KB, free: 533.1 MB)
15/06/03 18:34:32 INFO BlockManagerMaster: Updated info of block rdd_719_0
15/06/03 18:34:32 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:32 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:32 INFO Executor: Finished task 0.0 in stage 29636.0 (TID 150). 2788 bytes result sent to driver
15/06/03 18:34:32 INFO TaskSetManager: Finished task 0.0 in stage 29636.0 (TID 150) in 15 ms on localhost (1/1)
15/06/03 18:34:32 INFO TaskSchedulerImpl: Removed TaskSet 29636.0, whose tasks have all completed, from pool
15/06/03 18:34:32 INFO DAGScheduler: Stage 29636 (mapPartitions at GraphImpl.scala:235) finished in 0.015 s
15/06/03 18:34:32 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:32 INFO DAGScheduler: running: Set()
15/06/03 18:34:32 INFO DAGScheduler: waiting: Set(Stage 29637)
15/06/03 18:34:32 INFO DAGScheduler: failed: Set()
15/06/03 18:34:32 INFO DAGScheduler: Missing parents for Stage 29637: List()
15/06/03 18:34:32 INFO DAGScheduler: Submitting Stage 29637 (MapPartitionsRDD[729] at map at VertexRDDImpl.scala:90), which is now runnable
15/06/03 18:34:32 INFO MemoryStore: ensureFreeSpace(25432) called with curMem=520275, maxMem=559153152
15/06/03 18:34:32 INFO MemoryStore: Block broadcast_152 stored as values in memory (estimated size 24.8 KB, free 532.7 MB)
15/06/03 18:34:32 INFO MemoryStore: ensureFreeSpace(11455) called with curMem=545707, maxMem=559153152
15/06/03 18:34:32 INFO MemoryStore: Block broadcast_152_piece0 stored as bytes in memory (estimated size 11.2 KB, free 532.7 MB)
15/06/03 18:34:32 INFO BlockManagerInfo: Added broadcast_152_piece0 in memory on localhost:60836 (size: 11.2 KB, free: 533.1 MB)
15/06/03 18:34:32 INFO BlockManagerMaster: Updated info of block broadcast_152_piece0
15/06/03 18:34:32 INFO SparkContext: Created broadcast 152 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:32 INFO DAGScheduler: Submitting 1 missing tasks from Stage 29637 (MapPartitionsRDD[729] at map at VertexRDDImpl.scala:90)
15/06/03 18:34:32 INFO TaskSchedulerImpl: Adding task set 29637.0 with 1 tasks
15/06/03 18:34:32 INFO TaskSetManager: Starting task 0.0 in stage 29637.0 (TID 151, localhost, PROCESS_LOCAL, 5326 bytes)
15/06/03 18:34:32 INFO Executor: Running task 0.0 in stage 29637.0 (TID 151)
15/06/03 18:34:32 INFO CacheManager: Partition rdd_727_0 not found, computing it
15/06/03 18:34:32 INFO BlockManager: Found block rdd_713_0 locally
15/06/03 18:34:32 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:32 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:32 INFO MemoryStore: ensureFreeSpace(1664) called with curMem=557162, maxMem=559153152
15/06/03 18:34:32 INFO MemoryStore: Block rdd_727_0 stored as values in memory (estimated size 1664.0 B, free 532.7 MB)
15/06/03 18:34:32 INFO BlockManagerInfo: Added rdd_727_0 in memory on localhost:60836 (size: 1664.0 B, free: 533.1 MB)
15/06/03 18:34:32 INFO BlockManagerMaster: Updated info of block rdd_727_0
15/06/03 18:34:32 INFO Executor: Finished task 0.0 in stage 29637.0 (TID 151). 2693 bytes result sent to driver
15/06/03 18:34:32 INFO TaskSetManager: Finished task 0.0 in stage 29637.0 (TID 151) in 9 ms on localhost (1/1)
15/06/03 18:34:32 INFO TaskSchedulerImpl: Removed TaskSet 29637.0, whose tasks have all completed, from pool
15/06/03 18:34:32 INFO DAGScheduler: Stage 29637 (reduce at VertexRDDImpl.scala:90) finished in 0.009 s
15/06/03 18:34:32 INFO DAGScheduler: Job 37 finished: reduce at VertexRDDImpl.scala:90, took 0.348278 s
15/06/03 18:34:32 INFO Pregel: Pregel finished iteration 35
15/06/03 18:34:32 INFO ZippedPartitionsRDD2: Removing RDD 708 from persistence list
15/06/03 18:34:32 INFO BlockManager: Removing RDD 708
15/06/03 18:34:32 INFO BlockManager: Removing block rdd_708_0
15/06/03 18:34:32 INFO MemoryStore: Block rdd_708_0 of size 1664 dropped from memory (free 558595990)
15/06/03 18:34:32 INFO ZippedPartitionsRDD2: Removing RDD 711 from persistence list
15/06/03 18:34:32 INFO BlockManager: Removing RDD 711
15/06/03 18:34:32 INFO ZippedPartitionsRDD2: Removing RDD 694 from persistence list
15/06/03 18:34:32 INFO BlockManager: Removing block rdd_711_0
15/06/03 18:34:32 INFO MemoryStore: Block rdd_711_0 of size 1544 dropped from memory (free 558597534)
15/06/03 18:34:32 INFO BlockManager: Removing RDD 694
15/06/03 18:34:32 INFO ZippedPartitionsRDD2: Removing RDD 700 from persistence list
15/06/03 18:34:32 INFO BlockManager: Removing block rdd_694_0
15/06/03 18:34:32 INFO MemoryStore: Block rdd_694_0 of size 1496 dropped from memory (free 558599030)
15/06/03 18:34:32 INFO BlockManager: Removing RDD 700
15/06/03 18:34:32 INFO BlockManager: Removing block rdd_700_0
15/06/03 18:34:32 INFO MemoryStore: Block rdd_700_0 of size 2976 dropped from memory (free 558602006)
15/06/03 18:34:32 INFO SparkContext: Starting job: reduce at VertexRDDImpl.scala:90
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 2 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 3 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 5 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 4 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 1 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 0 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 8 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 7 is 145 bytes
15/06/03 18:34:32 INFO BlockManager: Removing broadcast 147
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 6 is 145 bytes
15/06/03 18:34:32 INFO BlockManager: Removing block broadcast_147
15/06/03 18:34:32 INFO MemoryStore: Block broadcast_147 of size 14832 dropped from memory (free 558616838)
15/06/03 18:34:32 INFO BlockManager: Removing block broadcast_147_piece0
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 11 is 145 bytes
15/06/03 18:34:32 INFO MemoryStore: Block broadcast_147_piece0 of size 7704 dropped from memory (free 558624542)
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 10 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 9 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 14 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 13 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 12 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 17 is 145 bytes
15/06/03 18:34:32 INFO BlockManagerInfo: Removed broadcast_147_piece0 on localhost:60836 in memory (size: 7.5 KB, free: 533.1 MB)
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 16 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 15 is 145 bytes
15/06/03 18:34:32 INFO BlockManagerMaster: Updated info of block broadcast_147_piece0
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 20 is 145 bytes
15/06/03 18:34:32 INFO ContextCleaner: Cleaned broadcast 147
15/06/03 18:34:32 INFO BlockManager: Removing broadcast 146
15/06/03 18:34:32 INFO BlockManager: Removing block broadcast_146_piece0
15/06/03 18:34:32 INFO MemoryStore: Block broadcast_146_piece0 of size 10965 dropped from memory (free 558635507)
15/06/03 18:34:32 INFO BlockManagerInfo: Removed broadcast_146_piece0 on localhost:60836 in memory (size: 10.7 KB, free: 533.1 MB)
15/06/03 18:34:32 INFO BlockManagerMaster: Updated info of block broadcast_146_piece0
15/06/03 18:34:32 INFO BlockManager: Removing block broadcast_146
15/06/03 18:34:32 INFO MemoryStore: Block broadcast_146 of size 24280 dropped from memory (free 558659787)
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 19 is 145 bytes
15/06/03 18:34:32 INFO ContextCleaner: Cleaned broadcast 146
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 18 is 145 bytes
15/06/03 18:34:32 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 23 is 145 bytes
15/06/03 18:34:32 INFO BlockManager: Removing broadcast 145
15/06/03 18:34:32 INFO BlockManager: Removing block broadcast_145
15/06/03 18:34:32 INFO MemoryStore: Block broadcast_145 of size 24624 dropped from memory (free 558684411)
15/06/03 18:34:32 INFO BlockManager: Removing block broadcast_145_piece0
15/06/03 18:34:32 INFO MemoryStore: Block broadcast_145_piece0 of size 11079 dropped from memory (free 558695490)
15/06/03 18:34:32 INFO BlockManagerInfo: Removed broadcast_145_piece0 on localhost:60836 in memory (size: 10.8 KB, free: 533.1 MB)
15/06/03 18:34:32 INFO BlockManagerMaster: Updated info of block broadcast_145_piece0
15/06/03 18:34:32 INFO ContextCleaner: Cleaned broadcast 145
15/06/03 18:34:32 INFO BlockManager: Removing broadcast 148
15/06/03 18:34:32 INFO BlockManager: Removing block broadcast_148_piece0
15/06/03 18:34:32 INFO MemoryStore: Block broadcast_148_piece0 of size 11191 dropped from memory (free 558706681)
15/06/03 18:34:32 INFO BlockManagerInfo: Removed broadcast_148_piece0 on localhost:60836 in memory (size: 10.9 KB, free: 533.1 MB)
15/06/03 18:34:32 INFO BlockManagerMaster: Updated info of block broadcast_148_piece0
15/06/03 18:34:32 INFO BlockManager: Removing block broadcast_148
15/06/03 18:34:32 INFO MemoryStore: Block broadcast_148 of size 24904 dropped from memory (free 558731585)
15/06/03 18:34:32 INFO ContextCleaner: Cleaned broadcast 148
15/06/03 18:34:32 INFO BlockManager: Removing broadcast 152
15/06/03 18:34:32 INFO BlockManager: Removing block broadcast_152
15/06/03 18:34:32 INFO MemoryStore: Block broadcast_152 of size 25432 dropped from memory (free 558757017)
15/06/03 18:34:32 INFO BlockManager: Removing block broadcast_152_piece0
15/06/03 18:34:32 INFO MemoryStore: Block broadcast_152_piece0 of size 11455 dropped from memory (free 558768472)
15/06/03 18:34:32 INFO BlockManagerInfo: Removed broadcast_152_piece0 on localhost:60836 in memory (size: 11.2 KB, free: 533.2 MB)
15/06/03 18:34:32 INFO BlockManagerMaster: Updated info of block broadcast_152_piece0
15/06/03 18:34:32 INFO ContextCleaner: Cleaned broadcast 152
15/06/03 18:34:33 INFO BlockManager: Removing broadcast 151
15/06/03 18:34:33 INFO BlockManager: Removing block broadcast_151_piece0
15/06/03 18:34:33 INFO MemoryStore: Block broadcast_151_piece0 of size 7737 dropped from memory (free 558776209)
15/06/03 18:34:33 INFO BlockManagerInfo: Removed broadcast_151_piece0 on localhost:60836 in memory (size: 7.6 KB, free: 533.2 MB)
15/06/03 18:34:33 INFO BlockManagerMaster: Updated info of block broadcast_151_piece0
15/06/03 18:34:33 INFO BlockManager: Removing block broadcast_151
15/06/03 18:34:33 INFO MemoryStore: Block broadcast_151 of size 15056 dropped from memory (free 558791265)
15/06/03 18:34:33 INFO ContextCleaner: Cleaned broadcast 151
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 22 is 145 bytes
15/06/03 18:34:33 INFO BlockManager: Removing broadcast 150
15/06/03 18:34:33 INFO BlockManager: Removing block broadcast_150
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 21 is 145 bytes
15/06/03 18:34:33 INFO MemoryStore: Block broadcast_150 of size 24808 dropped from memory (free 558816073)
15/06/03 18:34:33 INFO BlockManager: Removing block broadcast_150_piece0
15/06/03 18:34:33 INFO MemoryStore: Block broadcast_150_piece0 of size 11091 dropped from memory (free 558827164)
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 26 is 145 bytes
15/06/03 18:34:33 INFO BlockManagerInfo: Removed broadcast_150_piece0 on localhost:60836 in memory (size: 10.8 KB, free: 533.2 MB)
15/06/03 18:34:33 INFO BlockManagerMaster: Updated info of block broadcast_150_piece0
15/06/03 18:34:33 INFO ContextCleaner: Cleaned broadcast 150
15/06/03 18:34:33 INFO BlockManager: Removing broadcast 149
15/06/03 18:34:33 INFO BlockManager: Removing block broadcast_149_piece0
15/06/03 18:34:33 INFO MemoryStore: Block broadcast_149_piece0 of size 11230 dropped from memory (free 558838394)
15/06/03 18:34:33 INFO BlockManagerInfo: Removed broadcast_149_piece0 on localhost:60836 in memory (size: 11.0 KB, free: 533.2 MB)
15/06/03 18:34:33 INFO BlockManagerMaster: Updated info of block broadcast_149_piece0
15/06/03 18:34:33 INFO BlockManager: Removing block broadcast_149
15/06/03 18:34:33 INFO MemoryStore: Block broadcast_149 of size 25144 dropped from memory (free 558863538)
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 25 is 145 bytes
15/06/03 18:34:33 INFO ContextCleaner: Cleaned broadcast 149
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 24 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 29 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 28 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 27 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 32 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 31 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 30 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 35 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 34 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 33 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 38 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 37 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 36 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 41 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 40 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 39 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 44 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 43 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 42 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 47 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 46 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 45 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 50 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 49 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 48 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 53 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 52 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 51 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 56 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 55 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 54 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 59 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 58 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 57 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 62 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 61 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 60 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 65 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 64 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 63 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 68 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 67 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 66 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 71 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 70 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 69 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 74 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 73 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 72 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 77 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 76 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 75 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 80 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 79 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 78 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 83 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 82 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 81 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 86 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 85 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 84 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 89 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 88 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 87 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 92 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 91 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 90 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 95 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 94 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 93 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 98 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 97 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 96 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 101 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 100 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 99 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 104 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 103 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 102 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 107 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 106 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 105 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 110 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 109 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 108 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 113 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 112 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 111 is 145 bytes
15/06/03 18:34:33 INFO DAGScheduler: Registering RDD 736 (mapPartitions at VertexRDDImpl.scala:218)
15/06/03 18:34:33 INFO DAGScheduler: Registering RDD 740 (mapPartitions at VertexRDDImpl.scala:222)
15/06/03 18:34:33 INFO DAGScheduler: Registering RDD 744 (mapPartitions at GraphImpl.scala:235)
15/06/03 18:34:33 INFO DAGScheduler: Got job 38 (reduce at VertexRDDImpl.scala:90) with 1 output partitions (allowLocal=false)
15/06/03 18:34:33 INFO DAGScheduler: Final stage: Stage 31977(reduce at VertexRDDImpl.scala:90)
15/06/03 18:34:33 INFO DAGScheduler: Parents of final stage: List(Stage 31970, Stage 31928, Stage 31920, Stage 31899, Stage 31950, Stage 31972, Stage 31900, Stage 31922, Stage 31944, Stage 31974, Stage 31966, Stage 31916, Stage 31946, Stage 31938, Stage 31968, Stage 31918, Stage 31910, Stage 31940, Stage 31962, Stage 31912, Stage 31942, Stage 31934, Stage 31964, Stage 31914, Stage 31906, Stage 31936, Stage 31958, Stage 31908, Stage 31930, Stage 31960, Stage 31952, Stage 31902, Stage 31932, Stage 31924, Stage 31954, Stage 31976, Stage 31904, Stage 31926, Stage 31956, Stage 31948)
15/06/03 18:34:33 INFO DAGScheduler: Missing parents: List(Stage 31976)
15/06/03 18:34:33 INFO DAGScheduler: Submitting Stage 31860 (ReplicatedVertexView.updateVertices - shippedVerts true false (broadcast) MapPartitionsRDD[736] at mapPartitions at VertexRDDImpl.scala:218), which has no missing parents
15/06/03 18:34:33 INFO MemoryStore: ensureFreeSpace(25672) called with curMem=289614, maxMem=559153152
15/06/03 18:34:33 INFO MemoryStore: Block broadcast_153 stored as values in memory (estimated size 25.1 KB, free 532.9 MB)
15/06/03 18:34:33 INFO MemoryStore: ensureFreeSpace(11546) called with curMem=315286, maxMem=559153152
15/06/03 18:34:33 INFO MemoryStore: Block broadcast_153_piece0 stored as bytes in memory (estimated size 11.3 KB, free 532.9 MB)
15/06/03 18:34:33 INFO BlockManagerInfo: Added broadcast_153_piece0 in memory on localhost:60836 (size: 11.3 KB, free: 533.2 MB)
15/06/03 18:34:33 INFO BlockManagerMaster: Updated info of block broadcast_153_piece0
15/06/03 18:34:33 INFO SparkContext: Created broadcast 153 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:33 INFO DAGScheduler: Submitting 1 missing tasks from Stage 31860 (ReplicatedVertexView.updateVertices - shippedVerts true false (broadcast) MapPartitionsRDD[736] at mapPartitions at VertexRDDImpl.scala:218)
15/06/03 18:34:33 INFO TaskSchedulerImpl: Adding task set 31860.0 with 1 tasks
15/06/03 18:34:33 INFO DAGScheduler: Submitting Stage 31975 (ReplicatedVertexView.withActiveSet - shippedActives (broadcast) MapPartitionsRDD[740] at mapPartitions at VertexRDDImpl.scala:222), which has no missing parents
15/06/03 18:34:33 INFO TaskSetManager: Starting task 0.0 in stage 31860.0 (TID 152, localhost, PROCESS_LOCAL, 5411 bytes)
15/06/03 18:34:33 INFO Executor: Running task 0.0 in stage 31860.0 (TID 152)
15/06/03 18:34:33 INFO MemoryStore: ensureFreeSpace(25328) called with curMem=326832, maxMem=559153152
15/06/03 18:34:33 INFO MemoryStore: Block broadcast_154 stored as values in memory (estimated size 24.7 KB, free 532.9 MB)
15/06/03 18:34:33 INFO MemoryStore: ensureFreeSpace(11412) called with curMem=352160, maxMem=559153152
15/06/03 18:34:33 INFO MemoryStore: Block broadcast_154_piece0 stored as bytes in memory (estimated size 11.1 KB, free 532.9 MB)
15/06/03 18:34:33 INFO BlockManagerInfo: Added broadcast_154_piece0 in memory on localhost:60836 (size: 11.1 KB, free: 533.2 MB)
15/06/03 18:34:33 INFO BlockManagerMaster: Updated info of block broadcast_154_piece0
15/06/03 18:34:33 INFO SparkContext: Created broadcast 154 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:33 INFO DAGScheduler: Submitting 1 missing tasks from Stage 31975 (ReplicatedVertexView.withActiveSet - shippedActives (broadcast) MapPartitionsRDD[740] at mapPartitions at VertexRDDImpl.scala:222)
15/06/03 18:34:33 INFO TaskSchedulerImpl: Adding task set 31975.0 with 1 tasks
15/06/03 18:34:33 INFO BlockManager: Found block rdd_713_0 locally
15/06/03 18:34:33 INFO CacheManager: Partition rdd_732_0 not found, computing it
15/06/03 18:34:33 INFO BlockManager: Found block rdd_713_0 locally
15/06/03 18:34:33 INFO CacheManager: Partition rdd_730_0 not found, computing it
15/06/03 18:34:33 INFO BlockManager: Found block rdd_713_0 locally
15/06/03 18:34:33 INFO BlockManager: Found block rdd_727_0 locally
15/06/03 18:34:33 INFO MemoryStore: ensureFreeSpace(1544) called with curMem=363572, maxMem=559153152
15/06/03 18:34:33 INFO MemoryStore: Block rdd_730_0 stored as values in memory (estimated size 1544.0 B, free 532.9 MB)
15/06/03 18:34:33 INFO BlockManagerInfo: Added rdd_730_0 in memory on localhost:60836 (size: 1544.0 B, free: 533.2 MB)
15/06/03 18:34:33 INFO BlockManagerMaster: Updated info of block rdd_730_0
15/06/03 18:34:33 INFO MemoryStore: ensureFreeSpace(1496) called with curMem=365116, maxMem=559153152
15/06/03 18:34:33 INFO MemoryStore: Block rdd_732_0 stored as values in memory (estimated size 1496.0 B, free 532.9 MB)
15/06/03 18:34:33 INFO BlockManagerInfo: Added rdd_732_0 in memory on localhost:60836 (size: 1496.0 B, free: 533.2 MB)
15/06/03 18:34:33 INFO BlockManagerMaster: Updated info of block rdd_732_0
15/06/03 18:34:33 INFO Executor: Finished task 0.0 in stage 31860.0 (TID 152). 2627 bytes result sent to driver
15/06/03 18:34:33 INFO TaskSetManager: Starting task 0.0 in stage 31975.0 (TID 153, localhost, PROCESS_LOCAL, 5347 bytes)
15/06/03 18:34:33 INFO TaskSetManager: Finished task 0.0 in stage 31860.0 (TID 152) in 29 ms on localhost (1/1)
15/06/03 18:34:33 INFO Executor: Running task 0.0 in stage 31975.0 (TID 153)
15/06/03 18:34:33 INFO TaskSchedulerImpl: Removed TaskSet 31860.0, whose tasks have all completed, from pool
15/06/03 18:34:33 INFO DAGScheduler: Stage 31860 (mapPartitions at VertexRDDImpl.scala:218) finished in 0.029 s
15/06/03 18:34:33 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:33 INFO DAGScheduler: running: Set(Stage 31975)
15/06/03 18:34:33 INFO DAGScheduler: waiting: Set(Stage 31976, Stage 31977)
15/06/03 18:34:33 INFO DAGScheduler: failed: Set()
15/06/03 18:34:33 INFO DAGScheduler: Missing parents for Stage 31976: List(Stage 31975)
15/06/03 18:34:33 INFO DAGScheduler: Missing parents for Stage 31977: List(Stage 31976)
15/06/03 18:34:33 INFO BlockManager: Found block rdd_730_0 locally
15/06/03 18:34:33 INFO Executor: Finished task 0.0 in stage 31975.0 (TID 153). 2003 bytes result sent to driver
15/06/03 18:34:33 INFO TaskSetManager: Finished task 0.0 in stage 31975.0 (TID 153) in 13 ms on localhost (1/1)
15/06/03 18:34:33 INFO TaskSchedulerImpl: Removed TaskSet 31975.0, whose tasks have all completed, from pool
15/06/03 18:34:33 INFO DAGScheduler: Stage 31975 (mapPartitions at VertexRDDImpl.scala:222) finished in 0.033 s
15/06/03 18:34:33 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:33 INFO DAGScheduler: running: Set()
15/06/03 18:34:33 INFO DAGScheduler: waiting: Set(Stage 31976, Stage 31977)
15/06/03 18:34:33 INFO DAGScheduler: failed: Set()
15/06/03 18:34:33 INFO DAGScheduler: Missing parents for Stage 31976: List()
15/06/03 18:34:33 INFO DAGScheduler: Missing parents for Stage 31977: List(Stage 31976)
15/06/03 18:34:33 INFO DAGScheduler: Submitting Stage 31976 (GraphImpl.aggregateMessages - preAgg MapPartitionsRDD[744] at mapPartitions at GraphImpl.scala:235), which is now runnable
15/06/03 18:34:33 INFO MemoryStore: ensureFreeSpace(15288) called with curMem=366612, maxMem=559153152
15/06/03 18:34:33 INFO MemoryStore: Block broadcast_155 stored as values in memory (estimated size 14.9 KB, free 532.9 MB)
15/06/03 18:34:33 INFO MemoryStore: ensureFreeSpace(7821) called with curMem=381900, maxMem=559153152
15/06/03 18:34:33 INFO MemoryStore: Block broadcast_155_piece0 stored as bytes in memory (estimated size 7.6 KB, free 532.9 MB)
15/06/03 18:34:33 INFO BlockManagerInfo: Added broadcast_155_piece0 in memory on localhost:60836 (size: 7.6 KB, free: 533.1 MB)
15/06/03 18:34:33 INFO BlockManagerMaster: Updated info of block broadcast_155_piece0
15/06/03 18:34:33 INFO SparkContext: Created broadcast 155 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:33 INFO DAGScheduler: Submitting 1 missing tasks from Stage 31976 (GraphImpl.aggregateMessages - preAgg MapPartitionsRDD[744] at mapPartitions at GraphImpl.scala:235)
15/06/03 18:34:33 INFO TaskSchedulerImpl: Adding task set 31976.0 with 1 tasks
15/06/03 18:34:33 INFO TaskSetManager: Starting task 0.0 in stage 31976.0 (TID 154, localhost, PROCESS_LOCAL, 3452 bytes)
15/06/03 18:34:33 INFO Executor: Running task 0.0 in stage 31976.0 (TID 154)
15/06/03 18:34:33 INFO CacheManager: Partition rdd_738_0 not found, computing it
15/06/03 18:34:33 INFO BlockManager: Found block rdd_719_0 locally
15/06/03 18:34:33 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:33 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:33 INFO MemoryStore: ensureFreeSpace(2976) called with curMem=389721, maxMem=559153152
15/06/03 18:34:33 INFO MemoryStore: Block rdd_738_0 stored as values in memory (estimated size 2.9 KB, free 532.9 MB)
15/06/03 18:34:33 INFO BlockManagerInfo: Added rdd_738_0 in memory on localhost:60836 (size: 2.9 KB, free: 533.1 MB)
15/06/03 18:34:33 INFO BlockManagerMaster: Updated info of block rdd_738_0
15/06/03 18:34:33 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:33 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:33 INFO Executor: Finished task 0.0 in stage 31976.0 (TID 154). 2788 bytes result sent to driver
15/06/03 18:34:33 INFO TaskSetManager: Finished task 0.0 in stage 31976.0 (TID 154) in 17 ms on localhost (1/1)
15/06/03 18:34:33 INFO TaskSchedulerImpl: Removed TaskSet 31976.0, whose tasks have all completed, from pool
15/06/03 18:34:33 INFO DAGScheduler: Stage 31976 (mapPartitions at GraphImpl.scala:235) finished in 0.017 s
15/06/03 18:34:33 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:33 INFO DAGScheduler: running: Set()
15/06/03 18:34:33 INFO DAGScheduler: waiting: Set(Stage 31977)
15/06/03 18:34:33 INFO DAGScheduler: failed: Set()
15/06/03 18:34:33 INFO DAGScheduler: Missing parents for Stage 31977: List()
15/06/03 18:34:33 INFO DAGScheduler: Submitting Stage 31977 (MapPartitionsRDD[748] at map at VertexRDDImpl.scala:90), which is now runnable
15/06/03 18:34:33 INFO MemoryStore: ensureFreeSpace(25960) called with curMem=392697, maxMem=559153152
15/06/03 18:34:33 INFO MemoryStore: Block broadcast_156 stored as values in memory (estimated size 25.4 KB, free 532.9 MB)
15/06/03 18:34:33 INFO MemoryStore: ensureFreeSpace(11625) called with curMem=418657, maxMem=559153152
15/06/03 18:34:33 INFO MemoryStore: Block broadcast_156_piece0 stored as bytes in memory (estimated size 11.4 KB, free 532.8 MB)
15/06/03 18:34:33 INFO BlockManagerInfo: Added broadcast_156_piece0 in memory on localhost:60836 (size: 11.4 KB, free: 533.1 MB)
15/06/03 18:34:33 INFO BlockManagerMaster: Updated info of block broadcast_156_piece0
15/06/03 18:34:33 INFO SparkContext: Created broadcast 156 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:33 INFO DAGScheduler: Submitting 1 missing tasks from Stage 31977 (MapPartitionsRDD[748] at map at VertexRDDImpl.scala:90)
15/06/03 18:34:33 INFO TaskSchedulerImpl: Adding task set 31977.0 with 1 tasks
15/06/03 18:34:33 INFO TaskSetManager: Starting task 0.0 in stage 31977.0 (TID 155, localhost, PROCESS_LOCAL, 5431 bytes)
15/06/03 18:34:33 INFO Executor: Running task 0.0 in stage 31977.0 (TID 155)
15/06/03 18:34:33 INFO CacheManager: Partition rdd_746_0 not found, computing it
15/06/03 18:34:33 INFO BlockManager: Found block rdd_732_0 locally
15/06/03 18:34:33 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:33 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:33 INFO MemoryStore: ensureFreeSpace(1664) called with curMem=430282, maxMem=559153152
15/06/03 18:34:33 INFO MemoryStore: Block rdd_746_0 stored as values in memory (estimated size 1664.0 B, free 532.8 MB)
15/06/03 18:34:33 INFO BlockManagerInfo: Added rdd_746_0 in memory on localhost:60836 (size: 1664.0 B, free: 533.1 MB)
15/06/03 18:34:33 INFO BlockManagerMaster: Updated info of block rdd_746_0
15/06/03 18:34:33 INFO Executor: Finished task 0.0 in stage 31977.0 (TID 155). 2693 bytes result sent to driver
15/06/03 18:34:33 INFO TaskSetManager: Finished task 0.0 in stage 31977.0 (TID 155) in 9 ms on localhost (1/1)
15/06/03 18:34:33 INFO TaskSchedulerImpl: Removed TaskSet 31977.0, whose tasks have all completed, from pool
15/06/03 18:34:33 INFO DAGScheduler: Stage 31977 (reduce at VertexRDDImpl.scala:90) finished in 0.009 s
15/06/03 18:34:33 INFO DAGScheduler: Job 38 finished: reduce at VertexRDDImpl.scala:90, took 0.448018 s
15/06/03 18:34:33 INFO Pregel: Pregel finished iteration 36
15/06/03 18:34:33 INFO ZippedPartitionsRDD2: Removing RDD 727 from persistence list
15/06/03 18:34:33 INFO BlockManager: Removing RDD 727
15/06/03 18:34:33 INFO ZippedPartitionsRDD2: Removing RDD 730 from persistence list
15/06/03 18:34:33 INFO BlockManager: Removing block rdd_727_0
15/06/03 18:34:33 INFO MemoryStore: Block rdd_727_0 of size 1664 dropped from memory (free 558722870)
15/06/03 18:34:33 INFO BlockManager: Removing RDD 730
15/06/03 18:34:33 INFO BlockManager: Removing block rdd_730_0
15/06/03 18:34:33 INFO ZippedPartitionsRDD2: Removing RDD 713 from persistence list
15/06/03 18:34:33 INFO MemoryStore: Block rdd_730_0 of size 1544 dropped from memory (free 558724414)
15/06/03 18:34:33 INFO BlockManager: Removing RDD 713
15/06/03 18:34:33 INFO ZippedPartitionsRDD2: Removing RDD 719 from persistence list
15/06/03 18:34:33 INFO BlockManager: Removing block rdd_713_0
15/06/03 18:34:33 INFO MemoryStore: Block rdd_713_0 of size 1496 dropped from memory (free 558725910)
15/06/03 18:34:33 INFO BlockManager: Removing RDD 719
15/06/03 18:34:33 INFO BlockManager: Removing block rdd_719_0
15/06/03 18:34:33 INFO MemoryStore: Block rdd_719_0 of size 2976 dropped from memory (free 558728886)
15/06/03 18:34:33 INFO SparkContext: Starting job: reduce at VertexRDDImpl.scala:90
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 2 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 3 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 5 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 4 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 1 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 0 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 8 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 7 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 6 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 11 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 10 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 9 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 14 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 13 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 12 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 17 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 16 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 15 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 20 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 19 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 18 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 23 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 22 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 21 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 26 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 25 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 24 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 29 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 28 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 27 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 32 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 31 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 30 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 35 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 34 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 33 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 38 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 37 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 36 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 41 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 40 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 39 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 44 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 43 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 42 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 47 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 46 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 45 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 50 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 49 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 48 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 53 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 52 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 51 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 56 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 55 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 54 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 59 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 58 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 57 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 62 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 61 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 60 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 65 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 64 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 63 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 68 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 67 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 66 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 71 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 70 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 69 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 74 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 73 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 72 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 77 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 76 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 75 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 80 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 79 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 78 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 83 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 82 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 81 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 86 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 85 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 84 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 89 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 88 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 87 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 92 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 91 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 90 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 95 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 94 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 93 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 98 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 97 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 96 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 101 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 100 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 99 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 104 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 103 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 102 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 107 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 106 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 105 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 110 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 109 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 108 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 113 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 112 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 111 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 116 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 115 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 114 is 145 bytes
15/06/03 18:34:33 INFO DAGScheduler: Registering RDD 755 (mapPartitions at VertexRDDImpl.scala:218)
15/06/03 18:34:33 INFO DAGScheduler: Registering RDD 759 (mapPartitions at VertexRDDImpl.scala:222)
15/06/03 18:34:33 INFO DAGScheduler: Registering RDD 763 (mapPartitions at GraphImpl.scala:235)
15/06/03 18:34:33 INFO DAGScheduler: Got job 39 (reduce at VertexRDDImpl.scala:90) with 1 output partitions (allowLocal=false)
15/06/03 18:34:33 INFO DAGScheduler: Final stage: Stage 34437(reduce at VertexRDDImpl.scala:90)
15/06/03 18:34:33 INFO DAGScheduler: Parents of final stage: List(Stage 34380, Stage 34410, Stage 34432, Stage 34360, Stage 34382, Stage 34404, Stage 34434, Stage 34426, Stage 34384, Stage 34376, Stage 34406, Stage 34398, Stage 34428, Stage 34378, Stage 34400, Stage 34430, Stage 34422, Stage 34372, Stage 34402, Stage 34394, Stage 34424, Stage 34374, Stage 34366, Stage 34396, Stage 34418, Stage 34368, Stage 34390, Stage 34420, Stage 34412, Stage 34370, Stage 34362, Stage 34392, Stage 34414, Stage 34436, Stage 34364, Stage 34386, Stage 34416, Stage 34357, Stage 34408, Stage 34358, Stage 34388)
15/06/03 18:34:33 INFO DAGScheduler: Missing parents: List(Stage 34436)
15/06/03 18:34:33 INFO DAGScheduler: Submitting Stage 34317 (ReplicatedVertexView.updateVertices - shippedVerts true false (broadcast) MapPartitionsRDD[755] at mapPartitions at VertexRDDImpl.scala:218), which has no missing parents
15/06/03 18:34:33 INFO MemoryStore: ensureFreeSpace(26200) called with curMem=424266, maxMem=559153152
15/06/03 18:34:33 INFO MemoryStore: Block broadcast_157 stored as values in memory (estimated size 25.6 KB, free 532.8 MB)
15/06/03 18:34:33 INFO MemoryStore: ensureFreeSpace(11710) called with curMem=450466, maxMem=559153152
15/06/03 18:34:33 INFO MemoryStore: Block broadcast_157_piece0 stored as bytes in memory (estimated size 11.4 KB, free 532.8 MB)
15/06/03 18:34:33 INFO BlockManagerInfo: Added broadcast_157_piece0 in memory on localhost:60836 (size: 11.4 KB, free: 533.1 MB)
15/06/03 18:34:33 INFO BlockManagerMaster: Updated info of block broadcast_157_piece0
15/06/03 18:34:33 INFO SparkContext: Created broadcast 157 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:33 INFO DAGScheduler: Submitting 1 missing tasks from Stage 34317 (ReplicatedVertexView.updateVertices - shippedVerts true false (broadcast) MapPartitionsRDD[755] at mapPartitions at VertexRDDImpl.scala:218)
15/06/03 18:34:33 INFO TaskSchedulerImpl: Adding task set 34317.0 with 1 tasks
15/06/03 18:34:33 INFO DAGScheduler: Submitting Stage 34435 (ReplicatedVertexView.withActiveSet - shippedActives (broadcast) MapPartitionsRDD[759] at mapPartitions at VertexRDDImpl.scala:222), which has no missing parents
15/06/03 18:34:33 INFO TaskSetManager: Starting task 0.0 in stage 34317.0 (TID 156, localhost, PROCESS_LOCAL, 5516 bytes)
15/06/03 18:34:33 INFO Executor: Running task 0.0 in stage 34317.0 (TID 156)
15/06/03 18:34:33 INFO MemoryStore: ensureFreeSpace(25856) called with curMem=462176, maxMem=559153152
15/06/03 18:34:33 INFO MemoryStore: Block broadcast_158 stored as values in memory (estimated size 25.3 KB, free 532.8 MB)
15/06/03 18:34:33 INFO MemoryStore: ensureFreeSpace(11553) called with curMem=488032, maxMem=559153152
15/06/03 18:34:33 INFO MemoryStore: Block broadcast_158_piece0 stored as bytes in memory (estimated size 11.3 KB, free 532.8 MB)
15/06/03 18:34:33 INFO BlockManagerInfo: Added broadcast_158_piece0 in memory on localhost:60836 (size: 11.3 KB, free: 533.1 MB)
15/06/03 18:34:33 INFO BlockManager: Removing broadcast 155
15/06/03 18:34:33 INFO BlockManager: Removing block broadcast_155
15/06/03 18:34:33 INFO MemoryStore: Block broadcast_155 of size 15288 dropped from memory (free 558668855)
15/06/03 18:34:33 INFO BlockManager: Removing block broadcast_155_piece0
15/06/03 18:34:33 INFO MemoryStore: Block broadcast_155_piece0 of size 7821 dropped from memory (free 558676676)
15/06/03 18:34:33 INFO BlockManagerInfo: Removed broadcast_155_piece0 on localhost:60836 in memory (size: 7.6 KB, free: 533.1 MB)
15/06/03 18:34:33 INFO BlockManagerMaster: Updated info of block broadcast_155_piece0
15/06/03 18:34:33 INFO BlockManager: Found block rdd_732_0 locally
15/06/03 18:34:33 INFO ContextCleaner: Cleaned broadcast 155
15/06/03 18:34:33 INFO CacheManager: Partition rdd_751_0 not found, computing it
15/06/03 18:34:33 INFO BlockManager: Found block rdd_732_0 locally
15/06/03 18:34:33 INFO BlockManager: Removing broadcast 154
15/06/03 18:34:33 INFO BlockManager: Removing block broadcast_154_piece0
15/06/03 18:34:33 INFO MemoryStore: Block broadcast_154_piece0 of size 11412 dropped from memory (free 558688088)
15/06/03 18:34:33 INFO CacheManager: Partition rdd_749_0 not found, computing it
15/06/03 18:34:33 INFO BlockManager: Found block rdd_732_0 locally
15/06/03 18:34:33 INFO BlockManager: Found block rdd_746_0 locally
15/06/03 18:34:33 INFO BlockManagerInfo: Removed broadcast_154_piece0 on localhost:60836 in memory (size: 11.1 KB, free: 533.1 MB)
15/06/03 18:34:33 INFO MemoryStore: ensureFreeSpace(1544) called with curMem=465064, maxMem=559153152
15/06/03 18:34:33 INFO MemoryStore: Block rdd_749_0 stored as values in memory (estimated size 1544.0 B, free 532.8 MB)
15/06/03 18:34:33 INFO BlockManagerMaster: Updated info of block broadcast_154_piece0
15/06/03 18:34:33 INFO BlockManager: Removing block broadcast_154
15/06/03 18:34:33 INFO MemoryStore: Block broadcast_154 of size 25328 dropped from memory (free 558711872)
15/06/03 18:34:33 INFO ContextCleaner: Cleaned broadcast 154
15/06/03 18:34:33 INFO BlockManagerInfo: Added rdd_749_0 in memory on localhost:60836 (size: 1544.0 B, free: 533.1 MB)
15/06/03 18:34:33 INFO BlockManager: Removing broadcast 156
15/06/03 18:34:33 INFO BlockManager: Removing block broadcast_156_piece0
15/06/03 18:34:33 INFO MemoryStore: Block broadcast_156_piece0 of size 11625 dropped from memory (free 558723497)
15/06/03 18:34:33 INFO BlockManagerMaster: Updated info of block rdd_749_0
15/06/03 18:34:33 INFO BlockManagerInfo: Removed broadcast_156_piece0 on localhost:60836 in memory (size: 11.4 KB, free: 533.1 MB)
15/06/03 18:34:33 INFO MemoryStore: ensureFreeSpace(1496) called with curMem=429655, maxMem=559153152
15/06/03 18:34:33 INFO BlockManagerMaster: Updated info of block broadcast_156_piece0
15/06/03 18:34:33 INFO BlockManager: Removing block broadcast_156
15/06/03 18:34:33 INFO MemoryStore: Block broadcast_156 of size 25960 dropped from memory (free 558747961)
15/06/03 18:34:33 INFO MemoryStore: Block rdd_751_0 stored as values in memory (estimated size 1496.0 B, free 532.9 MB)
15/06/03 18:34:33 INFO ContextCleaner: Cleaned broadcast 156
15/06/03 18:34:33 INFO BlockManagerMaster: Updated info of block broadcast_158_piece0
15/06/03 18:34:33 INFO BlockManagerInfo: Added rdd_751_0 in memory on localhost:60836 (size: 1496.0 B, free: 533.1 MB)
15/06/03 18:34:33 INFO BlockManagerMaster: Updated info of block rdd_751_0
15/06/03 18:34:33 INFO SparkContext: Created broadcast 158 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:33 INFO DAGScheduler: Submitting 1 missing tasks from Stage 34435 (ReplicatedVertexView.withActiveSet - shippedActives (broadcast) MapPartitionsRDD[759] at mapPartitions at VertexRDDImpl.scala:222)
15/06/03 18:34:33 INFO TaskSchedulerImpl: Adding task set 34435.0 with 1 tasks
15/06/03 18:34:33 INFO Executor: Finished task 0.0 in stage 34317.0 (TID 156). 2627 bytes result sent to driver
15/06/03 18:34:33 INFO TaskSetManager: Starting task 0.0 in stage 34435.0 (TID 157, localhost, PROCESS_LOCAL, 5452 bytes)
15/06/03 18:34:33 INFO TaskSetManager: Finished task 0.0 in stage 34317.0 (TID 156) in 115 ms on localhost (1/1)
15/06/03 18:34:33 INFO TaskSchedulerImpl: Removed TaskSet 34317.0, whose tasks have all completed, from pool
15/06/03 18:34:33 INFO Executor: Running task 0.0 in stage 34435.0 (TID 157)
15/06/03 18:34:33 INFO DAGScheduler: Stage 34317 (mapPartitions at VertexRDDImpl.scala:218) finished in 0.116 s
15/06/03 18:34:33 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:33 INFO DAGScheduler: running: Set(Stage 34435)
15/06/03 18:34:33 INFO DAGScheduler: waiting: Set(Stage 34436, Stage 34437)
15/06/03 18:34:33 INFO DAGScheduler: failed: Set()
15/06/03 18:34:33 INFO DAGScheduler: Missing parents for Stage 34436: List(Stage 34435)
15/06/03 18:34:33 INFO DAGScheduler: Missing parents for Stage 34437: List(Stage 34436)
15/06/03 18:34:33 INFO BlockManager: Found block rdd_749_0 locally
15/06/03 18:34:33 INFO Executor: Finished task 0.0 in stage 34435.0 (TID 157). 2003 bytes result sent to driver
15/06/03 18:34:33 INFO TaskSetManager: Finished task 0.0 in stage 34435.0 (TID 157) in 18 ms on localhost (1/1)
15/06/03 18:34:33 INFO TaskSchedulerImpl: Removed TaskSet 34435.0, whose tasks have all completed, from pool
15/06/03 18:34:33 INFO DAGScheduler: Stage 34435 (mapPartitions at VertexRDDImpl.scala:222) finished in 0.033 s
15/06/03 18:34:33 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:33 INFO DAGScheduler: running: Set()
15/06/03 18:34:33 INFO DAGScheduler: waiting: Set(Stage 34436, Stage 34437)
15/06/03 18:34:33 INFO DAGScheduler: failed: Set()
15/06/03 18:34:33 INFO DAGScheduler: Missing parents for Stage 34436: List()
15/06/03 18:34:33 INFO DAGScheduler: Missing parents for Stage 34437: List(Stage 34436)
15/06/03 18:34:33 INFO DAGScheduler: Submitting Stage 34436 (GraphImpl.aggregateMessages - preAgg MapPartitionsRDD[763] at mapPartitions at GraphImpl.scala:235), which is now runnable
15/06/03 18:34:33 INFO MemoryStore: ensureFreeSpace(15512) called with curMem=405191, maxMem=559153152
15/06/03 18:34:33 INFO MemoryStore: Block broadcast_159 stored as values in memory (estimated size 15.1 KB, free 532.8 MB)
15/06/03 18:34:33 INFO MemoryStore: ensureFreeSpace(7884) called with curMem=420703, maxMem=559153152
15/06/03 18:34:33 INFO MemoryStore: Block broadcast_159_piece0 stored as bytes in memory (estimated size 7.7 KB, free 532.8 MB)
15/06/03 18:34:33 INFO BlockManagerInfo: Added broadcast_159_piece0 in memory on localhost:60836 (size: 7.7 KB, free: 533.1 MB)
15/06/03 18:34:33 INFO BlockManagerMaster: Updated info of block broadcast_159_piece0
15/06/03 18:34:33 INFO SparkContext: Created broadcast 159 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:33 INFO DAGScheduler: Submitting 1 missing tasks from Stage 34436 (GraphImpl.aggregateMessages - preAgg MapPartitionsRDD[763] at mapPartitions at GraphImpl.scala:235)
15/06/03 18:34:33 INFO TaskSchedulerImpl: Adding task set 34436.0 with 1 tasks
15/06/03 18:34:33 INFO TaskSetManager: Starting task 0.0 in stage 34436.0 (TID 158, localhost, PROCESS_LOCAL, 3493 bytes)
15/06/03 18:34:33 INFO Executor: Running task 0.0 in stage 34436.0 (TID 158)
15/06/03 18:34:33 INFO CacheManager: Partition rdd_757_0 not found, computing it
15/06/03 18:34:33 INFO BlockManager: Found block rdd_738_0 locally
15/06/03 18:34:33 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:33 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:33 INFO MemoryStore: ensureFreeSpace(2976) called with curMem=428587, maxMem=559153152
15/06/03 18:34:33 INFO MemoryStore: Block rdd_757_0 stored as values in memory (estimated size 2.9 KB, free 532.8 MB)
15/06/03 18:34:33 INFO BlockManagerInfo: Added rdd_757_0 in memory on localhost:60836 (size: 2.9 KB, free: 533.1 MB)
15/06/03 18:34:33 INFO BlockManagerMaster: Updated info of block rdd_757_0
15/06/03 18:34:33 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:33 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:33 INFO Executor: Finished task 0.0 in stage 34436.0 (TID 158). 2788 bytes result sent to driver
15/06/03 18:34:33 INFO TaskSetManager: Finished task 0.0 in stage 34436.0 (TID 158) in 14 ms on localhost (1/1)
15/06/03 18:34:33 INFO TaskSchedulerImpl: Removed TaskSet 34436.0, whose tasks have all completed, from pool
15/06/03 18:34:33 INFO DAGScheduler: Stage 34436 (mapPartitions at GraphImpl.scala:235) finished in 0.015 s
15/06/03 18:34:33 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:33 INFO DAGScheduler: running: Set()
15/06/03 18:34:33 INFO DAGScheduler: waiting: Set(Stage 34437)
15/06/03 18:34:33 INFO DAGScheduler: failed: Set()
15/06/03 18:34:33 INFO DAGScheduler: Missing parents for Stage 34437: List()
15/06/03 18:34:33 INFO DAGScheduler: Submitting Stage 34437 (MapPartitionsRDD[767] at map at VertexRDDImpl.scala:90), which is now runnable
15/06/03 18:34:33 INFO MemoryStore: ensureFreeSpace(26488) called with curMem=431563, maxMem=559153152
15/06/03 18:34:33 INFO MemoryStore: Block broadcast_160 stored as values in memory (estimated size 25.9 KB, free 532.8 MB)
15/06/03 18:34:33 INFO MemoryStore: ensureFreeSpace(11843) called with curMem=458051, maxMem=559153152
15/06/03 18:34:33 INFO MemoryStore: Block broadcast_160_piece0 stored as bytes in memory (estimated size 11.6 KB, free 532.8 MB)
15/06/03 18:34:33 INFO BlockManagerInfo: Added broadcast_160_piece0 in memory on localhost:60836 (size: 11.6 KB, free: 533.1 MB)
15/06/03 18:34:33 INFO BlockManagerMaster: Updated info of block broadcast_160_piece0
15/06/03 18:34:33 INFO SparkContext: Created broadcast 160 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:33 INFO DAGScheduler: Submitting 1 missing tasks from Stage 34437 (MapPartitionsRDD[767] at map at VertexRDDImpl.scala:90)
15/06/03 18:34:33 INFO TaskSchedulerImpl: Adding task set 34437.0 with 1 tasks
15/06/03 18:34:33 INFO TaskSetManager: Starting task 0.0 in stage 34437.0 (TID 159, localhost, PROCESS_LOCAL, 5536 bytes)
15/06/03 18:34:33 INFO Executor: Running task 0.0 in stage 34437.0 (TID 159)
15/06/03 18:34:33 INFO CacheManager: Partition rdd_765_0 not found, computing it
15/06/03 18:34:33 INFO BlockManager: Found block rdd_751_0 locally
15/06/03 18:34:33 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:33 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:33 INFO MemoryStore: ensureFreeSpace(1664) called with curMem=469894, maxMem=559153152
15/06/03 18:34:33 INFO MemoryStore: Block rdd_765_0 stored as values in memory (estimated size 1664.0 B, free 532.8 MB)
15/06/03 18:34:33 INFO BlockManagerInfo: Added rdd_765_0 in memory on localhost:60836 (size: 1664.0 B, free: 533.1 MB)
15/06/03 18:34:33 INFO BlockManagerMaster: Updated info of block rdd_765_0
15/06/03 18:34:33 INFO Executor: Finished task 0.0 in stage 34437.0 (TID 159). 2693 bytes result sent to driver
15/06/03 18:34:33 INFO TaskSetManager: Finished task 0.0 in stage 34437.0 (TID 159) in 11 ms on localhost (1/1)
15/06/03 18:34:33 INFO TaskSchedulerImpl: Removed TaskSet 34437.0, whose tasks have all completed, from pool
15/06/03 18:34:33 INFO DAGScheduler: Stage 34437 (reduce at VertexRDDImpl.scala:90) finished in 0.011 s
15/06/03 18:34:33 INFO DAGScheduler: Job 39 finished: reduce at VertexRDDImpl.scala:90, took 0.451471 s
15/06/03 18:34:33 INFO Pregel: Pregel finished iteration 37
15/06/03 18:34:33 INFO ZippedPartitionsRDD2: Removing RDD 746 from persistence list
15/06/03 18:34:33 INFO BlockManager: Removing RDD 746
15/06/03 18:34:33 INFO ZippedPartitionsRDD2: Removing RDD 749 from persistence list
15/06/03 18:34:33 INFO BlockManager: Removing block rdd_746_0
15/06/03 18:34:33 INFO MemoryStore: Block rdd_746_0 of size 1664 dropped from memory (free 558683258)
15/06/03 18:34:33 INFO BlockManager: Removing RDD 749
15/06/03 18:34:33 INFO ZippedPartitionsRDD2: Removing RDD 732 from persistence list
15/06/03 18:34:33 INFO BlockManager: Removing block rdd_749_0
15/06/03 18:34:33 INFO MemoryStore: Block rdd_749_0 of size 1544 dropped from memory (free 558684802)
15/06/03 18:34:33 INFO BlockManager: Removing RDD 732
15/06/03 18:34:33 INFO BlockManager: Removing block rdd_732_0
15/06/03 18:34:33 INFO MemoryStore: Block rdd_732_0 of size 1496 dropped from memory (free 558686298)
15/06/03 18:34:33 INFO ZippedPartitionsRDD2: Removing RDD 738 from persistence list
15/06/03 18:34:33 INFO BlockManager: Removing RDD 738
15/06/03 18:34:33 INFO BlockManager: Removing block rdd_738_0
15/06/03 18:34:33 INFO MemoryStore: Block rdd_738_0 of size 2976 dropped from memory (free 558689274)
15/06/03 18:34:33 INFO SparkContext: Starting job: reduce at VertexRDDImpl.scala:90
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 2 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 3 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 5 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 4 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 1 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 0 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 8 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 7 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 6 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 11 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 10 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 9 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 14 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 13 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 12 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 17 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 16 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 15 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 20 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 19 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 18 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 23 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 22 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 21 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 26 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 25 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 24 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 29 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 28 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 27 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 32 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 31 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 30 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 35 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 34 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 33 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 38 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 37 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 36 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 41 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 40 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 39 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 44 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 43 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 42 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 47 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 46 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 45 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 50 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 49 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 48 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 53 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 52 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 51 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 56 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 55 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 54 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 59 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 58 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 57 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 62 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 61 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 60 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 65 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 64 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 63 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 68 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 67 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 66 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 71 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 70 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 69 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 74 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 73 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 72 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 77 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 76 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 75 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 80 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 79 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 78 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 83 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 82 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 81 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 86 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 85 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 84 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 89 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 88 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 87 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 92 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 91 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 90 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 95 is 145 bytes
15/06/03 18:34:33 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 94 is 145 bytes
15/06/03 18:34:34 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 93 is 145 bytes
15/06/03 18:34:34 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 98 is 145 bytes
15/06/03 18:34:34 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 97 is 145 bytes
15/06/03 18:34:34 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 96 is 145 bytes
15/06/03 18:34:34 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 101 is 145 bytes
15/06/03 18:34:34 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 100 is 145 bytes
15/06/03 18:34:34 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 99 is 145 bytes
15/06/03 18:34:34 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 104 is 145 bytes
15/06/03 18:34:34 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 103 is 145 bytes
15/06/03 18:34:34 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 102 is 145 bytes
15/06/03 18:34:34 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 107 is 145 bytes
15/06/03 18:34:34 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 106 is 145 bytes
15/06/03 18:34:34 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 105 is 145 bytes
15/06/03 18:34:34 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 110 is 145 bytes
15/06/03 18:34:34 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 109 is 145 bytes
15/06/03 18:34:34 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 108 is 145 bytes
15/06/03 18:34:34 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 113 is 145 bytes
15/06/03 18:34:34 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 112 is 145 bytes
15/06/03 18:34:34 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 111 is 145 bytes
15/06/03 18:34:34 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 116 is 145 bytes
15/06/03 18:34:34 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 115 is 145 bytes
15/06/03 18:34:34 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 114 is 145 bytes
15/06/03 18:34:34 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 119 is 145 bytes
15/06/03 18:34:34 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 118 is 145 bytes
15/06/03 18:34:34 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 117 is 145 bytes
15/06/03 18:34:34 INFO DAGScheduler: Registering RDD 774 (mapPartitions at VertexRDDImpl.scala:218)
15/06/03 18:34:34 INFO DAGScheduler: Registering RDD 778 (mapPartitions at VertexRDDImpl.scala:222)
15/06/03 18:34:34 INFO DAGScheduler: Registering RDD 782 (mapPartitions at GraphImpl.scala:235)
15/06/03 18:34:34 INFO DAGScheduler: Got job 40 (reduce at VertexRDDImpl.scala:90) with 1 output partitions (allowLocal=false)
15/06/03 18:34:34 INFO DAGScheduler: Final stage: Stage 37020(reduce at VertexRDDImpl.scala:90)
15/06/03 18:34:34 INFO DAGScheduler: Parents of final stage: List(Stage 36943, Stage 36973, Stage 36965, Stage 36995, Stage 37017, Stage 36945, Stage 36967, Stage 36938, Stage 36989, Stage 37019, Stage 37011, Stage 36939, Stage 36961, Stage 36991, Stage 36983, Stage 37013, Stage 36941, Stage 36963, Stage 36985, Stage 37015, Stage 37007, Stage 36957, Stage 36987, Stage 36979, Stage 37009, Stage 36959, Stage 36951, Stage 36981, Stage 37003, Stage 36953, Stage 36975, Stage 37005, Stage 36997, Stage 36955, Stage 36947, Stage 36977, Stage 36969, Stage 36999, Stage 36949, Stage 36971, Stage 37001, Stage 36993)
15/06/03 18:34:34 INFO DAGScheduler: Missing parents: List(Stage 37019)
15/06/03 18:34:34 INFO DAGScheduler: Submitting Stage 36897 (ReplicatedVertexView.updateVertices - shippedVerts true false (broadcast) MapPartitionsRDD[774] at mapPartitions at VertexRDDImpl.scala:218), which has no missing parents
15/06/03 18:34:34 INFO MemoryStore: ensureFreeSpace(26728) called with curMem=463878, maxMem=559153152
15/06/03 18:34:34 INFO MemoryStore: Block broadcast_161 stored as values in memory (estimated size 26.1 KB, free 532.8 MB)
15/06/03 18:34:34 INFO MemoryStore: ensureFreeSpace(11932) called with curMem=490606, maxMem=559153152
15/06/03 18:34:34 INFO MemoryStore: Block broadcast_161_piece0 stored as bytes in memory (estimated size 11.7 KB, free 532.8 MB)
15/06/03 18:34:34 INFO BlockManagerInfo: Added broadcast_161_piece0 in memory on localhost:60836 (size: 11.7 KB, free: 533.1 MB)
15/06/03 18:34:34 INFO BlockManagerMaster: Updated info of block broadcast_161_piece0
15/06/03 18:34:34 INFO SparkContext: Created broadcast 161 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:34 INFO DAGScheduler: Submitting 1 missing tasks from Stage 36897 (ReplicatedVertexView.updateVertices - shippedVerts true false (broadcast) MapPartitionsRDD[774] at mapPartitions at VertexRDDImpl.scala:218)
15/06/03 18:34:34 INFO TaskSchedulerImpl: Adding task set 36897.0 with 1 tasks
15/06/03 18:34:34 INFO DAGScheduler: Submitting Stage 37018 (ReplicatedVertexView.withActiveSet - shippedActives (broadcast) MapPartitionsRDD[778] at mapPartitions at VertexRDDImpl.scala:222), which has no missing parents
15/06/03 18:34:34 INFO TaskSetManager: Starting task 0.0 in stage 36897.0 (TID 160, localhost, PROCESS_LOCAL, 5621 bytes)
15/06/03 18:34:34 INFO Executor: Running task 0.0 in stage 36897.0 (TID 160)
15/06/03 18:34:34 INFO MemoryStore: ensureFreeSpace(26384) called with curMem=502538, maxMem=559153152
15/06/03 18:34:34 INFO MemoryStore: Block broadcast_162 stored as values in memory (estimated size 25.8 KB, free 532.7 MB)
15/06/03 18:34:34 INFO MemoryStore: ensureFreeSpace(11804) called with curMem=528922, maxMem=559153152
15/06/03 18:34:34 INFO MemoryStore: Block broadcast_162_piece0 stored as bytes in memory (estimated size 11.5 KB, free 532.7 MB)
15/06/03 18:34:34 INFO BlockManagerInfo: Added broadcast_162_piece0 in memory on localhost:60836 (size: 11.5 KB, free: 533.1 MB)
15/06/03 18:34:34 INFO BlockManagerMaster: Updated info of block broadcast_162_piece0
15/06/03 18:34:34 INFO SparkContext: Created broadcast 162 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:34 INFO DAGScheduler: Submitting 1 missing tasks from Stage 37018 (ReplicatedVertexView.withActiveSet - shippedActives (broadcast) MapPartitionsRDD[778] at mapPartitions at VertexRDDImpl.scala:222)
15/06/03 18:34:34 INFO TaskSchedulerImpl: Adding task set 37018.0 with 1 tasks
15/06/03 18:34:34 INFO BlockManager: Found block rdd_751_0 locally
15/06/03 18:34:34 INFO CacheManager: Partition rdd_770_0 not found, computing it
15/06/03 18:34:34 INFO BlockManager: Found block rdd_751_0 locally
15/06/03 18:34:34 INFO CacheManager: Partition rdd_768_0 not found, computing it
15/06/03 18:34:34 INFO BlockManager: Found block rdd_751_0 locally
15/06/03 18:34:34 INFO BlockManager: Found block rdd_765_0 locally
15/06/03 18:34:34 INFO MemoryStore: ensureFreeSpace(1544) called with curMem=540726, maxMem=559153152
15/06/03 18:34:34 INFO MemoryStore: Block rdd_768_0 stored as values in memory (estimated size 1544.0 B, free 532.7 MB)
15/06/03 18:34:34 INFO BlockManagerInfo: Added rdd_768_0 in memory on localhost:60836 (size: 1544.0 B, free: 533.1 MB)
15/06/03 18:34:34 INFO BlockManagerMaster: Updated info of block rdd_768_0
15/06/03 18:34:34 INFO MemoryStore: ensureFreeSpace(1496) called with curMem=542270, maxMem=559153152
15/06/03 18:34:34 INFO MemoryStore: Block rdd_770_0 stored as values in memory (estimated size 1496.0 B, free 532.7 MB)
15/06/03 18:34:34 INFO BlockManagerInfo: Added rdd_770_0 in memory on localhost:60836 (size: 1496.0 B, free: 533.1 MB)
15/06/03 18:34:34 INFO BlockManagerMaster: Updated info of block rdd_770_0
15/06/03 18:34:34 INFO Executor: Finished task 0.0 in stage 36897.0 (TID 160). 2627 bytes result sent to driver
15/06/03 18:34:34 INFO TaskSetManager: Starting task 0.0 in stage 37018.0 (TID 161, localhost, PROCESS_LOCAL, 5557 bytes)
15/06/03 18:34:34 INFO Executor: Running task 0.0 in stage 37018.0 (TID 161)
15/06/03 18:34:34 INFO TaskSetManager: Finished task 0.0 in stage 36897.0 (TID 160) in 31 ms on localhost (1/1)
15/06/03 18:34:34 INFO TaskSchedulerImpl: Removed TaskSet 36897.0, whose tasks have all completed, from pool
15/06/03 18:34:34 INFO DAGScheduler: Stage 36897 (mapPartitions at VertexRDDImpl.scala:218) finished in 0.032 s
15/06/03 18:34:34 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:34 INFO DAGScheduler: running: Set(Stage 37018)
15/06/03 18:34:34 INFO DAGScheduler: waiting: Set(Stage 37020, Stage 37019)
15/06/03 18:34:34 INFO DAGScheduler: failed: Set()
15/06/03 18:34:34 INFO DAGScheduler: Missing parents for Stage 37020: List(Stage 37019)
15/06/03 18:34:34 INFO DAGScheduler: Missing parents for Stage 37019: List(Stage 37018)
15/06/03 18:34:34 INFO BlockManager: Found block rdd_768_0 locally
15/06/03 18:34:34 INFO Executor: Finished task 0.0 in stage 37018.0 (TID 161). 2003 bytes result sent to driver
15/06/03 18:34:34 INFO TaskSetManager: Finished task 0.0 in stage 37018.0 (TID 161) in 14 ms on localhost (1/1)
15/06/03 18:34:34 INFO TaskSchedulerImpl: Removed TaskSet 37018.0, whose tasks have all completed, from pool
15/06/03 18:34:34 INFO DAGScheduler: Stage 37018 (mapPartitions at VertexRDDImpl.scala:222) finished in 0.035 s
15/06/03 18:34:34 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:34 INFO DAGScheduler: running: Set()
15/06/03 18:34:34 INFO DAGScheduler: waiting: Set(Stage 37020, Stage 37019)
15/06/03 18:34:34 INFO DAGScheduler: failed: Set()
15/06/03 18:34:34 INFO DAGScheduler: Missing parents for Stage 37020: List(Stage 37019)
15/06/03 18:34:34 INFO DAGScheduler: Missing parents for Stage 37019: List()
15/06/03 18:34:34 INFO DAGScheduler: Submitting Stage 37019 (GraphImpl.aggregateMessages - preAgg MapPartitionsRDD[782] at mapPartitions at GraphImpl.scala:235), which is now runnable
15/06/03 18:34:34 INFO MemoryStore: ensureFreeSpace(15744) called with curMem=543766, maxMem=559153152
15/06/03 18:34:34 INFO MemoryStore: Block broadcast_163 stored as values in memory (estimated size 15.4 KB, free 532.7 MB)
15/06/03 18:34:34 INFO MemoryStore: ensureFreeSpace(8046) called with curMem=559510, maxMem=559153152
15/06/03 18:34:34 INFO MemoryStore: Block broadcast_163_piece0 stored as bytes in memory (estimated size 7.9 KB, free 532.7 MB)
15/06/03 18:34:34 INFO BlockManagerInfo: Added broadcast_163_piece0 in memory on localhost:60836 (size: 7.9 KB, free: 533.1 MB)
15/06/03 18:34:34 INFO BlockManagerMaster: Updated info of block broadcast_163_piece0
15/06/03 18:34:34 INFO SparkContext: Created broadcast 163 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:34 INFO DAGScheduler: Submitting 1 missing tasks from Stage 37019 (GraphImpl.aggregateMessages - preAgg MapPartitionsRDD[782] at mapPartitions at GraphImpl.scala:235)
15/06/03 18:34:34 INFO TaskSchedulerImpl: Adding task set 37019.0 with 1 tasks
15/06/03 18:34:34 INFO TaskSetManager: Starting task 0.0 in stage 37019.0 (TID 162, localhost, PROCESS_LOCAL, 3534 bytes)
15/06/03 18:34:34 INFO Executor: Running task 0.0 in stage 37019.0 (TID 162)
15/06/03 18:34:34 INFO CacheManager: Partition rdd_776_0 not found, computing it
15/06/03 18:34:34 INFO BlockManager: Found block rdd_757_0 locally
15/06/03 18:34:34 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:34 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:34 INFO MemoryStore: ensureFreeSpace(2976) called with curMem=567556, maxMem=559153152
15/06/03 18:34:34 INFO MemoryStore: Block rdd_776_0 stored as values in memory (estimated size 2.9 KB, free 532.7 MB)
15/06/03 18:34:34 INFO BlockManagerInfo: Added rdd_776_0 in memory on localhost:60836 (size: 2.9 KB, free: 533.1 MB)
15/06/03 18:34:34 INFO BlockManagerMaster: Updated info of block rdd_776_0
15/06/03 18:34:34 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:34 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/06/03 18:34:34 INFO Executor: Finished task 0.0 in stage 37019.0 (TID 162). 2788 bytes result sent to driver
15/06/03 18:34:34 INFO TaskSetManager: Finished task 0.0 in stage 37019.0 (TID 162) in 15 ms on localhost (1/1)
15/06/03 18:34:34 INFO TaskSchedulerImpl: Removed TaskSet 37019.0, whose tasks have all completed, from pool
15/06/03 18:34:34 INFO DAGScheduler: Stage 37019 (mapPartitions at GraphImpl.scala:235) finished in 0.016 s
15/06/03 18:34:34 INFO DAGScheduler: looking for newly runnable stages
15/06/03 18:34:34 INFO DAGScheduler: running: Set()
15/06/03 18:34:34 INFO DAGScheduler: waiting: Set(Stage 37020)
15/06/03 18:34:34 INFO DAGScheduler: failed: Set()
15/06/03 18:34:34 INFO DAGScheduler: Missing parents for Stage 37020: List()
15/06/03 18:34:34 INFO DAGScheduler: Submitting Stage 37020 (MapPartitionsRDD[786] at map at VertexRDDImpl.scala:90), which is now runnable
15/06/03 18:34:34 INFO MemoryStore: ensureFreeSpace(27016) called with curMem=570532, maxMem=559153152
15/06/03 18:34:34 INFO MemoryStore: Block broadcast_164 stored as values in memory (estimated size 26.4 KB, free 532.7 MB)
15/06/03 18:34:34 INFO MemoryStore: ensureFreeSpace(11974) called with curMem=597548, maxMem=559153152
15/06/03 18:34:34 INFO MemoryStore: Block broadcast_164_piece0 stored as bytes in memory (estimated size 11.7 KB, free 532.7 MB)
15/06/03 18:34:34 INFO BlockManagerInfo: Added broadcast_164_piece0 in memory on localhost:60836 (size: 11.7 KB, free: 533.1 MB)
15/06/03 18:34:34 INFO BlockManagerMaster: Updated info of block broadcast_164_piece0
15/06/03 18:34:34 INFO SparkContext: Created broadcast 164 from broadcast at DAGScheduler.scala:839
15/06/03 18:34:34 INFO DAGScheduler: Submitting 1 missing tasks from Stage 37020 (MapPartitionsRDD[786] at map at VertexRDDImpl.scala:90)
15/06/03 18:34:34 INFO TaskSchedulerImpl: Adding task set 37020.0 with 1 tasks
15/06/03 18:34:34 INFO TaskSetManager: Starting task 0.0 in stage 37020.0 (TID 163, localhost, PROCESS_LOCAL, 5641 bytes)
15/06/03 18:34:34 INFO Executor: Running task 0.0 in stage 37020.0 (TID 163)
15/06/03 18:34:34 INFO CacheManager: Partition rdd_784_0 not found, computing it
15/06/03 18:34:34 INFO BlockManager: Found block rdd_770_0 locally
15/06/03 18:34:34 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks
15/06/03 18:34:34 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
15/06/03 18:34:34 INFO MemoryStore: ensureFreeSpace(1664) called with curMem=609522, maxMem=559153152
15/06/03 18:34:34 INFO MemoryStore: Block rdd_784_0 stored as values in memory (estimated size 1664.0 B, free 532.7 MB)
15/06/03 18:34:34 INFO BlockManagerInfo: Added rdd_784_0 in memory on localhost:60836 (size: 1664.0 B, free: 533.1 MB)
15/06/03 18:34:34 INFO BlockManagerMaster: Updated info of block rdd_784_0
15/06/03 18:34:34 INFO Executor: Finished task 0.0 in stage 37020.0 (TID 163). 2693 bytes result sent to driver
15/06/03 18:34:34 INFO TaskSetManager: Finished task 0.0 in stage 37020.0 (TID 163) in 11 ms on localhost (1/1)
15/06/03 18:34:34 INFO TaskSchedulerImpl: Removed TaskSet 37020.0, whose tasks have all completed, from pool
15/06/03 18:34:34 INFO DAGScheduler: Stage 37020 (reduce at VertexRDDImpl.scala:90) finished in 0.011 s
15/06/03 18:34:34 INFO DAGScheduler: Job 40 finished: reduce at VertexRDDImpl.scala:90, took 0.392264 s
15/06/03 18:34:34 INFO Pregel: Pregel finished iteration 38
15/06/03 18:34:34 INFO ZippedPartitionsRDD2: Removing RDD 765 from persistence list
15/06/03 18:34:34 INFO BlockManager: Removing RDD 765
15/06/03 18:34:34 INFO ZippedPartitionsRDD2: Removing RDD 768 from persistence list
15/06/03 18:34:34 INFO BlockManager: Removing block rdd_765_0
15/06/03 18:34:34 INFO MemoryStore: Block rdd_765_0 of size 1664 dropped from memory (free 558543630)
15/06/03 18:34:34 INFO BlockManager: Removing RDD 768
15/06/03 18:34:34 INFO BlockManager: Removing block rdd_768_0
15/06/03 18:34:34 INFO MemoryStore: Block rdd_768_0 of size 1544 dropped from memory (free 558545174)
15/06/03 18:34:34 INFO ZippedPartitionsRDD2: Removing RDD 751 from persistence list
15/06/03 18:34:34 INFO BlockManager: Removing RDD 751
15/06/03 18:34:34 INFO ZippedPartitionsRDD2: Removing RDD 757 from persistence list
15/06/03 18:34:34 INFO BlockManager: Removing block rdd_751_0
15/06/03 18:34:34 INFO MemoryStore: Block rdd_751_0 of size 1496 dropped from memory (free 558546670)
15/06/03 18:34:34 INFO BlockManager: Removing RDD 757
15/06/03 18:34:34 INFO BlockManager: Removing block rdd_757_0
15/06/03 18:34:34 INFO MemoryStore: Block rdd_757_0 of size 2976 dropped from memory (free 558549646)
15/06/03 18:34:34 INFO SparkContext: Starting job: reduce at VertexRDDImpl.scala:90
15/06/03 18:34:34 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 2 is 145 bytes
15/06/03 18:34:34 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 3 is 145 bytes
15/06/03 18:34:34 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 5 is 145 bytes
15/06/03 18:34:34 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 4 is 145 bytes
15/06/03 18:34:34 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 1 is 145 bytes
15/06/03 18:34:34 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 0 is 145 bytes
15/06/03 18:34:34 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 8 is 145 bytes
15/06/03 18:34:34 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 7 is 145 bytes
15/06/03 18:34:34 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 6 is 145 bytes
15/06/03 18:34:34 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 11 is 145 bytes
15/06/03 18:34:34 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 10 is 145 bytes
15/06/03 18:34:34 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 9 is 145 bytes
15/06/03 18:34:34 INFO MapOutputTrackerMaster: Size of output statuses for shuffl