Skip to content

Instantly share code, notes, and snippets.

@spmallette
Created February 26, 2014 19:28
Show Gist options
  • Save spmallette/9236621 to your computer and use it in GitHub Desktop.
Save spmallette/9236621 to your computer and use it in GitHub Desktop.
220_0018_r_000001_0 Scheduled 1 outputs (0 slow hosts and0 dup hosts)
2014-02-26 17:51:52,055 INFO org.apache.hadoop.mapred.ReduceTask: attempt_201402242220_0018_r_000001_0 Scheduled 1 outputs (0 slow hosts and0 dup hosts)
2014-02-26 17:51:52,061 INFO org.apache.hadoop.mapred.ReduceTask: attempt_201402242220_0018_r_000001_0 Scheduled 1 outputs (0 slow hosts and0 dup hosts)
2014-02-26 17:51:53,006 INFO org.apache.hadoop.mapred.ReduceTask: GetMapEventsThread exiting
2014-02-26 17:51:53,007 INFO org.apache.hadoop.mapred.ReduceTask: getMapsEventsThread joined.
2014-02-26 17:51:53,007 INFO org.apache.hadoop.mapred.ReduceTask: Closed ram manager
2014-02-26 17:52:28,703 INFO org.apache.hadoop.mapred.ReduceTask: attempt_201402242220_0018_r_000001_0 Merge of the 461 files in-memory complete. Local file is /var/lib/hadoop/mapred/taskTracker/hduser/jobcache/job_201402242220_0018/attempt_201402242220_0018_r_000001_0/output/map_5827.out of size 218666478
2014-02-26 17:54:35,074 INFO org.apache.hadoop.mapred.ReduceTask: attempt_201402242220_0018_r_000001_0 Finished merging 10 map output files on disk of total-size 2195889411. Local output file is file:/var/lib/hadoop/mapred/taskTracker/hduser/jobcache/job_201402242220_0018/attempt_201402242220_0018_r_000001_0/output/map_2808.out.merged of size 2164168126
2014-02-26 17:54:35,075 INFO org.apache.hadoop.mapred.ReduceTask: Interleaved on-disk merge complete: 23 files left.
2014-02-26 17:54:35,075 INFO org.apache.hadoop.mapred.ReduceTask: In-memory merge complete: 125 files left.
2014-02-26 17:54:35,884 INFO org.apache.hadoop.mapred.ReduceTask: Keeping 125 segments, 248870550 bytes in memory for intermediate, on-disk merge
2014-02-26 17:54:35,886 INFO org.apache.hadoop.mapred.ReduceTask: Merging 23 files, 9270556900 bytes from disk
2014-02-26 17:54:35,886 INFO org.apache.hadoop.mapred.ReduceTask: Merging 0 segments, 0 bytes from memory into reduce
2014-02-26 17:54:35,887 INFO org.apache.hadoop.mapred.Merger: Merging 148 sorted segments
2014-02-26 17:54:36,010 INFO org.apache.hadoop.mapred.Merger: Merging 130 intermediate segments out of a total of 148
2014-02-26 17:58:42,127 INFO org.apache.hadoop.mapred.Merger: Merging 10 intermediate segments out of a total of 19
2014-02-26 18:09:29,908 INFO org.apache.hadoop.mapred.Merger: Down to the last merge-pass, with 10 segments left of total size: 29567315443 bytes
2014-02-26 19:26:42,506 INFO org.apache.hadoop.mapred.TaskLogsTruncater: Initializing logs' truncater with mapRetainSize=-1 and reduceRetainSize=-1
2014-02-26 19:26:42,523 INFO org.apache.hadoop.io.nativeio.NativeIO: Initialized cache for UID to User mapping with a cache timeout of 14400 seconds.
2014-02-26 19:26:42,523 INFO org.apache.hadoop.io.nativeio.NativeIO: Got UserName mapred for UID 202 from the native implementation
2014-02-26 19:26:42,525 WARN org.apache.hadoop.mapred.Child: Error running child
java.lang.ArrayIndexOutOfBoundsException: 110
at org.apache.hadoop.io.GenericWritable.readFields(GenericWritable.java:120)
at com.thinkaurelius.faunus.Holder.readFields(Holder.java:76)
at org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:67)
at org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:40)
at org.apache.hadoop.mapreduce.ReduceContext.nextKeyValue(ReduceContext.java:117)
at org.apache.hadoop.mapreduce.ReduceContext.nextKey(ReduceContext.java:92)
at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:176)
at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:649)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:418)
at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
at org.apache.hadoop.mapred.Child.main(Child.java:249)
2014-02-26 19:26:42,528 INFO org.apache.hadoop.mapred.Task: Runnning cleanup for the task
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment