Created
May 14, 2012 20:51
-
-
Save ceteri/2696821 to your computer and use it in GitHub Desktop.
Word Count app run in Multitool
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
12/05/14 14:42:57 INFO multitool.Main: key: source | |
12/05/14 14:42:58 INFO multitool.Main: key: expr | |
12/05/14 14:42:58 INFO multitool.Main: key: gen | |
12/05/14 14:42:58 INFO multitool.Main: key: group | |
12/05/14 14:42:58 INFO multitool.Main: key: count | |
12/05/14 14:42:58 INFO multitool.Main: key: group | |
12/05/14 14:42:58 INFO multitool.Main: key: sink | |
12/05/14 14:43:01 INFO util.HadoopUtil: resolving application jar from found main method on: multitool.Main | |
12/05/14 14:43:01 INFO planner.HadoopPlanner: using application jar: /Users/paco/src/concur/cascading.multitool/./build/multitool.jar | |
12/05/14 14:43:01 INFO property.AppProps: using app.id: 1C9D0188E5018B980067AAC12AE43BBA | |
12/05/14 14:43:13 INFO util.Version: Concurrent, Inc - Cascading 2.0.0-wip-301 [null] | |
12/05/14 14:43:13 INFO flow.Flow: [multitool] starting | |
12/05/14 14:43:13 INFO flow.Flow: [multitool] source: Hfs["TextLine[[0:1]->[ALL]]"]["input.txt"]"] | |
12/05/14 14:43:13 INFO flow.Flow: [multitool] sink: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["output"]"] | |
12/05/14 14:43:13 INFO flow.Flow: [multitool] parallel execution is enabled: false | |
12/05/14 14:43:13 INFO flow.Flow: [multitool] starting jobs: 2 | |
12/05/14 14:43:13 INFO flow.Flow: [multitool] allocating threads: 1 | |
12/05/14 14:43:13 INFO flow.FlowStep: [multitool] starting step: (1/2) TempHfs["SequenceFile[[0, 'count']]"][multitool/65124/] | |
12/05/14 14:43:14 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable | |
12/05/14 14:43:14 INFO mapred.FileInputFormat: Total input paths to process : 1 | |
12/05/14 14:43:15 INFO flow.FlowStep: [multitool] submitted hadoop job: job_local_0001 | |
12/05/14 14:43:18 INFO mapred.Task: Using ResourceCalculatorPlugin : null | |
12/05/14 14:43:18 INFO io.MultiInputSplit: current split input path: file:/Users/paco/src/concur/cascading.multitool/input.txt | |
12/05/14 14:43:18 INFO mapred.MapTask: numReduceTasks: 1 | |
12/05/14 14:43:18 INFO mapred.MapTask: io.sort.mb = 100 | |
12/05/14 14:43:26 INFO mapred.MapTask: data buffer = 79691776/99614720 | |
12/05/14 14:43:26 INFO mapred.MapTask: record buffer = 262144/327680 | |
12/05/14 14:43:27 INFO hadoop.FlowMapper: sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["input.txt"]"] | |
12/05/14 14:43:27 INFO hadoop.FlowMapper: sinking to: GroupBy(multitool)[by:[{1}:0]] | |
12/05/14 14:43:27 INFO mapred.LocalJobRunner: file:/Users/paco/src/concur/cascading.multitool/input.txt:0+726 | |
12/05/14 14:43:28 INFO mapred.MapTask: Starting flush of map output | |
12/05/14 14:43:28 INFO mapred.MapTask: Finished spill 0 | |
12/05/14 14:43:28 INFO mapred.Task: Task:attempt_local_0001_m_000000_0 is done. And is in the process of commiting | |
12/05/14 14:43:30 INFO mapred.LocalJobRunner: file:/Users/paco/src/concur/cascading.multitool/input.txt:0+726 | |
12/05/14 14:43:30 INFO mapred.LocalJobRunner: file:/Users/paco/src/concur/cascading.multitool/input.txt:0+726 | |
12/05/14 14:43:30 INFO mapred.Task: Task 'attempt_local_0001_m_000000_0' done. | |
12/05/14 14:43:30 INFO mapred.Task: Using ResourceCalculatorPlugin : null | |
12/05/14 14:43:30 INFO mapred.LocalJobRunner: | |
12/05/14 14:43:30 INFO mapred.Merger: Merging 1 sorted segments | |
12/05/14 14:43:30 INFO mapred.Merger: Down to the last merge-pass, with 1 segments left of total size: 1889 bytes | |
12/05/14 14:43:30 INFO mapred.LocalJobRunner: | |
12/05/14 14:43:30 INFO hadoop.FlowReducer: sourcing from: GroupBy(multitool)[by:[{1}:0]] | |
12/05/14 14:43:30 INFO hadoop.FlowReducer: sinking to: TempHfs["SequenceFile[[0, 'count']]"][multitool/65124/] | |
12/05/14 14:43:30 INFO mapred.Task: Task:attempt_local_0001_r_000000_0 is done. And is in the process of commiting | |
12/05/14 14:43:30 INFO mapred.LocalJobRunner: | |
12/05/14 14:43:30 INFO mapred.Task: Task attempt_local_0001_r_000000_0 is allowed to commit now | |
12/05/14 14:43:30 INFO mapred.FileOutputCommitter: Saved output of task 'attempt_local_0001_r_000000_0' to file:/tmp/hadoop-paco/multitool_65124_58940C7E03F8F5276633231AB59A4D84 | |
12/05/14 14:43:33 INFO mapred.LocalJobRunner: reduce > reduce | |
12/05/14 14:43:33 INFO mapred.Task: Task 'attempt_local_0001_r_000000_0' done. | |
12/05/14 14:43:34 INFO flow.FlowStep: [multitool] starting step: (2/2) Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["output"]"] | |
12/05/14 14:43:34 INFO mapred.FileInputFormat: Total input paths to process : 1 | |
12/05/14 14:43:34 INFO mapred.Task: Using ResourceCalculatorPlugin : null | |
12/05/14 14:43:34 INFO flow.FlowStep: [multitool] submitted hadoop job: job_local_0002 | |
12/05/14 14:43:34 INFO io.MultiInputSplit: current split input path: file:/tmp/hadoop-paco/multitool_65124_58940C7E03F8F5276633231AB59A4D84/part-00000 | |
12/05/14 14:43:34 INFO mapred.MapTask: numReduceTasks: 1 | |
12/05/14 14:43:34 INFO mapred.MapTask: io.sort.mb = 100 | |
12/05/14 14:43:42 INFO mapred.MapTask: data buffer = 79691776/99614720 | |
12/05/14 14:43:42 INFO mapred.MapTask: record buffer = 262144/327680 | |
12/05/14 14:43:42 INFO hadoop.FlowMapper: sourcing from: TempHfs["SequenceFile[[0, 'count']]"][multitool/65124/] | |
12/05/14 14:43:42 INFO hadoop.FlowMapper: sinking to: GroupBy(multitool)[by:[{1}:1]] | |
12/05/14 14:43:42 INFO mapred.MapTask: Starting flush of map output | |
12/05/14 14:43:42 INFO mapred.MapTask: Finished spill 0 | |
12/05/14 14:43:42 INFO mapred.Task: Task:attempt_local_0002_m_000000_0 is done. And is in the process of commiting | |
12/05/14 14:43:45 INFO mapred.LocalJobRunner: file:/tmp/hadoop-paco/multitool_65124_58940C7E03F8F5276633231AB59A4D84/part-00000:0+2190 | |
12/05/14 14:43:45 INFO mapred.LocalJobRunner: file:/tmp/hadoop-paco/multitool_65124_58940C7E03F8F5276633231AB59A4D84/part-00000:0+2190 | |
12/05/14 14:43:45 INFO mapred.Task: Task 'attempt_local_0002_m_000000_0' done. | |
12/05/14 14:43:45 INFO mapred.Task: Using ResourceCalculatorPlugin : null | |
12/05/14 14:43:45 INFO mapred.LocalJobRunner: | |
12/05/14 14:43:45 INFO mapred.Merger: Merging 1 sorted segments | |
12/05/14 14:43:45 INFO mapred.Merger: Down to the last merge-pass, with 1 segments left of total size: 2698 bytes | |
12/05/14 14:43:45 INFO mapred.LocalJobRunner: | |
12/05/14 14:43:45 INFO hadoop.FlowReducer: sourcing from: GroupBy(multitool)[by:[{1}:1]] | |
12/05/14 14:43:45 INFO hadoop.FlowReducer: sinking to: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["output"]"] | |
12/05/14 14:43:45 INFO mapred.Task: Task:attempt_local_0002_r_000000_0 is done. And is in the process of commiting | |
12/05/14 14:43:45 INFO mapred.LocalJobRunner: | |
12/05/14 14:43:45 INFO mapred.Task: Task attempt_local_0002_r_000000_0 is allowed to commit now | |
12/05/14 14:43:45 INFO mapred.FileOutputCommitter: Saved output of task 'attempt_local_0002_r_000000_0' to file:/Users/paco/src/concur/cascading.multitool/output | |
12/05/14 14:43:48 INFO mapred.LocalJobRunner: reduce > reduce | |
12/05/14 14:43:48 INFO mapred.Task: Task 'attempt_local_0002_r_000000_0' done. | |
12/05/14 14:43:52 INFO util.Hadoop18TapUtil: deleting temp path output/_temporary |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment