Created
November 18, 2012 11:45
-
-
Save yosshi/4104729 to your computer and use it in GitHub Desktop.
はじめてのHadoop 円周率を計算 Estimated value of Pi is 3.14150400000000000000
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| % hadoop jar /usr/local/Cellar/hadoop/1.0.4/libexec/hadoop-examples-1.0.4.jar pi 10 200000 | |
| Number of Maps = 10 | |
| Samples per Map = 200000 | |
| 12/11/18 20:43:33 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable | |
| Wrote input for Map #0 | |
| Wrote input for Map #1 | |
| Wrote input for Map #2 | |
| Wrote input for Map #3 | |
| Wrote input for Map #4 | |
| Wrote input for Map #5 | |
| Wrote input for Map #6 | |
| Wrote input for Map #7 | |
| Wrote input for Map #8 | |
| Wrote input for Map #9 | |
| Starting Job | |
| 12/11/18 20:43:33 INFO mapred.FileInputFormat: Total input paths to process : 10 | |
| 12/11/18 20:43:33 INFO mapred.JobClient: Running job: job_local_0001 | |
| 12/11/18 20:43:33 INFO mapred.Task: Using ResourceCalculatorPlugin : null | |
| 12/11/18 20:43:33 INFO mapred.MapTask: numReduceTasks: 1 | |
| 12/11/18 20:43:33 INFO mapred.MapTask: io.sort.mb = 100 | |
| 12/11/18 20:43:34 INFO mapred.MapTask: data buffer = 79691776/99614720 | |
| 12/11/18 20:43:34 INFO mapred.MapTask: record buffer = 262144/327680 | |
| 12/11/18 20:43:34 INFO mapred.MapTask: Starting flush of map output | |
| 12/11/18 20:43:34 INFO mapred.MapTask: Finished spill 0 | |
| 12/11/18 20:43:34 INFO mapred.Task: Task:attempt_local_0001_m_000000_0 is done. And is in the process of commiting | |
| 12/11/18 20:43:34 INFO mapred.JobClient: map 0% reduce 0% | |
| 12/11/18 20:43:36 INFO mapred.LocalJobRunner: Generated 200000 samples. | |
| 12/11/18 20:43:36 INFO mapred.Task: Task 'attempt_local_0001_m_000000_0' done. | |
| 12/11/18 20:43:36 INFO mapred.Task: Using ResourceCalculatorPlugin : null | |
| 12/11/18 20:43:36 INFO mapred.MapTask: numReduceTasks: 1 | |
| 12/11/18 20:43:36 INFO mapred.MapTask: io.sort.mb = 100 | |
| 12/11/18 20:43:36 INFO mapred.MapTask: data buffer = 79691776/99614720 | |
| 12/11/18 20:43:36 INFO mapred.MapTask: record buffer = 262144/327680 | |
| 12/11/18 20:43:36 INFO mapred.MapTask: Starting flush of map output | |
| 12/11/18 20:43:36 INFO mapred.MapTask: Finished spill 0 | |
| 12/11/18 20:43:37 INFO mapred.Task: Task:attempt_local_0001_m_000001_0 is done. And is in the process of commiting | |
| 12/11/18 20:43:37 INFO mapred.JobClient: map 100% reduce 0% | |
| 12/11/18 20:43:39 INFO mapred.LocalJobRunner: Generated 200000 samples. | |
| 12/11/18 20:43:39 INFO mapred.Task: Task 'attempt_local_0001_m_000001_0' done. | |
| 12/11/18 20:43:39 INFO mapred.Task: Using ResourceCalculatorPlugin : null | |
| 12/11/18 20:43:39 INFO mapred.MapTask: numReduceTasks: 1 | |
| 12/11/18 20:43:39 INFO mapred.MapTask: io.sort.mb = 100 | |
| 12/11/18 20:43:39 INFO mapred.MapTask: data buffer = 79691776/99614720 | |
| 12/11/18 20:43:39 INFO mapred.MapTask: record buffer = 262144/327680 | |
| 12/11/18 20:43:39 INFO mapred.MapTask: Starting flush of map output | |
| 12/11/18 20:43:39 INFO mapred.MapTask: Finished spill 0 | |
| 12/11/18 20:43:39 INFO mapred.Task: Task:attempt_local_0001_m_000002_0 is done. And is in the process of commiting | |
| 12/11/18 20:43:42 INFO mapred.LocalJobRunner: Generated 200000 samples. | |
| 12/11/18 20:43:42 INFO mapred.Task: Task 'attempt_local_0001_m_000002_0' done. | |
| 12/11/18 20:43:42 INFO mapred.Task: Using ResourceCalculatorPlugin : null | |
| 12/11/18 20:43:42 INFO mapred.MapTask: numReduceTasks: 1 | |
| 12/11/18 20:43:42 INFO mapred.MapTask: io.sort.mb = 100 | |
| 12/11/18 20:43:42 INFO mapred.MapTask: data buffer = 79691776/99614720 | |
| 12/11/18 20:43:42 INFO mapred.MapTask: record buffer = 262144/327680 | |
| 12/11/18 20:43:42 INFO mapred.MapTask: Starting flush of map output | |
| 12/11/18 20:43:42 INFO mapred.MapTask: Finished spill 0 | |
| 12/11/18 20:43:42 INFO mapred.Task: Task:attempt_local_0001_m_000003_0 is done. And is in the process of commiting | |
| 12/11/18 20:43:45 INFO mapred.LocalJobRunner: Generated 200000 samples. | |
| 12/11/18 20:43:45 INFO mapred.Task: Task 'attempt_local_0001_m_000003_0' done. | |
| 12/11/18 20:43:45 INFO mapred.Task: Using ResourceCalculatorPlugin : null | |
| 12/11/18 20:43:45 INFO mapred.MapTask: numReduceTasks: 1 | |
| 12/11/18 20:43:45 INFO mapred.MapTask: io.sort.mb = 100 | |
| 12/11/18 20:43:45 INFO mapred.MapTask: data buffer = 79691776/99614720 | |
| 12/11/18 20:43:45 INFO mapred.MapTask: record buffer = 262144/327680 | |
| 12/11/18 20:43:45 INFO mapred.MapTask: Starting flush of map output | |
| 12/11/18 20:43:45 INFO mapred.MapTask: Finished spill 0 | |
| 12/11/18 20:43:45 INFO mapred.Task: Task:attempt_local_0001_m_000004_0 is done. And is in the process of commiting | |
| 12/11/18 20:43:48 INFO mapred.LocalJobRunner: Generated 200000 samples. | |
| 12/11/18 20:43:48 INFO mapred.Task: Task 'attempt_local_0001_m_000004_0' done. | |
| 12/11/18 20:43:48 INFO mapred.Task: Using ResourceCalculatorPlugin : null | |
| 12/11/18 20:43:48 INFO mapred.MapTask: numReduceTasks: 1 | |
| 12/11/18 20:43:48 INFO mapred.MapTask: io.sort.mb = 100 | |
| 12/11/18 20:43:48 INFO mapred.MapTask: data buffer = 79691776/99614720 | |
| 12/11/18 20:43:48 INFO mapred.MapTask: record buffer = 262144/327680 | |
| 12/11/18 20:43:49 INFO mapred.MapTask: Starting flush of map output | |
| 12/11/18 20:43:49 INFO mapred.MapTask: Finished spill 0 | |
| 12/11/18 20:43:49 INFO mapred.Task: Task:attempt_local_0001_m_000005_0 is done. And is in the process of commiting | |
| 12/11/18 20:43:51 INFO mapred.LocalJobRunner: Generated 200000 samples. | |
| 12/11/18 20:43:51 INFO mapred.Task: Task 'attempt_local_0001_m_000005_0' done. | |
| 12/11/18 20:43:51 INFO mapred.Task: Using ResourceCalculatorPlugin : null | |
| 12/11/18 20:43:51 INFO mapred.MapTask: numReduceTasks: 1 | |
| 12/11/18 20:43:51 INFO mapred.MapTask: io.sort.mb = 100 | |
| 12/11/18 20:43:51 INFO mapred.MapTask: data buffer = 79691776/99614720 | |
| 12/11/18 20:43:51 INFO mapred.MapTask: record buffer = 262144/327680 | |
| 12/11/18 20:43:51 INFO mapred.MapTask: Starting flush of map output | |
| 12/11/18 20:43:51 INFO mapred.MapTask: Finished spill 0 | |
| 12/11/18 20:43:51 INFO mapred.Task: Task:attempt_local_0001_m_000006_0 is done. And is in the process of commiting | |
| 12/11/18 20:43:54 INFO mapred.LocalJobRunner: Generated 200000 samples. | |
| 12/11/18 20:43:54 INFO mapred.Task: Task 'attempt_local_0001_m_000006_0' done. | |
| 12/11/18 20:43:54 INFO mapred.Task: Using ResourceCalculatorPlugin : null | |
| 12/11/18 20:43:54 INFO mapred.MapTask: numReduceTasks: 1 | |
| 12/11/18 20:43:54 INFO mapred.MapTask: io.sort.mb = 100 | |
| 12/11/18 20:43:54 INFO mapred.MapTask: data buffer = 79691776/99614720 | |
| 12/11/18 20:43:54 INFO mapred.MapTask: record buffer = 262144/327680 | |
| 12/11/18 20:43:54 INFO mapred.MapTask: Starting flush of map output | |
| 12/11/18 20:43:54 INFO mapred.MapTask: Finished spill 0 | |
| 12/11/18 20:43:54 INFO mapred.Task: Task:attempt_local_0001_m_000007_0 is done. And is in the process of commiting | |
| 12/11/18 20:43:57 INFO mapred.LocalJobRunner: Generated 200000 samples. | |
| 12/11/18 20:43:57 INFO mapred.Task: Task 'attempt_local_0001_m_000007_0' done. | |
| 12/11/18 20:43:57 INFO mapred.Task: Using ResourceCalculatorPlugin : null | |
| 12/11/18 20:43:57 INFO mapred.MapTask: numReduceTasks: 1 | |
| 12/11/18 20:43:57 INFO mapred.MapTask: io.sort.mb = 100 | |
| 12/11/18 20:43:57 INFO mapred.MapTask: data buffer = 79691776/99614720 | |
| 12/11/18 20:43:57 INFO mapred.MapTask: record buffer = 262144/327680 | |
| 12/11/18 20:43:57 INFO mapred.MapTask: Starting flush of map output | |
| 12/11/18 20:43:57 INFO mapred.MapTask: Finished spill 0 | |
| 12/11/18 20:43:57 INFO mapred.Task: Task:attempt_local_0001_m_000008_0 is done. And is in the process of commiting | |
| 12/11/18 20:44:00 INFO mapred.LocalJobRunner: Generated 200000 samples. | |
| 12/11/18 20:44:00 INFO mapred.Task: Task 'attempt_local_0001_m_000008_0' done. | |
| 12/11/18 20:44:00 INFO mapred.Task: Using ResourceCalculatorPlugin : null | |
| 12/11/18 20:44:00 INFO mapred.MapTask: numReduceTasks: 1 | |
| 12/11/18 20:44:00 INFO mapred.MapTask: io.sort.mb = 100 | |
| 12/11/18 20:44:00 INFO mapred.MapTask: data buffer = 79691776/99614720 | |
| 12/11/18 20:44:00 INFO mapred.MapTask: record buffer = 262144/327680 | |
| 12/11/18 20:44:00 INFO mapred.MapTask: Starting flush of map output | |
| 12/11/18 20:44:00 INFO mapred.MapTask: Finished spill 0 | |
| 12/11/18 20:44:00 INFO mapred.Task: Task:attempt_local_0001_m_000009_0 is done. And is in the process of commiting | |
| 12/11/18 20:44:03 INFO mapred.LocalJobRunner: Generated 200000 samples. | |
| 12/11/18 20:44:03 INFO mapred.Task: Task 'attempt_local_0001_m_000009_0' done. | |
| 12/11/18 20:44:03 INFO mapred.Task: Using ResourceCalculatorPlugin : null | |
| 12/11/18 20:44:03 INFO mapred.LocalJobRunner: | |
| 12/11/18 20:44:03 INFO mapred.Merger: Merging 10 sorted segments | |
| 12/11/18 20:44:03 INFO mapred.Merger: Down to the last merge-pass, with 10 segments left of total size: 240 bytes | |
| 12/11/18 20:44:03 INFO mapred.LocalJobRunner: | |
| 12/11/18 20:44:03 INFO mapred.Task: Task:attempt_local_0001_r_000000_0 is done. And is in the process of commiting | |
| 12/11/18 20:44:03 INFO mapred.LocalJobRunner: | |
| 12/11/18 20:44:03 INFO mapred.Task: Task attempt_local_0001_r_000000_0 is allowed to commit now | |
| 12/11/18 20:44:03 INFO mapred.FileOutputCommitter: Saved output of task 'attempt_local_0001_r_000000_0' to file:/Users/yoshiyuki_miyagi/PiEstimator_TMP_3_141592654/out | |
| 12/11/18 20:44:06 INFO mapred.LocalJobRunner: reduce > reduce | |
| 12/11/18 20:44:06 INFO mapred.Task: Task 'attempt_local_0001_r_000000_0' done. | |
| 12/11/18 20:44:07 INFO mapred.JobClient: map 100% reduce 100% | |
| 12/11/18 20:44:07 INFO mapred.JobClient: Job complete: job_local_0001 | |
| 12/11/18 20:44:07 INFO mapred.JobClient: Counters: 18 | |
| 12/11/18 20:44:07 INFO mapred.JobClient: File Input Format Counters | |
| 12/11/18 20:44:07 INFO mapred.JobClient: Bytes Read=1300 | |
| 12/11/18 20:44:07 INFO mapred.JobClient: File Output Format Counters | |
| 12/11/18 20:44:07 INFO mapred.JobClient: Bytes Written=109 | |
| 12/11/18 20:44:07 INFO mapred.JobClient: FileSystemCounters | |
| 12/11/18 20:44:07 INFO mapred.JobClient: FILE_BYTES_READ=1644730 | |
| 12/11/18 20:44:07 INFO mapred.JobClient: FILE_BYTES_WRITTEN=1969520 | |
| 12/11/18 20:44:07 INFO mapred.JobClient: Map-Reduce Framework | |
| 12/11/18 20:44:07 INFO mapred.JobClient: Map output materialized bytes=280 | |
| 12/11/18 20:44:07 INFO mapred.JobClient: Map input records=10 | |
| 12/11/18 20:44:07 INFO mapred.JobClient: Reduce shuffle bytes=0 | |
| 12/11/18 20:44:07 INFO mapred.JobClient: Spilled Records=40 | |
| 12/11/18 20:44:07 INFO mapred.JobClient: Map output bytes=180 | |
| 12/11/18 20:44:07 INFO mapred.JobClient: Total committed heap usage (bytes)=2047193088 | |
| 12/11/18 20:44:07 INFO mapred.JobClient: Map input bytes=240 | |
| 12/11/18 20:44:07 INFO mapred.JobClient: SPLIT_RAW_BYTES=1180 | |
| 12/11/18 20:44:07 INFO mapred.JobClient: Combine input records=0 | |
| 12/11/18 20:44:07 INFO mapred.JobClient: Reduce input records=20 | |
| 12/11/18 20:44:07 INFO mapred.JobClient: Reduce input groups=20 | |
| 12/11/18 20:44:07 INFO mapred.JobClient: Combine output records=0 | |
| 12/11/18 20:44:07 INFO mapred.JobClient: Reduce output records=0 | |
| 12/11/18 20:44:07 INFO mapred.JobClient: Map output records=20 | |
| Job Finished in 34.302 seconds | |
| Estimated value of Pi is 3.14150400000000000000 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment