Last active
December 17, 2015 16:29
-
-
Save jsianes/5639197 to your computer and use it in GitHub Desktop.
Execution result for WordCount Hadoop example
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| [ambari-qa@master-node ~]$ hadoop dfs -ls ./entrada | |
| Found 2 items | |
| drwx------ - ambari-qa hdfs 0 2013-05-23 18:41 /user/ambari-qa/entrada/entrada | |
| -rw------- 3 ambari-qa hdfs 568233984 2013-05-23 20:31 /user/ambari-qa/entrada/input1.txt | |
| [ambari-qa@master-node ~]$ hadoop jar /usr/lib/hadoop/hadoop-examples.jar wordcount ./entrada/input1.txt ./salida | |
| 13/05/23 20:38:15 INFO input.FileInputFormat: Total input paths to process : 1 | |
| 13/05/23 20:38:15 INFO lzo.GPLNativeCodeLoader: Loaded native gpl library | |
| 13/05/23 20:38:15 INFO lzo.LzoCodec: Successfully loaded & initialized native-lzo library [hadoop-lzo rev cf4e7cbf8ed0f0622504d008101c2729dc0c9ff3] | |
| 13/05/23 20:38:15 WARN snappy.LoadSnappy: Snappy native library is available | |
| 13/05/23 20:38:15 INFO util.NativeCodeLoader: Loaded the native-hadoop library | |
| 13/05/23 20:38:15 INFO snappy.LoadSnappy: Snappy native library loaded | |
| 13/05/23 20:38:15 INFO mapred.JobClient: Running job: job_201305232015_0004 | |
| 13/05/23 20:38:16 INFO mapred.JobClient: map 0% reduce 0% | |
| 13/05/23 20:38:29 INFO mapred.JobClient: map 20% reduce 0% | |
| 13/05/23 20:38:39 INFO mapred.JobClient: map 20% reduce 6% | |
| 13/05/23 20:39:46 INFO mapred.JobClient: map 21% reduce 6% | |
| 13/05/23 20:39:50 INFO mapred.JobClient: map 22% reduce 6% | |
| 13/05/23 20:39:54 INFO mapred.JobClient: map 23% reduce 6% | |
| 13/05/23 20:39:56 INFO mapred.JobClient: map 24% reduce 6% | |
| 13/05/23 20:39:57 INFO mapred.JobClient: map 26% reduce 6% | |
| 13/05/23 20:40:00 INFO mapred.JobClient: map 27% reduce 6% | |
| 13/05/23 20:40:03 INFO mapred.JobClient: map 31% reduce 6% | |
| 13/05/23 20:40:13 INFO mapred.JobClient: map 33% reduce 6% | |
| 13/05/23 20:40:15 INFO mapred.JobClient: map 34% reduce 6% | |
| 13/05/23 20:40:16 INFO mapred.JobClient: map 35% reduce 6% | |
| 13/05/23 20:40:18 INFO mapred.JobClient: map 40% reduce 6% | |
| 13/05/23 20:40:21 INFO mapred.JobClient: map 42% reduce 6% | |
| 13/05/23 20:40:28 INFO mapred.JobClient: map 44% reduce 6% | |
| 13/05/23 20:40:30 INFO mapred.JobClient: map 45% reduce 6% | |
| 13/05/23 20:40:31 INFO mapred.JobClient: map 46% reduce 6% | |
| 13/05/23 20:40:33 INFO mapred.JobClient: map 50% reduce 6% | |
| 13/05/23 20:40:36 INFO mapred.JobClient: map 52% reduce 6% | |
| 13/05/23 20:40:40 INFO mapred.JobClient: map 54% reduce 6% | |
| 13/05/23 20:40:43 INFO mapred.JobClient: map 55% reduce 6% | |
| 13/05/23 20:40:45 INFO mapred.JobClient: map 57% reduce 6% | |
| 13/05/23 20:40:46 INFO mapred.JobClient: map 58% reduce 6% | |
| 13/05/23 20:40:48 INFO mapred.JobClient: map 61% reduce 6% | |
| 13/05/23 20:40:49 INFO mapred.JobClient: map 62% reduce 6% | |
| 13/05/23 20:40:52 INFO mapred.JobClient: map 64% reduce 6% | |
| 13/05/23 20:40:55 INFO mapred.JobClient: map 65% reduce 6% | |
| 13/05/23 20:40:57 INFO mapred.JobClient: map 66% reduce 6% | |
| 13/05/23 20:40:58 INFO mapred.JobClient: map 68% reduce 6% | |
| 13/05/23 20:41:00 INFO mapred.JobClient: map 71% reduce 6% | |
| 13/05/23 20:41:01 INFO mapred.JobClient: map 72% reduce 6% | |
| 13/05/23 20:41:03 INFO mapred.JobClient: map 73% reduce 6% | |
| 13/05/23 20:41:04 INFO mapred.JobClient: map 74% reduce 6% | |
| 13/05/23 20:41:07 INFO mapred.JobClient: map 75% reduce 6% | |
| 13/05/23 20:41:11 INFO mapred.JobClient: map 76% reduce 6% | |
| 13/05/23 20:41:13 INFO mapred.JobClient: map 79% reduce 6% | |
| 13/05/23 20:41:14 INFO mapred.JobClient: map 80% reduce 6% | |
| 13/05/23 20:41:16 INFO mapred.JobClient: map 84% reduce 6% | |
| 13/05/23 20:41:17 INFO mapred.JobClient: map 85% reduce 6% | |
| 13/05/23 20:41:19 INFO mapred.JobClient: map 86% reduce 6% | |
| 13/05/23 20:41:25 INFO mapred.JobClient: map 88% reduce 6% | |
| 13/05/23 20:41:26 INFO mapred.JobClient: map 89% reduce 6% | |
| 13/05/23 20:41:28 INFO mapred.JobClient: map 91% reduce 6% | |
| 13/05/23 20:41:29 INFO mapred.JobClient: map 94% reduce 6% | |
| 13/05/23 20:41:32 INFO mapred.JobClient: map 95% reduce 6% | |
| 13/05/23 20:41:37 INFO mapred.JobClient: map 97% reduce 6% | |
| 13/05/23 20:41:38 INFO mapred.JobClient: map 98% reduce 6% | |
| 13/05/23 20:41:41 INFO mapred.JobClient: map 100% reduce 6% | |
| 13/05/23 20:41:43 INFO mapred.JobClient: map 100% reduce 13% | |
| 13/05/23 20:41:47 INFO mapred.JobClient: map 100% reduce 100% | |
| 13/05/23 20:41:48 INFO mapred.JobClient: Job complete: job_201305232015_0004 | |
| 13/05/23 20:41:48 INFO mapred.JobClient: Counters: 30 | |
| 13/05/23 20:41:48 INFO mapred.JobClient: Job Counters | |
| 13/05/23 20:41:48 INFO mapred.JobClient: Launched reduce tasks=1 | |
| 13/05/23 20:41:48 INFO mapred.JobClient: SLOTS_MILLIS_MAPS=808592 | |
| 13/05/23 20:41:48 INFO mapred.JobClient: Total time spent by all reduces waiting after reserving slots (ms)=0 | |
| 13/05/23 20:41:48 INFO mapred.JobClient: Total time spent by all maps waiting after reserving slots (ms)=0 | |
| 13/05/23 20:41:48 INFO mapred.JobClient: Rack-local map tasks=4 | |
| 13/05/23 20:41:48 INFO mapred.JobClient: Launched map tasks=5 | |
| 13/05/23 20:41:48 INFO mapred.JobClient: Data-local map tasks=1 | |
| 13/05/23 20:41:48 INFO mapred.JobClient: SLOTS_MILLIS_REDUCES=195476 | |
| 13/05/23 20:41:48 INFO mapred.JobClient: File Output Format Counters | |
| 13/05/23 20:41:48 INFO mapred.JobClient: Bytes Written=67079 | |
| 13/05/23 20:41:48 INFO mapred.JobClient: FileSystemCounters | |
| 13/05/23 20:41:48 INFO mapred.JobClient: FILE_BYTES_READ=2856148 | |
| 13/05/23 20:41:48 INFO mapred.JobClient: HDFS_BYTES_READ=568758942 | |
| 13/05/23 20:41:48 INFO mapred.JobClient: FILE_BYTES_WRITTEN=3649388 | |
| 13/05/23 20:41:48 INFO mapred.JobClient: HDFS_BYTES_WRITTEN=67079 | |
| 13/05/23 20:41:48 INFO mapred.JobClient: File Input Format Counters | |
| 13/05/23 20:41:48 INFO mapred.JobClient: Bytes Read=568758272 | |
| 13/05/23 20:41:48 INFO mapred.JobClient: Map-Reduce Framework | |
| 13/05/23 20:41:48 INFO mapred.JobClient: Map output materialized bytes=428427 | |
| 13/05/23 20:41:48 INFO mapred.JobClient: Map input records=6433794 | |
| 13/05/23 20:41:48 INFO mapred.JobClient: Reduce shuffle bytes=428427 | |
| 13/05/23 20:41:48 INFO mapred.JobClient: Spilled Records=173561 | |
| 13/05/23 20:41:48 INFO mapred.JobClient: Map output bytes=871502402 | |
| 13/05/23 20:41:48 INFO mapred.JobClient: CPU time spent (ms)=172100 | |
| 13/05/23 20:41:48 INFO mapred.JobClient: Total committed heap usage (bytes)=2202488832 | |
| 13/05/23 20:41:48 INFO mapred.JobClient: Combine input records=76638558 | |
| 13/05/23 20:41:48 INFO mapred.JobClient: SPLIT_RAW_BYTES=670 | |
| 13/05/23 20:41:48 INFO mapred.JobClient: Reduce input records=22639 | |
| 13/05/23 20:41:48 INFO mapred.JobClient: Reduce input groups=3774 | |
| 13/05/23 20:41:48 INFO mapred.JobClient: Combine output records=143375 | |
| 13/05/23 20:41:48 INFO mapred.JobClient: Physical memory (bytes) snapshot=1995444224 | |
| 13/05/23 20:41:48 INFO mapred.JobClient: Reduce output records=3774 | |
| 13/05/23 20:41:48 INFO mapred.JobClient: Virtual memory (bytes) snapshot=7960113152 | |
| 13/05/23 20:41:48 INFO mapred.JobClient: Map output records=76517822 | |
| [ambari-qa@master-node ~]$ hadoop dfs -ls ./salida | |
| Found 1 items | |
| -rw------- 3 ambari-qa hdfs 67079 2013-05-23 20:41 /user/ambari-qa/salida/part-r-00000 | |
| [ambari-qa@master-node ~]$ hadoop dfs -cat ./salida/part-r-00000 > salida.txt | |
| [ambari-qa@master-node ~]$ head -10 salida.txt | |
| "pci=nocrs" 5910 | |
| #0 10409 | |
| #1 29277 | |
| #2 10409 | |
| #22 4499 | |
| #3 10409 | |
| #4 10409 | |
| #5 10409 | |
| #6 5910 | |
| #7 5910 | |
| [ambari-qa@master-node ~]$ |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment