Created
September 28, 2017 09:25
-
-
Save vinodnerella/ac558eeb3d113c220e23bed581580ac9 to your computer and use it in GitHub Desktop.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
[testkerberos@cdh000 ~]$ hadoop jar /opt/cloudera/parcels/CDH-5.12.1-1.cdh5.12.1.p0.3/jars/hadoop-examples.jar pi 10 1000 | |
Number of Maps = 10 | |
Samples per Map = 1000 | |
Wrote input for Map #0 | |
Wrote input for Map #1 | |
Wrote input for Map #2 | |
Wrote input for Map #3 | |
Wrote input for Map #4 | |
Wrote input for Map #5 | |
Wrote input for Map #6 | |
Wrote input for Map #7 | |
Wrote input for Map #8 | |
Wrote input for Map #9 | |
Starting Job | |
17/09/28 06:56:02 INFO hdfs.DFSClient: Created token for testkerberos: HDFS_DELEGATION_TOKEN owner=testkerberos@MYITVERSITY.COM, renewer=yarn, realUser=, issueDate=1506581762808, maxDate=1507186562808, sequenceNumber=7, masterKeyId=13 on ha-hdfs:nameservice1 | |
17/09/28 06:56:02 INFO security.TokenCache: Got dt for hdfs://nameservice1; Kind: HDFS_DELEGATION_TOKEN, Service: ha-hdfs:nameservice1, Ident: (token for testkerberos: HDFS_DELEGATION_TOKEN owner=testkerberos@MYITVERSITY.COM, renewer=yarn, realUser=, issueDate=1506581762808, maxDate=1507186562808, sequenceNumber=7, masterKeyId=13) | |
17/09/28 06:56:03 INFO input.FileInputFormat: Total input paths to process : 10 | |
17/09/28 06:56:04 INFO mapreduce.JobSubmitter: number of splits:10 | |
17/09/28 06:56:04 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1506575284029_0007 | |
17/09/28 06:56:04 INFO mapreduce.JobSubmitter: Kind: HDFS_DELEGATION_TOKEN, Service: ha-hdfs:nameservice1, Ident: (token for testkerberos: HDFS_DELEGATION_TOKEN owner=testkerberos@MYITVERSITY.COM, renewer=yarn, realUser=, issueDate=1506581762808, maxDate=1507186562808, sequenceNumber=7, masterKeyId=13) | |
17/09/28 06:56:05 INFO impl.YarnClientImpl: Submitted application application_1506575284029_0007 | |
17/09/28 06:56:05 INFO mapreduce.Job: The url to track the job: http://cdh003:8088/proxy/application_1506575284029_0007/ | |
17/09/28 06:56:05 INFO mapreduce.Job: Running job: job_1506575284029_0007 | |
17/09/28 06:56:20 INFO mapreduce.Job: Job job_1506575284029_0007 running in uber mode : false | |
17/09/28 06:56:20 INFO mapreduce.Job: map 0% reduce 0% | |
17/09/28 06:56:34 INFO mapreduce.Job: map 10% reduce 0% | |
17/09/28 06:56:47 INFO mapreduce.Job: map 20% reduce 0% | |
17/09/28 06:56:53 INFO mapreduce.Job: map 30% reduce 0% | |
17/09/28 06:56:59 INFO mapreduce.Job: map 40% reduce 0% | |
17/09/28 06:57:10 INFO mapreduce.Job: map 50% reduce 0% | |
17/09/28 06:57:17 INFO mapreduce.Job: map 60% reduce 0% | |
17/09/28 06:57:24 INFO mapreduce.Job: map 70% reduce 0% | |
17/09/28 06:57:31 INFO mapreduce.Job: map 80% reduce 0% | |
17/09/28 06:57:37 INFO mapreduce.Job: map 90% reduce 0% | |
17/09/28 06:57:44 INFO mapreduce.Job: map 100% reduce 0% | |
17/09/28 06:57:52 INFO mapreduce.Job: map 100% reduce 100% | |
17/09/28 06:57:54 INFO mapreduce.Job: Job job_1506575284029_0007 completed successfully | |
17/09/28 06:57:54 INFO mapreduce.Job: Counters: 50 | |
File System Counters | |
FILE: Number of bytes read=109 | |
FILE: Number of bytes written=1664518 | |
FILE: Number of read operations=0 | |
FILE: Number of large read operations=0 | |
FILE: Number of write operations=0 | |
HDFS: Number of bytes read=2700 | |
HDFS: Number of bytes written=215 | |
HDFS: Number of read operations=43 | |
HDFS: Number of large read operations=0 | |
HDFS: Number of write operations=3 | |
Job Counters | |
Launched map tasks=10 | |
Launched reduce tasks=1 | |
Data-local map tasks=4 | |
Rack-local map tasks=6 | |
Total time spent by all maps in occupied slots (ms)=72197 | |
Total time spent by all reduces in occupied slots (ms)=6892 | |
Total time spent by all map tasks (ms)=72197 | |
Total time spent by all reduce tasks (ms)=6892 | |
Total vcore-milliseconds taken by all map tasks=72197 | |
Total vcore-milliseconds taken by all reduce tasks=6892 | |
Total megabyte-milliseconds taken by all map tasks=73929728 | |
Total megabyte-milliseconds taken by all reduce tasks=7057408 | |
Map-Reduce Framework | |
Map input records=10 | |
Map output records=20 | |
Map output bytes=180 | |
Map output materialized bytes=340 | |
Input split bytes=1520 | |
Combine input records=0 | |
Combine output records=0 | |
Reduce input groups=2 | |
Reduce shuffle bytes=340 | |
Reduce input records=20 | |
Reduce output records=0 | |
Spilled Records=40 | |
Shuffled Maps =10 | |
Failed Shuffles=0 | |
Merged Map outputs=10 | |
GC time elapsed (ms)=849 | |
CPU time spent (ms)=8360 | |
Physical memory (bytes) snapshot=4089552896 | |
Virtual memory (bytes) snapshot=16717639680 | |
Total committed heap usage (bytes)=3346178048 | |
Shuffle Errors | |
BAD_ID=0 | |
CONNECTION=0 | |
IO_ERROR=0 | |
WRONG_LENGTH=0 | |
WRONG_MAP=0 | |
WRONG_REDUCE=0 | |
File Input Format Counters | |
Bytes Read=1180 | |
File Output Format Counters | |
Bytes Written=97 | |
Job Finished in 112.566 seconds | |
Estimated value of Pi is 3.14080000000000000000 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment