Skip to content

Instantly share code, notes, and snippets.

@dgadiraju

dgadiraju/cdh-admin-wordcount-compressed.sh

Created February 5, 2019 15:58
Show Gist options
  • Save dgadiraju/fddf23bb57c5279a9cedd1b59c4a492a to your computer and use it in GitHub Desktop.
Save dgadiraju/fddf23bb57c5279a9cedd1b59c4a492a to your computer and use it in GitHub Desktop.
Download ZIP
Raw
cdh-admin-wordcount-compressed.sh
yarn jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar \
wordcount \
-Dmapreduce.input.fileinputformat.split.minsize=268435456 \
-Dmapreduce.job.reduces=8 \
-Dmapreduce.output.fileoutputformat.compress=true \
-Dmapreduce.output.fileoutputformat.compress.codec=org.apache.hadoop.io.compress.GzipCodec \
/user/itversity/randomtextwriter/part-m-00000 \
/user/itversity/wordcount_compressed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment