Skip to content

Instantly share code, notes, and snippets.

@ouyi

ouyi/emr_test_apps.sh

Last active January 24, 2018 15:02
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save ouyi/bae4f02a882eb11411696a7789fb7482 to your computer and use it in GitHub Desktop.
Save ouyi/bae4f02a882eb11411696a7789fb7482 to your computer and use it in GitHub Desktop.
Download ZIP
Built-in test apps on EMR 5.x
Raw
emr_test_apps.sh
cat >> input.txt
hello
world
world
<ctrl+D>
wget https://raw.githubusercontent.com/ouyi/tez-demo/master/src/test/resources/input.txt
hadoop fs -copyFromLocal input.txt /tmp/
hadoop fs -copyToLocal /apps/tez/tez.tar.gz .
tar xzf tez.tar.gz
hadoop fs -rm -r -f /tmp/output && hadoop jar tez-examples-0.8.4.jar orderedwordcount /tmp/input.txt /tmp/output
wget https://github.com/ouyi/tez-demo/raw/master/src/main/resources/wordcount.pig
hadoop fs -rm -r -f /tmp/output && pig -x tez -p input=/tmp/input.txt -p output=/tmp/output wordcount.pig
hadoop fs -rm -r -f /tmp/output && pig -x mapreduce -p input=/tmp/input.txt -p output=/tmp/output wordcount.pig
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment