Skip to content

Instantly share code, notes, and snippets.

@ivanlei
Created July 20, 2013 16:30
Show Gist options
  • Save ivanlei/6045631 to your computer and use it in GitHub Desktop.
Save ivanlei/6045631 to your computer and use it in GitHub Desktop.
Example command for running an mrjob with job pooling and EMR instances shared between IAM in the same AWS account.
export AWS_ACCESS_KEY_ID=XXX
export AWS_SECRET_ACCESS_KEY=XXX
python ./mr_word_freq_count.py ./wordlist \
--pool-emr-job-flows \
--runner=emr \
--visible-to-all-users \
--num-ec2-instances=1 \
--aws-region=us-west-2 --emr-endpoint=us-west-2.elasticmapreduce.amazonaws.com
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment