Skip to content

Instantly share code, notes, and snippets.

@MohammedJabarullah

MohammedJabarullah/aws_emr_cli.md

Last active August 24, 2016 15:09
Show Gist options
  • Save MohammedJabarullah/8c79a7b8c3de5b3cdd25b97133c48f9d to your computer and use it in GitHub Desktop.
Save MohammedJabarullah/8c79a7b8c3de5b3cdd25b97133c48f9d to your computer and use it in GitHub Desktop.
Download ZIP
AWS EMR CLI
Raw
aws_emr_cli.md

AWS EMR CLI

List of clusters

aws emr list-clusters

Get LogUri to get log locations

aws emr describe-cluster --cluster-id j-2X0TJ4AYW2U99 | grep LogUri
# "LogUri": "s3n://aws-logs-[account_number]-us-east-1/elasticmapreduce/",

List of steps from the cluster

aws emr list-steps --cluster-id j-2X0TJ4AYW2U99

download log files (use with --dryrun to see what will be downloaded)

aws s3 cp s3://aws-logs-[account_number]-us-east-1/elasticmapreduce/j-2X0TJ4AYW2U99/ . --include "*.gz" --recursive

Deflate the downloaded files

gzip -rd *

Grep through logs

grep -i -E "(error)|(exception)"  -R *
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment