Use AWS CLI and jq to do some things.md

Use AWS CLI and jq to do some things

I'm just getting up to speed on AWS, and so I'm learning my way around the CLI. It's nice and easy to do lots of things from the CLI, like querying, creating and destroying instances and images. There are lots of fiddly bits involved in parsing output so it can be used by a script or something, though, so here are some notes.

Tools

AWS CLI

Obviously you need the AWS CLI. It's an easy install using pip. Go here to find documentation and other install methods.

pip install awscli

jq

jq is a powerful JSON parsing tool. It's really a must-have if you're going to do anything with JSON. Find it here.

It's in the Fedora yum repo, so all you have to do is

yum -y install jq

Examples

OK, so let's query for all the AMIs I own in my personal account:

aws --profile personal ec2 describe-images --owners <your ID here> --output json | jq '.Images[] | {ImageId}' | jq --raw-output '.ImageId'

Now let's query for all the snapshots I own in my personal account:

aws --profile personal ec2 describe-snapshots --owner-ids <your ID here> --output json | jq '.Snapshots[] | {SnapshotId}' | jq --raw-output '.SnapshotId'

It turns out this was actually a thing I was doing today. I was playing with Packer and generating AMIs, which creates an AMI and a snapshot for each image deployed. I wanted to clean up everything periodically, so I can feed the outputs from the above commands into another command which cleans each thing up.

# Clean up AMIs

list=($(aws --profile personal ec2 describe-images --owners <your ID here> --output json | jq '.Images[] | {ImageId}' | jq --raw-output '.ImageId'))

for image in ${list[@]}; do
  aws --profile personal ec2 deregister-image --image-id ${image}
done

# Clean up snapshots

list=($(aws --profile personal ec2 describe-snapshots --owner-ids <your ID here> --output json | jq '.Snapshots[] | {SnapshotId}' | jq --raw-output '.SnapshotId'))

for snapshot in ${list[@]}; do
  aws --profile personal ec2 delete-snapshot --snapshot-id ${snapshot}
done

You should check out jmespath.

It's a little cryptic at times, but it'll remove a dependency. It's built into the AWS cli.

An example from the AWS cli docs (the --query field takes jmespath):

aws ec2 describe-snapshots --filters Name=tag-key,Values="Group" Name=tag-value,Values="Prod" --query 'Snapshots[*].{ID:SnapshotId,Time:StartTime}'

One of the issues with --filters is that not all services support filtering. The autoscaling service for example.