AWS CLI JMESPath cheatsheet.

README.md

AWS CLI JMESPath cheatsheet

Random query recipes of JMESPath for the AWS CLI tools that I might have written or stumbled upon.

• Examples
  • Return listing of all available AWS regions
  • List all Lambda functions within AWS account region
  • Test if specific Lambda function exists
  • List all Route 53 record names and their type for a zone
  • List all CloudWatch log groups
  • List all CloudWatch log groups with event expiry
  • Status of CloudFormation stack
  • List all logical resource IDs of CloudFormation stack
  • EC2 system and instance reachability status as string pair
  • EC2 terminate instance ID and current state
  • EC2 instance availability zone and public IP address
  • EC2 instance get autoscale group name by tag
  • EC2 instance get autoscale group name by auto scaling API
  • EC2 marketplace AMI ID's for a given product ID
  • ECR list all repositories
  • VPC network interfaces associated to a security group ID
  • Verify ARN identity of current API credentials
• Reference

Examples

Return listing of all available AWS regions

Lists all enabled regions for the current account. One per each line.

aws ec2 describe-regions \
  --output text
  --query "Regions[].[RegionName]"

# ap-south-1
# eu-west-2
# eu-west-1
# ap-northeast-2
# ap-northeast-1
# etc.

List all Lambda functions within AWS account region

aws lambda list-functions \
  --output text \
  --query "Functions[].[FunctionName]"

Test if specific Lambda function exists

aws lambda list-functions \
  --output text \
  --query "Functions[?FunctionName=='MY_FUNCTION_NAME'].CodeSha256"

List all Route 53 record names and their type for a zone

aws route53 list-resource-record-sets \
  --hosted-zone-id HOSTED_ZONE_ID \
  --output text \
  --query "ResourceRecordSets[].[join(': ',[Name,Type])]"

List all CloudWatch log groups

aws logs describe-log-groups \
  --output text \
  --query "logGroups[].[logGroupName]"

List all CloudWatch log groups with event expiry

aws logs describe-log-groups \
  --output text \
  --query "logGroups[].[join(': ',[logGroupName,to_string(retentionInDays || 'Never Expire')])]"

Status of CloudFormation stack

aws cloudformation describe-stacks \
  --stack-name STACK_NAME \
  --output text \
  --query "Stacks[0].StackStatus"

List all logical resource IDs of CloudFormation stack

aws cloudformation describe-stack-resources \
  --stack-name STACK_NAME \
  --output text \
  --query "StackResources[].[LogicalResourceId]"

EC2 system and instance reachability status as string pair

aws ec2 describe-instance-status \
  --instance-ids INSTANCE_ID \
  --output text \
  --query "join(':',InstanceStatuses[0].[InstanceStatus,SystemStatus][].Details[0].Status)"

Returns a value in the form of passed:passed.

EC2 terminate instance ID and current state

aws ec2 terminate-instances \
  --instance-ids INSTANCE_ID \
  --output text \
  --query "TerminatingInstances[0].join(':',[InstanceId,CurrentState.Name])"

EC2 instance availability zone and public IP address

aws ec2 describe-instances \
  --instance-ids INSTANCE_ID \
  --output text \
  --query "Reservations[0].Instances[0].join(':',[Placement.AvailabilityZone,PublicIpAddress || ''])"

EC2 instance get autoscale group name by tag

aws ec2 describe-tags \
  --filters "Name=resource-id,Values=INSTANCE_ID" \
  --output text \
  --query "Tags[?Key=='aws:autoscaling:groupName'].Value"

EC2 instance get autoscale group name by auto scaling API

aws autoscaling describe-auto-scaling-instances \
  --instance-ids INSTANCE_ID \
  --output text \
  --query "AutoScalingInstances[*].AutoScalingGroupName"

EC2 marketplace AMI ID's for a given product ID

aws ec2 describe-images \
  --filters "Name=name,Values=*-PRODUCT_ID-*" \
  --output text \
  --query "reverse(sort_by(Images,&CreationDate))[].[join(':',[ImageId,CreationDate,Description])]"

ECR list all repositories

aws ecr describe-repositories \
  --output text \
  --query "repositories[].[repositoryName]"

VPC network interfaces associated to a security group ID

aws ec2 describe-network-interfaces \
  --filters "Name=group-id,Values=SECURITY_GROUP_ID" \
  --output text \
  --query "NetworkInterfaces[].[NetworkInterfaceId]"

Verify ARN identity of current API credentials

Will return true if identity prefix matches that of expected, otherwise false. Useful to ensure a script is run against the correct identity/AWS account.

aws sts get-caller-identity \
  --query "starts_with(Arn,'arn:aws:ARN_IDENTITY_PREFIX/')"

Reference

• http://jmespath.org/
• http://jmespath.org/tutorial.html
• https://docs.aws.amazon.com/cli/latest/userguide/controlling-output.html#controlling-output-filter

Thanks for the cheatsheet! I'm trying to incorporate a bash variable into starts_with and cant seem to figure it out. Wondering if you have some insight into the following:

# continue with one of your examples as it's the closest to what i'm doing

HELLO_WORLD="hello-world" #note the dash

aws ec2 describe-tags \
	--filters "Name=resource-id,Values=INSTANCE_ID" \
	--output text \
	--query "Tags[?starts_with(Key, '${HELLO_WORLD}') == `true`]"

The above query returns...
Bad value for --query Tags[?starts_with(Key, '${HELLO_WORLD}') ==]: invalid token: Parse error at column 66, token "]" (RBRACKET), for expression:

It's the same as if I hard coded the value in the query

aws ec2 describe-tags \
	--filters "Name=resource-id,Values=INSTANCE_ID" \
	--output text \
	--query "Tags[?starts_with(Key, 'hello-world') == `true`]"

# Bad value for --query Tags[?starts_with(Key, 'hello-world') ==]: invalid token: Parse error at column 66, token "]" (RBRACKET), for expression:

Now flipping the quotes and stuff from the last line...

aws ec2 describe-tags \
	--filters "Name=resource-id,Values=INSTANCE_ID" \
	--output text \
	--query 'Tags[?starts_with(Key, "'${HELLO_WORLD}'") == `true`]'

# In function starts_with(), invalid type for value: None, expected one of: ['string'], received: "null"

It's the same as if I hard coded the value in the query

aws ec2 describe-tags \
	--filters "Name=resource-id,Values=INSTANCE_ID" \
	--output text \
	--query 'Tags[?starts_with(Key, "hello-world") == `true`]'

# In function starts_with(), invalid type for value: None, expected one of: ['string'], received: "null"

And of course it doesnt work without the quotes around hello-world

aws ec2 describe-tags \
	--filters "Name=resource-id,Values=INSTANCE_ID" \
	--output text \
	--query 'Tags[?starts_with(Key, hello-world) == `true`]'

# Bad jmespath expression: Unknown token '-':

Appreciate if there is any other hack that can address this use case.

@MrAtheist I think your backticks around true are the issue? I would think you would need 'true'?

Or you maybe don't need the == true`` at all?

Or you maybe don't need the "== true" at all?

That does the trick. Thanks for the tips! 🍻

That does the trick. Thanks for the tips!

Happy to help!

Here's a puzzler: If I run this command:
aws ec2 describe-snapshots --owner-ids "self" --query 'Snapshots[*].SnapshotId' --output text
I get back a list of snap ids, each separated by a tab character. Theoretically, I should be able to use the JMESPath "join" built-in function to join together the snap ids with a newline instead of a tab. But I'll be darned if I can't get that to work in Bash. It seems like I should be able to do this:
aws ec2 describe-snapshots --owner-ids "self" --query 'join("\n",Snapshots[*].SnapshotId)' --output text
...but I get an error saying "n" is not a valid command. I've tried all the different ways of quoting that I know of, but I get error every time. Is it just impossible to join with an escaped character in aws cli command run in Bash?

@lousyd

aws ec2 describe-snapshots --owner-ids "self" --query 'join("\n",Snapshots[*].SnapshotId)' --output text
I get error every time. Is it just impossible to join with an escaped character in aws cli command run in Bash?

By trial and error, backtick the newline.

'join(`\n`,Snapshots[*].SnapshotId)'

If using double quotes around the query, say to pass in a shell variable, you have to escape the backticks.

"join(\`\n\`,Snapshots[*].SnapshotId)"

$ aws --version
aws-cli/1.16.186 Python/3.5.2 Linux/4.4.0-1087-aws botocore/1.12.176
$ aws ec2 describe-snapshots --owner-ids "self" --query 'join(`\n`,Snapshots[*].SnapshotId)' --output text
snap-0f52b45a2905b2bd9
snap-055b234912e32dc6f
snap-07a523f79ecf5f55f
...

Anyone got a solution for this:
I'm trying to filter by a single string from the 'Description' field in security groups.

If the description contains "This is a word", then this fails:

aws ec2 describe-security-groups --query "SecurityGroups[?IpPermissions[?contains(IpRanges[].Description, 'word')]]"

But this succeeds:

aws ec2 describe-security-groups --query "SecurityGroups[?IpPermissions[?contains(IpRanges[].Description, 'This is a word')]]"

Ideally I would like to have a fuzzy search.

@kaipee - it's because you're searching in an array, not a string.

https://jmespath.org/specification.html#contains

If $subject is an array, this function returns true if one of the elements in the array is equal to the provided $search value.

If the provided $subject is a string, this function returns true if the string contains the provided $search argument.

Thanks @magnetikonline , I assumed as much.

Would you happen to know how to return the values of IpPermissions>IpRanges>Description[] so that I can query the strings?

Would you happen to know how to return the values of IpPermissions>IpRanges>Description[] so that I can query the strings?

no, I'm not sure how you could do that to be honest.... @kaipee - unless you just grep the result.

"# In function starts_with(), invalid type for value: None, expected one of: ['string'], received: "null" “
In some cases, there is no "Tags" field in the returned data, this requires a judgment call.
see https://jmespath.org/specification.html -->Or Expressions
now we can use :
aws ec2 describe-instances
--filters "Name=instance.group-id,Values=sg-xxxxxx"
--query "Reservations[].Instances[].{Instance:InstanceId, Name:join('',Tags[?Key=='Name'].Value||[''])}"