AWS CLI JMESPath cheatsheet.

README.md

AWS CLI JMESPath cheatsheet

Random query recipes of JMESPath for the AWS CLI tools that I might have written or stumbled upon.

• Examples
  • List available AWS regions
  • List Lambda functions within AWS account region
  • Test if specific Lambda function exists
  • List Route 53 record names and their type for a zone
  • List CloudWatch log groups
  • List CloudWatch log groups with event expiry
  • List SQS queue names
  • Status of CloudFormation stack
  • List logical resource IDs of CloudFormation stack
  • EC2 system and instance reachability status as string pair
  • EC2 terminate instance ID and return current state
  • EC2 instance availability zone and public IP address
  • EC2 instance get autoscale group name by tag
  • EC2 instance get autoscale group name via auto scaling API
  • EC2 marketplace AMI ID's for a given product ID
  • ECR list repositories
  • RDS list engine versions auto upgrade
  • VPC network interfaces associated to a security group ID
  • Verify ARN identity of current API credentials
  • List CodeBuild build IDs
• Reference

Examples

List available AWS regions

Lists enabled regions for the current AWS account. One per each line.

aws ec2 describe-regions \
  --output text
  --query "Regions[].[RegionName]"

# ap-south-1
# eu-west-2
# eu-west-1
# ap-northeast-2
# ap-northeast-1
# etc.

List Lambda functions within AWS account region

aws lambda list-functions \
  --output text \
  --query "Functions[].[FunctionName]"

Test if specific Lambda function exists

aws lambda list-functions \
  --output text \
  --query "Functions[?FunctionName=='MY_FUNCTION_NAME'].CodeSha256"

List Route 53 record names and their type for a zone

aws route53 list-resource-record-sets \
  --hosted-zone-id HOSTED_ZONE_ID \
  --output text \
  --query "ResourceRecordSets[].[join(': ',[Name,Type])]"

List CloudWatch log groups

aws logs describe-log-groups \
  --output text \
  --query "logGroups[].[logGroupName]"

List CloudWatch log groups with event expiry

aws logs describe-log-groups \
  --output text \
  --query "logGroups[].[join(': ',[logGroupName,to_string(retentionInDays || 'Never Expire')])]"

List SQS queue names

aws sqs list-queues \
  --output text \
  --query 'QueueUrls.join(`\n`,@)'

Status of CloudFormation stack

aws cloudformation describe-stacks \
  --stack-name STACK_NAME \
  --output text \
  --query "Stacks[0].StackStatus"

List logical resource IDs of CloudFormation stack

aws cloudformation describe-stack-resources \
  --stack-name STACK_NAME \
  --output text \
  --query "StackResources[].[LogicalResourceId]"

EC2 system and instance reachability status as string pair

aws ec2 describe-instance-status \
  --instance-ids INSTANCE_ID \
  --output text \
  --query "join(':',InstanceStatuses[0].[InstanceStatus,SystemStatus][].Details[0].Status)"

Returns a value in the form of passed:passed.

EC2 terminate instance ID and return current state

aws ec2 terminate-instances \
  --instance-ids INSTANCE_ID \
  --output text \
  --query "TerminatingInstances[0].join(':',[InstanceId,CurrentState.Name])"

EC2 instance availability zone and public IP address

aws ec2 describe-instances \
  --instance-ids INSTANCE_ID \
  --output text \
  --query "Reservations[0].Instances[0].join(':',[Placement.AvailabilityZone,PublicIpAddress || ''])"

EC2 instance get autoscale group name by tag

aws ec2 describe-tags \
  --filters "Name=resource-id,Values=INSTANCE_ID" \
  --output text \
  --query "Tags[?Key=='aws:autoscaling:groupName'].Value"

EC2 instance get autoscale group name via auto scaling API

aws autoscaling describe-auto-scaling-instances \
  --instance-ids INSTANCE_ID \
  --output text \
  --query "AutoScalingInstances[*].AutoScalingGroupName"

EC2 marketplace AMI ID's for a given product ID

aws ec2 describe-images \
  --filters "Name=name,Values=*-PRODUCT_ID-*" \
  --output text \
  --query "reverse(sort_by(Images,&CreationDate))[].[join(':',[ImageId,CreationDate,Description])]"

ECR list repositories

aws ecr describe-repositories \
  --output text \
  --query "repositories[].[repositoryName]"

RDS list engine versions auto upgrade

aws rds describe-db-engine-versions \
  --engine postgres \
  --engine-version ENGINE_VERSION \
  --output table \
  --query "DBEngineVersions[*].ValidUpgradeTarget[*].{AutoUpgrade:AutoUpgrade,EngineVersion:EngineVersion}"

VPC network interfaces associated to a security group ID

aws ec2 describe-network-interfaces \
  --filters "Name=group-id,Values=SECURITY_GROUP_ID" \
  --output text \
  --query "NetworkInterfaces[].[NetworkInterfaceId]"

Verify ARN identity of current API credentials

Tip

Handy trick to ensure a script is run against the expected IAM identity/AWS account.

Will return true if identity prefix matches that of expected, otherwise false.

aws sts get-caller-identity \
  --query "starts_with(Arn,'arn:aws:ARN_IDENTITY_PREFIX/')"

List CodeBuild build IDs

aws codebuild list-builds \
  --output text \
  --query 'ids.join(`\n`,@)' \

Reference

• https://jmespath.org/
• https://jmespath.org/tutorial.html
• https://docs.aws.amazon.com/cli/latest/userguide/cli-usage-filter.html#cli-usage-filter-client-side
• https://glenveegee.github.io/jmespath-edit/

Here's a puzzler: If I run this command:
aws ec2 describe-snapshots --owner-ids "self" --query 'Snapshots[*].SnapshotId' --output text
I get back a list of snap ids, each separated by a tab character. Theoretically, I should be able to use the JMESPath "join" built-in function to join together the snap ids with a newline instead of a tab. But I'll be darned if I can't get that to work in Bash. It seems like I should be able to do this:
aws ec2 describe-snapshots --owner-ids "self" --query 'join("\n",Snapshots[*].SnapshotId)' --output text
...but I get an error saying "n" is not a valid command. I've tried all the different ways of quoting that I know of, but I get error every time. Is it just impossible to join with an escaped character in aws cli command run in Bash?

@lousyd

aws ec2 describe-snapshots --owner-ids "self" --query 'join("\n",Snapshots[*].SnapshotId)' --output text
I get error every time. Is it just impossible to join with an escaped character in aws cli command run in Bash?

By trial and error, backtick the newline.

'join(`\n`,Snapshots[*].SnapshotId)'

If using double quotes around the query, say to pass in a shell variable, you have to escape the backticks.

"join(\`\n\`,Snapshots[*].SnapshotId)"

$ aws --version
aws-cli/1.16.186 Python/3.5.2 Linux/4.4.0-1087-aws botocore/1.12.176
$ aws ec2 describe-snapshots --owner-ids "self" --query 'join(`\n`,Snapshots[*].SnapshotId)' --output text
snap-0f52b45a2905b2bd9
snap-055b234912e32dc6f
snap-07a523f79ecf5f55f
...

Anyone got a solution for this:
I'm trying to filter by a single string from the 'Description' field in security groups.

If the description contains "This is a word", then this fails:

aws ec2 describe-security-groups --query "SecurityGroups[?IpPermissions[?contains(IpRanges[].Description, 'word')]]"

But this succeeds:

aws ec2 describe-security-groups --query "SecurityGroups[?IpPermissions[?contains(IpRanges[].Description, 'This is a word')]]"

Ideally I would like to have a fuzzy search.

@kaipee - it's because you're searching in an array, not a string.

https://jmespath.org/specification.html#contains

If $subject is an array, this function returns true if one of the elements in the array is equal to the provided $search value.

If the provided $subject is a string, this function returns true if the string contains the provided $search argument.

Thanks @magnetikonline , I assumed as much.

Would you happen to know how to return the values of IpPermissions>IpRanges>Description[] so that I can query the strings?

Would you happen to know how to return the values of IpPermissions>IpRanges>Description[] so that I can query the strings?

no, I'm not sure how you could do that to be honest.... @kaipee - unless you just grep the result.

"# In function starts_with(), invalid type for value: None, expected one of: ['string'], received: "null" “
In some cases, there is no "Tags" field in the returned data, this requires a judgment call.
see https://jmespath.org/specification.html -->Or Expressions
now we can use :
aws ec2 describe-instances
--filters "Name=instance.group-id,Values=sg-xxxxxx"
--query "Reservations[].Instances[].{Instance:InstanceId, Name:join('',Tags[?Key=='Name'].Value||[''])}"