Skip to content

Instantly share code, notes, and snippets.

@magnetikonline
Last active July 3, 2024 13:56
Show Gist options
  • Save magnetikonline/6a382a4c4412bbb68e33e137b9a74168 to your computer and use it in GitHub Desktop.
Save magnetikonline/6a382a4c4412bbb68e33e137b9a74168 to your computer and use it in GitHub Desktop.
AWS CLI JMESPath cheatsheet.

AWS CLI JMESPath cheatsheet

Random query recipes of JMESPath for the AWS CLI tools that I might have written or stumbled upon.

Examples

List available AWS regions

Lists enabled regions for the current AWS account. One per each line.

aws ec2 describe-regions \
  --output text
  --query "Regions[].[RegionName]"

# ap-south-1
# eu-west-2
# eu-west-1
# ap-northeast-2
# ap-northeast-1
# etc.

List Lambda functions within AWS account region

aws lambda list-functions \
  --output text \
  --query "Functions[].[FunctionName]"

Test if specific Lambda function exists

aws lambda list-functions \
  --output text \
  --query "Functions[?FunctionName=='MY_FUNCTION_NAME'].CodeSha256"

List Route 53 record names and their type for a zone

aws route53 list-resource-record-sets \
  --hosted-zone-id HOSTED_ZONE_ID \
  --output text \
  --query "ResourceRecordSets[].[join(': ',[Name,Type])]"

List CloudWatch log groups

aws logs describe-log-groups \
  --output text \
  --query "logGroups[].[logGroupName]"

List CloudWatch log groups with event expiry

aws logs describe-log-groups \
  --output text \
  --query "logGroups[].[join(': ',[logGroupName,to_string(retentionInDays || 'Never Expire')])]"

List SQS queue names

aws sqs list-queues \
  --output text \
  --query 'QueueUrls.join(`\n`,@)'

Status of CloudFormation stack

aws cloudformation describe-stacks \
  --stack-name STACK_NAME \
  --output text \
  --query "Stacks[0].StackStatus"

List logical resource IDs of CloudFormation stack

aws cloudformation describe-stack-resources \
  --stack-name STACK_NAME \
  --output text \
  --query "StackResources[].[LogicalResourceId]"

EC2 system and instance reachability status as string pair

aws ec2 describe-instance-status \
  --instance-ids INSTANCE_ID \
  --output text \
  --query "join(':',InstanceStatuses[0].[InstanceStatus,SystemStatus][].Details[0].Status)"

Returns a value in the form of passed:passed.

EC2 terminate instance ID and return current state

aws ec2 terminate-instances \
  --instance-ids INSTANCE_ID \
  --output text \
  --query "TerminatingInstances[0].join(':',[InstanceId,CurrentState.Name])"

EC2 instance availability zone and public IP address

aws ec2 describe-instances \
  --instance-ids INSTANCE_ID \
  --output text \
  --query "Reservations[0].Instances[0].join(':',[Placement.AvailabilityZone,PublicIpAddress || ''])"

EC2 instance get autoscale group name by tag

aws ec2 describe-tags \
  --filters "Name=resource-id,Values=INSTANCE_ID" \
  --output text \
  --query "Tags[?Key=='aws:autoscaling:groupName'].Value"

EC2 instance get autoscale group name via auto scaling API

aws autoscaling describe-auto-scaling-instances \
  --instance-ids INSTANCE_ID \
  --output text \
  --query "AutoScalingInstances[*].AutoScalingGroupName"

EC2 marketplace AMI ID's for a given product ID

aws ec2 describe-images \
  --filters "Name=name,Values=*-PRODUCT_ID-*" \
  --output text \
  --query "reverse(sort_by(Images,&CreationDate))[].[join(':',[ImageId,CreationDate,Description])]"

ECR list repositories

aws ecr describe-repositories \
  --output text \
  --query "repositories[].[repositoryName]"

RDS list engine versions auto upgrade

aws rds describe-db-engine-versions \
  --engine postgres \
  --engine-version ENGINE_VERSION \
  --output table \
  --query "DBEngineVersions[*].ValidUpgradeTarget[*].{AutoUpgrade:AutoUpgrade,EngineVersion:EngineVersion}"

VPC network interfaces associated to a security group ID

aws ec2 describe-network-interfaces \
  --filters "Name=group-id,Values=SECURITY_GROUP_ID" \
  --output text \
  --query "NetworkInterfaces[].[NetworkInterfaceId]"

Verify ARN identity of current API credentials

Tip

Handy trick to ensure a script is run against the expected IAM identity/AWS account.

Will return true if identity prefix matches that of expected, otherwise false.

aws sts get-caller-identity \
  --query "starts_with(Arn,'arn:aws:ARN_IDENTITY_PREFIX/')"

List CodeBuild build IDs

aws codebuild list-builds \
  --output text \
  --query 'ids.join(`\n`,@)' \

Reference

@MrAtheist
Copy link

Thanks for the cheatsheet! I'm trying to incorporate a bash variable into starts_with and cant seem to figure it out. Wondering if you have some insight into the following:

# continue with one of your examples as it's the closest to what i'm doing

HELLO_WORLD="hello-world" #note the dash

aws ec2 describe-tags \
	--filters "Name=resource-id,Values=INSTANCE_ID" \
	--output text \
	--query "Tags[?starts_with(Key, '${HELLO_WORLD}') == `true`]"

The above query returns...
Bad value for --query Tags[?starts_with(Key, '${HELLO_WORLD}') ==]: invalid token: Parse error at column 66, token "]" (RBRACKET), for expression:

It's the same as if I hard coded the value in the query

aws ec2 describe-tags \
	--filters "Name=resource-id,Values=INSTANCE_ID" \
	--output text \
	--query "Tags[?starts_with(Key, 'hello-world') == `true`]"

# Bad value for --query Tags[?starts_with(Key, 'hello-world') ==]: invalid token: Parse error at column 66, token "]" (RBRACKET), for expression:

Now flipping the quotes and stuff from the last line...

aws ec2 describe-tags \
	--filters "Name=resource-id,Values=INSTANCE_ID" \
	--output text \
	--query 'Tags[?starts_with(Key, "'${HELLO_WORLD}'") == `true`]'

# In function starts_with(), invalid type for value: None, expected one of: ['string'], received: "null"

It's the same as if I hard coded the value in the query

aws ec2 describe-tags \
	--filters "Name=resource-id,Values=INSTANCE_ID" \
	--output text \
	--query 'Tags[?starts_with(Key, "hello-world") == `true`]'

# In function starts_with(), invalid type for value: None, expected one of: ['string'], received: "null"

And of course it doesnt work without the quotes around hello-world

aws ec2 describe-tags \
	--filters "Name=resource-id,Values=INSTANCE_ID" \
	--output text \
	--query 'Tags[?starts_with(Key, hello-world) == `true`]'

# Bad jmespath expression: Unknown token '-':

Appreciate if there is any other hack that can address this use case.

@magnetikonline
Copy link
Author

@MrAtheist I think your backticks around true are the issue? I would think you would need 'true'?

Or you maybe don't need the == true`` at all?

@MrAtheist
Copy link

Or you maybe don't need the "== true" at all?

That does the trick. Thanks for the tips! 🍻

@magnetikonline
Copy link
Author

That does the trick. Thanks for the tips!

Happy to help!

@lousyd
Copy link

lousyd commented May 17, 2019

Here's a puzzler: If I run this command:
aws ec2 describe-snapshots --owner-ids "self" --query 'Snapshots[*].SnapshotId' --output text
I get back a list of snap ids, each separated by a tab character. Theoretically, I should be able to use the JMESPath "join" built-in function to join together the snap ids with a newline instead of a tab. But I'll be darned if I can't get that to work in Bash. It seems like I should be able to do this:
aws ec2 describe-snapshots --owner-ids "self" --query 'join("\n",Snapshots[*].SnapshotId)' --output text
...but I get an error saying "n" is not a valid command. I've tried all the different ways of quoting that I know of, but I get error every time. Is it just impossible to join with an escaped character in aws cli command run in Bash?

@virgilwashere
Copy link

@lousyd

aws ec2 describe-snapshots --owner-ids "self" --query 'join("\n",Snapshots[*].SnapshotId)' --output text
I get error every time. Is it just impossible to join with an escaped character in aws cli command run in Bash?

By trial and error, backtick the newline.

'join(`\n`,Snapshots[*].SnapshotId)'

If using double quotes around the query, say to pass in a shell variable, you have to escape the backticks.

"join(\`\n\`,Snapshots[*].SnapshotId)"
$ aws --version
aws-cli/1.16.186 Python/3.5.2 Linux/4.4.0-1087-aws botocore/1.12.176
$ aws ec2 describe-snapshots --owner-ids "self" --query 'join(`\n`,Snapshots[*].SnapshotId)' --output text
snap-0f52b45a2905b2bd9
snap-055b234912e32dc6f
snap-07a523f79ecf5f55f
...

@kaipee
Copy link

kaipee commented Jan 8, 2021

Anyone got a solution for this:
I'm trying to filter by a single string from the 'Description' field in security groups.

If the description contains "This is a word", then this fails:

aws ec2 describe-security-groups --query "SecurityGroups[?IpPermissions[?contains(IpRanges[].Description, 'word')]]"

But this succeeds:

aws ec2 describe-security-groups --query "SecurityGroups[?IpPermissions[?contains(IpRanges[].Description, 'This is a word')]]"

Ideally I would like to have a fuzzy search.

@magnetikonline
Copy link
Author

@kaipee - it's because you're searching in an array, not a string.

https://jmespath.org/specification.html#contains

If $subject is an array, this function returns true if one of the elements in the array is equal to the provided $search value.

If the provided $subject is a string, this function returns true if the string contains the provided $search argument.

@kaipee
Copy link

kaipee commented Jan 11, 2021

Thanks @magnetikonline , I assumed as much.

Would you happen to know how to return the values of IpPermissions>IpRanges>Description[] so that I can query the strings?

@magnetikonline
Copy link
Author

Would you happen to know how to return the values of IpPermissions>IpRanges>Description[] so that I can query the strings?

no, I'm not sure how you could do that to be honest.... @kaipee - unless you just grep the result.

@moonmistake
Copy link

moonmistake commented Apr 29, 2021

"# In function starts_with(), invalid type for value: None, expected one of: ['string'], received: "null" “
In some cases, there is no "Tags" field in the returned data, this requires a judgment call.
see https://jmespath.org/specification.html -->Or Expressions
now we can use :
aws ec2 describe-instances
--filters "Name=instance.group-id,Values=sg-xxxxxx"
--query "Reservations[].Instances[].{Instance:InstanceId, Name:join('',Tags[?Key=='Name'].Value||[''])}"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment