Skip to content

Instantly share code, notes, and snippets.

@vatshat
Created January 29, 2019 18:25
Show Gist options
  • Star 7 You must be signed in to star a gist
  • Fork 1 You must be signed in to fork a gist
  • Save vatshat/7d401a9194d4e1b159bd8894035b6b5f to your computer and use it in GitHub Desktop.
Save vatshat/7d401a9194d4e1b159bd8894035b6b5f to your computer and use it in GitHub Desktop.
An example of how to use regex in the parse statement of a CloudWatch Insights query
#!/usr/bin/env bash
query_string=$(cat << EndOfMessage
fields @timestamp, @logStream, headers.X-Amzn-Trace-Id, @transId, @message
| parse @message /(transactionId:[ ]?)(?<@transId>[a-zA-Z0-9]+)/
| filter @transId = a4c475516be5445a87fbb81bb7a4b365
EndOfMessage
) \
&& \
query_id=`aws logs start-query --log-group-name /aws/lambda/console_log \
--start-time $(TZ='UTC' date -d "-1 day" +%s%3N) \
--end-time $(TZ='UTC' date +%s%3N) \
--query-string "$query_string" --output text` \
&& \
aws logs get-query-results --query-id $query_id | python -m json.tool
@vatshat
Copy link
Author

vatshat commented Jan 29, 2019

Take special note that CloudWatch will automatically create a new field/column in the query result using the Regex named group from the parse statement as the field name. So in the above query, a new field will be created named transId because of this name group ?<@transId>

@vatshat
Copy link
Author

vatshat commented Jan 29, 2019

Additionally, you don't have to specify the new field which you're creating from the parse statement in the field statement. CloudWatch will automatically include that field in the query result. I included in this example just for demonstration purposes

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment