Export Metrics from CloudWatch to CSV

readme.md

Export CloudWatch Metrics

This document outlines how to export CloudWatch Metrics to CSV.

Pre Reqs

1. AWS CLI (Command Line Interface) from here
2. jq is a lightweight and flexible command-line JSON processor from here

How to export

Use the following CLI:

aws cloudwatch get-metric-statistics 
--namespace AWS/EC2 --metric-name CPUUtilization 
--dimensions Name=InstanceId,Value=i-xxxxxxxxxxxxxxxxx 
--statistics Average 
--start-time 2020-03-01T00:00:00 
--end-time 2020-03-31T23:59:00 
--period 3600
--region us-east-1

You will specify the EC2 Instance Id in the --dimensions Name=InstanceId,Value=i-xxxxxxxxxxxxxxxxx parameter.

Valid options for --metric-name depend on the --name-space parameter. For AWS/EC2, the full list can be seen by running the following CLI command:

aws cloudwatch list-metrics --namespace "AWS/EC2"

Valid options for --statistics are:

SampleCount
Average
Sum
Minimum
Maximum

--start-time and --end-time specify the range.

--period The granularity, in seconds, of the returned data points.

--region The region where the CloudWatch metric is being meatured (us-east-1, us-west-2 etc.)

Data output will look something like the following:

{
    "Label": "CPUUtilization",
    "Datapoints": []
}

To convert it to CSV we will use jq. For this you have two options:

Option 1

Pipe all the aws cli output to jq:

aws cloudwatch get-metric-statistics 
--namespace AWS/EC2 --metric-name CPUUtilization 
--dimensions Name=InstanceId,Value=i-xxxxxxxxxxxxxxxxx 
--statistics Average 
--start-time 2020-03-01T00:00:00 
--end-time 2020-03-31T23:59:00 
--period 3600
--region us-east-1
| jq -r '.Datapoints[] | [.Timestamp, .Minimum, .Unit] | @csv'

Option 2

Export the data to JSON:

aws cloudwatch get-metric-statistics 
--namespace AWS/EC2 --metric-name CPUUtilization 
--dimensions Name=InstanceId,Value=i-xxxxxxxxxxxxxxxxx 
--statistics Average 
--start-time 2020-03-01T00:00:00 
--end-time 2020-03-31T23:59:00 
--period 3600
--region us-east-1 >> data.json

Use jq to convert json to csv:

jq -r '.Datapoints[] | [.Timestamp, .Minimum, .Unit] | @csv' data.json

Output

Here is how the output will look like:

"2020-03-24T11:00:00Z",0.327868852454245,"Percent"
"2020-03-11T21:00:00Z",0.327868852454245,"Percent"
"2020-03-15T04:00:00Z",0.322580645156596,"Percent"
"2020-03-27T18:00:00Z",0.327868852478101,"Percent"

Further Reading

AWS CLI Command Reference