Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
EC2 CloudWatch stats for Dashing
<!-- Get the Rickshawgraph widget from https://gist.github.com/jwalton/6614023 -->
<li data-row="1" data-col="1" data-sizex="1" data-sizey="1">
<div data-id="ec2-cpu" data-view="Rickshawgraph" data-title="CPU Usage"
data-moreinfo=""
style="background-color:#333A52;"
data-renderer="line"
data-min="0"
data-max="100"
data-summary-method="none"
data-legend="true"
data-colors="rgba(192,132,255,1):rgba(96,170,255,1)"
></div>
</li>
<li data-row="1" data-col="1" data-sizex="1" data-sizey="1">
<div data-id="ec2-cpugraph" data-view="Graph" data-title="CPU Usage"
data-moreinfo=""
style="background-color:#333A52;"
></div>
</li>

Get EC2 CloudWatch stats and graph them in Dashing.

#!/usr/bin/env ruby
#jobs/ec2.rb
require './lib/dashing-ec2'
dashing_ec2 = DashingEC2.new({
:access_key_id => "YOUR KEY ID HERE",
:secret_access_key => "YOUR SECRET HERE",
})
# See documentation here for cloud watch API here: https://github.com/aws/aws-sdk-ruby/blob/af638994bb7d01a8fd0f8a6d6357567968638100/lib/aws/cloud_watch/client.rb
# See documentation on various metrics and dimensions here: http://docs.aws.amazon.com/AWSEC2/2011-07-15/UserGuide/index.html?using-cloudwatch.html
# Note that Amazon charges [$0.01 per 1000 reqeuests](http://aws.amazon.com/pricing/cloudwatch/),
# so:
#
# | frequency | $/month/stat |
# |:---------:|:------------:|
# | 1m | $0.432 |
# | 10m | $0.043 |
# | 1h | $0.007 |
#
# In the free tier, stats are only available for 5m intervals, so querying more often than
# once every 5 minutes is kind of pointless. You've been warned. :)
#
SCHEDULER.every '10m', :first_in => 0 do |job|
cpu_usage = [
{name: 'server1', instance_id: "i-xxxxxxxx", region: 'us-east-1'},
{name: 'server2', instance_id: "i-yyyyyyyy", region: 'us-east-1'},
{name: 'server3', instance_id: "i-zzzzzzzz", region: 'us-east-1'}
]
cpu_series = []
cpu_usage.each do |item|
cpu_data = dashing_ec2.getInstanceStats(item[:instance_id], item[:region], "CPUUtilization", :average)
cpu_data[:name] = item[:name]
cpu_series.push cpu_data
end
# If you're using the Rickshaw Graph widget: https://gist.github.com/jwalton/6614023
send_event "ec2-cpu", { series: cpu_series }
# If you're just using the regular Dashing graph widget:
send_event "ec2-cpu-server1", { points: cpu_series[0][:data] }
send_event "ec2-cpu-server2", { points: cpu_series[1][:data] }
send_event "ec2-cpu-server3", { points: cpu_series[2][:data] }
end # SCHEDULER
#lib/dashing_ec2.rb
require 'aws-sdk'
require 'time'
class DashingEC2
def initialize(options)
@access_key_id = options[:access_key_id]
@secret_access_key = options[:secret_access_key]
@clientCache = {}
end
# Get statistics for an instance
#
# * `instance_id` is the instance to get data about.
# * `region` is the name of the region the instance is from (e.g. 'us-east-1'.) See
# [monitoring URIs](http://docs.aws.amazon.com/general/latest/gr/rande.html#cw_region).
# * `metric_name` is the metric to get. See
# [the list of build in metrics](http://docs.aws.amazon.com/AWSEC2/2011-07-15/UserGuide/index.html?using-cloudwatch.html).
# * `type` is `:average` or `:maximum`.
# * `options` are [:start_time, :end_time, :period, :dimensions] as per
# `get_metric_statistics()`, although all are optional. Also:
# * `:duration` - If supplied, and no start_time or end_time are supplied, then start_time
# and end_time will be computed based on this value in seconds. Defaults to 6 hours.
def getInstanceStats(instance_id, region, metric_name, type=:average, options={})
if type == :average
statName = "Average"
elsif type == :maximum
statName = "Maxmimum"
end
statKey = type
# Get an API client instance
cw = @clientCache[region]
if not cw
cw = @clientCache[region] = AWS::CloudWatch::Client.new({
server: "https://monitoring.#{region}.amazonaws.com",
access_key_id: @access_key_id,
secret_access_key: @secret_access_key
})
end
# Build a default set of options to pass to get_metric_statistics
duration = (options[:duration] or (60*60*6)) # Six hours
start_time = (options[:start_time] or (Time.now - duration))
end_time = (options[:end_time] or (Time.now))
get_metric_statistics_options = {
namespace: "AWS/EC2",
metric_name: metric_name,
statistics: [statName],
start_time: start_time.utc.iso8601,
end_time: end_time.utc.iso8601,
period: (options[:period] or (60 * 5)), # Default to 5 min stats
dimensions: (options[:dimensions] or [{name: "InstanceId", value: instance_id}])
}
# Go get stats
result = cw.get_metric_statistics(get_metric_statistics_options)
if ((not result[:datapoints]) or (result[:datapoints].length == 0))
# TODO: What kind of errors can I get back?
puts "\e[33mWarning: Got back no data for instanceId: #{region}:#{instance_id} for metric #{metric_name}\e[0m"
answer = nil
else
# Turn the result into a Rickshaw-style series
data = []
result[:datapoints].each do |datapoint|
point = {
x: (datapoint[:timestamp].to_i), # time in seconds since epoch
y: datapoint[statKey]
}
data.push point
end
data.sort! { |a,b| a[:x] <=> b[:x] }
answer = {
name: "#{metric_name} for #{instance_id}",
data: data
}
end
return answer
end
end
@moonchaser

This comment has been minimized.

Copy link

@moonchaser moonchaser commented Nov 27, 2013

I keep getting

Warning: Got back no data for instanceId: us-west-1a:i-xxxxxxxx for metric CPUUtilization

Do you know why it could be? Its not the authentication because if I put wrong credentials it reports back about it.

Do I need to do anything to enable cloudwatch? From my AWS console I can see cloudwatch graphs.

@giannivh

This comment has been minimized.

Copy link

@giannivh giannivh commented Jan 21, 2014

I'm having the same problem as @moonchaser.

His region is wrong though, "us-west-1a" should be "us-west-1", but the results are the same.

Authentication works, sending the request to get the metric also works, but the result is an empty set. There's a request ID in the result:

datapoints
label
CPUUtilization
response_metadata
{:request_id=>"xxx"}

Any thoughts?

@JorisM

This comment has been minimized.

Copy link

@JorisM JorisM commented Feb 5, 2014

I'm running into the same problem. I get back an empty array for datapoints:
{ :datapoints=>[], :label=>"CPUUtilization", :response_metadata =>{ :request_id=>"xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx"} }

@JorisM

This comment has been minimized.

Copy link

@JorisM JorisM commented Feb 5, 2014

I modified line 40 to make it work with the non default region (in my case eu-west-1):

        if not cw
            cw = @clientCache[region] = AWS::CloudWatch::Client.new({
                server: "monitoring.eu-west-1.amazonaws.com",
                access_key_id: @access_key_id,
                secret_access_key: @secret_access_key,
                region: region
            })
        end
@s0enke

This comment has been minimized.

Copy link

@s0enke s0enke commented Aug 22, 2014

Thanks for the work! I modified the widget/job/lib to work with arbitrary CloudWatch metrics. Check it out at https://gist.github.com/s0enke/68b3288bd1cbec3336ad

@SaMnCo

This comment has been minimized.

Copy link

@SaMnCo SaMnCo commented Aug 22, 2014

For newcomers:
add to Gemfile(and bundle install ) : gem 'aws-sdk'

you need to change the 'jobs/lib_dashing_ec2.rb'(from the gist) under "lib/dashing_ec2.rb"
(assuming you are in your dashboard folder:

mv ./jobs/lib_dashing_ec2.rb ./lib/dashing_ec2.rb

and change the require in the ec2.rb file: require './lib/dashing_ec2'

sed -i.bak 's/dashing-ec2/dashing_ec2/g' jobs/ec2.rb

After this you can start dashing.

@jayemko

This comment has been minimized.

Copy link

@jayemko jayemko commented Sep 30, 2014

Adding for newcomers also:
Line 30 has a typo. Change
statName = "Maxmimum"
to
statName = "Maximum"

@toadkicker

This comment has been minimized.

Copy link

@toadkicker toadkicker commented Jun 7, 2015

For AWS SDK v2 you need to use Aws:: instead of AWS, and auth tokens come from env vars.

require 'aws-sdk'
require 'time'

class DashingEC2

  def initialize(options)
    @clientCache = {}
  end

  # Get statistics for an instance
  #
  # * `instance_id` is the instance to get data about.
  # * `region` is the name of the region the instance is from (e.g. 'us-east-1'.)  See
  #   [monitoring URIs](http://docs.aws.amazon.com/general/latest/gr/rande.html#cw_region).
  # * `metric_name` is the metric to get.  See
  #   [the list of build in metrics](http://docs.aws.amazon.com/AWSEC2/2011-07-15/UserGuide/index.html?using-cloudwatch.html).
  # * `type` is `:average` or `:maximum`.
  # * `options` are [:start_time, :end_time, :period, :dimensions] as per
  #   `get_metric_statistics()`, although all are optional.  Also:
  #   * `:duration` - If supplied, and no start_time or end_time are supplied, then start_time
  #     and end_time will be computed based on this value in seconds.  Defaults to 6 hours.
  def getInstanceStats(instance_id, region, metric_name, type=:average, options={})
    if type == :average
      statName = "Average"
    elsif type == :maximum
      statName = "Maximum"
    end
    statKey = type

    # Get an API client instance
    cw = @clientCache[region]
    if not cw
      cw = @clientCache[region] = Aws::CloudWatch::Client.new
    end

    # Build a default set of options to pass to get_metric_statistics
    duration = (options[:duration] or (60*60*6)) # Six hours
    start_time = (options[:start_time] or (Time.now - duration))
    end_time = (options[:end_time] or (Time.now))
    get_metric_statistics_options = {
        namespace: "AWS/EC2",
        metric_name: metric_name,
        statistics: [statName],
        start_time: start_time.utc.iso8601,
        end_time: end_time.utc.iso8601,
        period: (options[:period] or (60 * 5)), # Default to 5 min stats
        dimensions: (options[:dimensions] or [{name: "InstanceId", value: instance_id}])
    }

    # Go get stats
    metrics_list = cw.list_metrics
    result = cw.get_metric_statistics(get_metric_statistics_options)

    begin
      if ((not result[:datapoints]) or (result[:datapoints].length == 0))
        # TODO: What kind of errors can I get back?
        puts "\e[33mWarning: Got back no data for instanceId: #{region}:#{instance_id} for metric #{metric_name}\e[0m"
        answer = nil
      else
        # Turn the result into a Rickshaw-style series
        data = []

        result[:datapoints].each do |datapoint|
          point = {
              x: (datapoint[:timestamp].to_i), # time in seconds since epoch
              y: datapoint[statKey]
          }
          data.push point
        end
        data.sort! { |a,b| a[:x] <=> b[:x] }

        answer = {
            name: "#{metric_name} for #{instance_id}",
            data: data
        }
      end
    rescue Aws::CloudWatch::Errors::ServiceError => e
      answer = e
    end


    return answer
  end

end
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment