StatsD/Graphite server on AWS/EC2 using Vagrant, Chef Solo, and Berkshelf

README.md

StatsD/Graphite server on EC2 using Vagrant, Chef Solo, and Berkshelf

Dependencies:

• Vagrant 1.2.x installed
• berkshelf gem (gem install berkshelf)
• vagrant-berkshelf plugin installed (vagrant plugin install vagrant-berkshelf)
• vagrant-omnibus plugin installed (vagrant plugin install vagrant-omnibus)
• AWS access key and secret access key
• AWS local keypair
• EC2 Security group(s) allowing for
  • http (80) for the graphite web view
  • udp (8125 by default) the protocol and port used by statsd
  • ssh (22) required in order for vagrant to provision the instance

The berkshelf provides cookbook dependency resolution. Vagrant handles the creating and provisioning of the ec2 instance. Depending on the ami you use, you may have to vagrant ssh into the instance after its created but before its provisioned and install chef (gem install chef). All it takes is the Vagrantfile and Berksfile!

Run

vagrant up --provider aws

Try it out

Open the ec2 instance url in a browser. There you'll see the Graphite interface.

Send it some stats:

shell:

gem install statsd-ruby
irb

ruby:

require 'statsd-ruby'
statsd = Statsd.new "<your ec2 instance host name>"
# Incrementer:
statsd.increment "hello.world"
# Timer
statsd.timing 'glork', 320
# Guage
statsd.gauge 'bork', 100

Run those statsd commands multiple times and if you're watching in Graphite you can see the new stats come in shortly after they happen.

Berksfile

site :opscode
cookbook 'apt'
cookbook 'statsd', git: 'https://github.com/librato/statsd-cookbook.git'
cookbook 'graphite', git: 'https://github.com/hw-cookbooks/graphite.git'

Vagrantfile

# -*- mode: ruby -*-
# vi: set ft=ruby :
VAGRANTFILE_API_VERSION = "2"
Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
  config.omnibus.chef_version = :latest
  config.vm.box = "dummy"
  config.vm.box_url = "https://github.com/mitchellh/vagrant-aws/raw/master/dummy.box"
  
  # Provider
  config.vm.provider :aws do |aws, override|
    aws.access_key_id = "<redacted>"
    aws.secret_access_key = "<redacted>"
    aws.keypair_name = "<redacted>"
    aws.ami = "ami-5f2abc6f" # "Ubuntu 12.04 LTS with Chef"
    aws.region = "us-west-2"
    aws.instance_type = "t1.micro"
    aws.security_groups = ["Basic", "StatsD"]
    override.ssh.username = "ubuntu"
    override.ssh.private_key_path = "path/to/your/key.pem"
    aws.tags = {
      'Name' => 'Stats (Vagrant Provision)'
    }
  end

  # Provisioning
  config.berkshelf.enabled = true
  config.vm.provision :chef_solo do |chef|
    chef.add_recipe "apt"
    chef.add_recipe "statsd"
    chef.add_recipe "graphite"
  end
end

Also, librarian could easily be used in place of Berkshelf. It too has a vagrant plugin.