Post for AWS Advent about Puppet and Security Groups

puppetawsadvent.md

Managing EC2 Security Groups using Puppet

At Puppet Labs we recently shipped a module to make managing AWS easier. This tutorial shows how it can be used to manage your security groups. EC2 Security groups act as a virtual firewall and are used to isolate instances and other AWS resources from each other and the internet.

An example

You can find the full details about installation and configuration for the module in the official README but the basic version, assuming a working Puppet and Ruby setup, is:

gem install aws-sdk-core
puppet module install puppetlabs-aws

You’ll also want to have your AWS API credentials in environment variables (or use IAM if you’re running from within AWS).

export AWS_ACCESS_KEY_ID=your_access_key_id
export AWS_SECRET_ACCESS_KEY=your_secret_access_key

First lets create a simple security group called test-sg in the us-east-1 region. Save the following to a file called securitygroup.pp:

ec2_securitygroup { 'test-sg':
  region      => 'us-east-1',
  ensure      => present,
  description => 'Security group for aws advent',
  ingress     => [{
    security_group => 'test-sg',
  }],
  tags        => {
    reason => 'awsadvent',
  },
}

Now lets run Puppet to create the group:

puppet apply securitygroup.pp --test

You should see something like the following output:

Info: Loading facts
Notice: Compiled catalog for pro.local in environment production in 0.05 seconds
Info: Applying configuration version '1418659587'
Info: Checking if security group test-sg exists in region us-east-1
Info: Creating security group test-sg in region us-east-1
Notice: /Stage[main]/Main/Ec2_securitygroup[test-sg]/ensure: created
Notice: Finished catalog run in 15.22 seconds

We’re running here with apply and the --test flag so we can easily see what’s happening, but if you have a Puppet master setup you can run with an agent too.

You will probably change your security groups over time as you’re infrastructure evolves. And managing that evolution is where Puppet’s declarative approach really shines. You can have confidence in the description of your infrastructure in code because Puppet can tell you about any changes when it runs.

Next lets add a new ingress rule to our existing group. Modify the securitygroup.pp file like so:

ec2_securitygroup { 'test-sg':
  ensure      => present,
  region      => 'us-east-1',
  description => 'Security group for aws advent',
  ingress     => [{
    protocol => 'tcp',
    port     => 80,
    cidr     => '0.0.0.0/0',
  },{
    security_group => 'test-sg',
  }],
  tags        => {
    reason => 'awsadvent',
  },
}

And again lets run Puppet to modify the group:

puppet apply securitygroup.pp --test

You should see something like the following output:

Info: Loading facts
Notice: Compiled catalog for pro.local in environment production in 0.04 seconds
Info: Applying configuration version '1418659692'
Info: Checking if security group test-sg exists in region us-east-1
Notice: /Stage[main]/Main/Ec2_securitygroup[test-sg]/ingress: ingress changed [{'security_group' => 'test-sg'}] to '{"protocol"=>"tcp", "port"=>"80", "cidr"=>"0.0.0.0/0"} {"security_group"=>"test-sg"}'
Notice: Finished catalog run in 13.59 seconds

Note the information about changes to the ingress rules as we expected. You can also check the changes in the AWS console.

The module also has full support for the Puppet resource command, so all of the functionality is available from the command line as well as the DSL. As an example lets clean-up and delete the group created above.

puppet resource ec2_securitygroup test-sg ensure=absent region=us-east-1

Hopefully that’s given you an idea of what’s possible with the Puppet AWS module. You can see more examples of the module in action in the main repository.

Advantages

Some of the advantages of using Puppet for managing AWS resources are:

• The familiar DSL - if you’re already using Puppet the syntax will already be familiar, if you’re not already using Puppet you’ll find lots of good references and documentation
• Puppet is a declarative tool - Puppet is used to declare the desired state of the world, this means it’s useful for maintaining state and changing resources over time, as well as creating new groups
• Existing tool support - whether it’s the Geppetto IDE, testing tools like rspec-puppet or syntax highlighting for your favourite editor lots of supporting tooling already exists

The future

The current preview release of the module supports EC2 instances, security groups and ELB load balancers, with work on support for VPC, Route53 and Autoscaling Groups available soon. We’re looking for as much feedback as possible at the moment so feel free to report issues on GitHub), ask questions on the puppet-user mailing list or contact me on twitter at @garethr