Skip to content

Instantly share code, notes, and snippets.

@brasey

brasey/Create an AMI using Packer.md

Last active September 16, 2019 15:43
Show Gist options
  • Star 8 You must be signed in to star a gist
  • Fork 1 You must be signed in to fork a gist
  • Save brasey/50fa03e8b8266cb7dc04 to your computer and use it in GitHub Desktop.
Save brasey/50fa03e8b8266cb7dc04 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
Create an AMI using Packer.md

Create an AMI using Packer

What is Packer?

Packer is a tool from Hashicorp that allows you to create machine images in a variety of formats from a single spec file. This can be a very powerful thing when you want to automate the deployment of applications to a variety (or even just more than one, really) platforms.

In our case, we want to deploy an application to VMware and AWS, and soon to Docker. Doing that with a single spec file gives us

  • platforms that are consistent, even if they're different
  • images that can be created via a build pipeline
  • images that can be tested automatically for functionality and security
  • platforms that are created from code, which can be version-controlled

It's easy to get started using the Packer tutorial.

Let's make some AMI!

Notes

I'm using Fedora 20. Fedora has a binary called packer already installed at /usr/sbin/packer. Don't let that confuse you.

The Packer tutorial is really self-explanatory. These are just my notes, so they might be a little me-specific.

I didn't have a personal AWS account. I had an Amazon account, obviously, but no AWS. They have a free tier that allows you to do a lot of things, though, so I signed up for that. This is where my steps start.

This template will actually install Redis and bake it into the image. Magic! Imagine some Puppet happening in this space.

I tried to build a Digital Ocean droplet in parallel, but the DO builder is flaky. The DO API was just updated to v2, and you can deploy using v1 or v2 (depending on how you specify tokens), but either way I tried was a rabbit hole. I didn't have time or energy to pursue it, but it might be interesting to pick up later.

Steps

Download Packer and put all the parts in /bin. From now on, call Packer by doing **/bin/packer** so we get the right one.

Set up AWS access
  1. go to AWS IAM in the web UI
  2. Add a user
  3. Add an admin-level group and put the user in it
  4. Download user credentials (access keys)

Put credentials in ~/.aws/config like this:

[default]
aws_access_key_id = foo
aws_secret_access_key = foo
output = text
region = us-east-1

[profile prod]
aws_access_key_id = foo
aws_secret_access_key = foo
output = text
region = us-east-1

[profile personal]
aws_access_key_id = foo
aws_secret_access_key = foo
output = text
region = us-east-1

That way you can use aws cli and just add --profile prod (for example) to use a particular profile.

Do some Packer

Create a directory to work in.

mkdir ~/working/packer
cd ~/working/packer

Create the example template from the tutorial

vim example.json
{
  "variables": {
    "aws_access_key": "",
    "aws_secret_key": ""
  },
  "builders": [{
    "type": "amazon-ebs",
    "access_key": "{{user `aws_access_key`}}",
    "secret_key": "{{user `aws_secret_key`}}",
    "region": "us-east-1",
    "source_ami": "ami-de0d9eb7",
    "instance_type": "t1.micro",
    "ssh_username": "ubuntu",
    "ami_name": "packer-example {{timestamp}}"
  }],
  "provisioners": [{
    "type": "shell",
    "inline": [
      "sleep 30",
      "sudo apt-get update",
      "sudo apt-get install -y redis-server"
    ]
  }]
}

Create a file to include your AWS auth info

vim credentials.json
{
  "aws_access_key": "foo",
  "aws_secret_key": "foo"
}

Now do what you came here for:

$ ~/bin/packer build --var-file credentials.json example.json
amazon-ebs output will be in this color.

==> amazon-ebs: Inspecting the source AMI...
==> amazon-ebs: Creating temporary keypair: packer 5463ae64-2291-1629-e57a-b57ad9fdd779
==> amazon-ebs: Creating temporary security group for this instance...
==> amazon-ebs: Authorizing SSH access on the temporary security group...
==> amazon-ebs: Launching a source AWS instance...
    amazon-ebs: Instance ID: i-37d763d6
==> amazon-ebs: Waiting for instance (i-37d763d6) to become ready...
==> amazon-ebs: Waiting for SSH to become available...
==> amazon-ebs: Connected to SSH!
==> amazon-ebs: Provisioning with shell script: /tmp/packer-shell001525607
    amazon-ebs: Ign http://us-east-1.ec2.archive.ubuntu.com precise InRelease
    amazon-ebs: Ign http://security.ubuntu.com precise-security InRelease
    amazon-ebs: Ign http://us-east-1.ec2.archive.ubuntu.com precise-updates InRelease
    amazon-ebs: Get:1 http://security.ubuntu.com precise-security Release.gpg [198 B]
    amazon-ebs: Hit http://us-east-1.ec2.archive.ubuntu.com precise Release.gpg
    amazon-ebs: Get:2 http://security.ubuntu.com precise-security Release [53.0 kB]
    amazon-ebs: Get:3 http://us-east-1.ec2.archive.ubuntu.com precise-updates Release.gpg [198 B]
    amazon-ebs: Hit http://us-east-1.ec2.archive.ubuntu.com precise Release
    amazon-ebs: Get:4 http://us-east-1.ec2.archive.ubuntu.com precise-updates Release [194 kB]
    amazon-ebs: Get:5 http://security.ubuntu.com precise-security/main Sources [114 kB]
    amazon-ebs: Get:6 http://security.ubuntu.com precise-security/universe Sources [33.0 kB]
    amazon-ebs: Get:7 http://security.ubuntu.com precise-security/main amd64 Packages [440 kB]
    amazon-ebs: Get:8 http://us-east-1.ec2.archive.ubuntu.com precise/main Sources [934 kB]
    amazon-ebs: Get:9 http://security.ubuntu.com precise-security/universe amd64 Packages [101 kB]
    amazon-ebs: Get:10 http://us-east-1.ec2.archive.ubuntu.com precise/universe Sources [5,019 kB]
    amazon-ebs: Get:11 http://security.ubuntu.com precise-security/main i386 Packages [474 kB]
    amazon-ebs: Get:12 http://security.ubuntu.com precise-security/universe i386 Packages [107 kB]
    amazon-ebs: Get:13 http://security.ubuntu.com precise-security/main TranslationIndex [208 B]
    amazon-ebs: Get:14 http://security.ubuntu.com precise-security/universe TranslationIndex [205 B]
    amazon-ebs: Get:15 http://security.ubuntu.com precise-security/main Translation-en [200 kB]
    amazon-ebs: Get:16 http://security.ubuntu.com precise-security/universe Translation-en [61.5 kB]
    amazon-ebs: Hit http://us-east-1.ec2.archive.ubuntu.com precise/main amd64 Packages
    amazon-ebs: Hit http://us-east-1.ec2.archive.ubuntu.com precise/universe amd64 Packages
    amazon-ebs: Hit http://us-east-1.ec2.archive.ubuntu.com precise/main i386 Packages
    amazon-ebs: Hit http://us-east-1.ec2.archive.ubuntu.com precise/universe i386 Packages
    amazon-ebs: Hit http://us-east-1.ec2.archive.ubuntu.com precise/main TranslationIndex
    amazon-ebs: Hit http://us-east-1.ec2.archive.ubuntu.com precise/universe TranslationIndex
    amazon-ebs: Get:17 http://us-east-1.ec2.archive.ubuntu.com precise-updates/main Sources [480 kB]
    amazon-ebs: Get:18 http://us-east-1.ec2.archive.ubuntu.com precise-updates/universe Sources [111 kB]
    amazon-ebs: Get:19 http://us-east-1.ec2.archive.ubuntu.com precise-updates/main amd64 Packages [842 kB]
    amazon-ebs: Get:20 http://us-east-1.ec2.archive.ubuntu.com precise-updates/universe amd64 Packages [249 kB]
    amazon-ebs: Get:21 http://us-east-1.ec2.archive.ubuntu.com precise-updates/main i386 Packages [875 kB]
    amazon-ebs: Get:22 http://us-east-1.ec2.archive.ubuntu.com precise-updates/universe i386 Packages [256 kB]
    amazon-ebs: Get:23 http://us-east-1.ec2.archive.ubuntu.com precise-updates/main TranslationIndex [10.6 kB]
    amazon-ebs: Get:24 http://us-east-1.ec2.archive.ubuntu.com precise-updates/universe TranslationIndex [8,333 B]
    amazon-ebs: Hit http://us-east-1.ec2.archive.ubuntu.com precise/main Translation-en
    amazon-ebs: Hit http://us-east-1.ec2.archive.ubuntu.com precise/universe Translation-en
    amazon-ebs: Get:25 http://us-east-1.ec2.archive.ubuntu.com precise-updates/main Translation-en [370 kB]
    amazon-ebs: Get:26 http://us-east-1.ec2.archive.ubuntu.com precise-updates/universe Translation-en [145 kB]
    amazon-ebs: Fetched 11.1 MB in 8s (1,236 kB/s)
    amazon-ebs: Reading package lists... Done
    amazon-ebs: Reading package lists... Done
    amazon-ebs: Building dependency tree
    amazon-ebs: Reading state information... Done
    amazon-ebs: The following NEW packages will be installed:
    amazon-ebs: redis-server
    amazon-ebs: 0 upgraded, 1 newly installed, 0 to remove and 150 not upgraded.
    amazon-ebs: Need to get 204 kB of archives.
    amazon-ebs: After this operation, 523 kB of additional disk space will be used.
    amazon-ebs: Get:1 http://us-east-1.ec2.archive.ubuntu.com/ubuntu/ precise/universe redis-server amd64 2:2.2.12-1build1 [204 kB]
    amazon-ebs: Fetched 204 kB in 0s (364 kB/s)
    amazon-ebs: Selecting previously unselected package redis-server.
    amazon-ebs: (Reading database ... 47420 files and directories currently installed.)
    amazon-ebs: Unpacking redis-server (from .../redis-server_2%3a2.2.12-1build1_amd64.deb) ...
    amazon-ebs: Processing triggers for initramfs-tools ...
    amazon-ebs: update-initramfs: Generating /boot/initrd.img-3.2.0-38-virtual
    amazon-ebs: Processing triggers for ureadahead ...
    amazon-ebs: Processing triggers for man-db ...
    amazon-ebs: Setting up redis-server (2:2.2.12-1build1) ...
    amazon-ebs: Starting redis-server: redis-server.
==> amazon-ebs: Stopping the source instance...
==> amazon-ebs: Waiting for the instance to stop...
==> amazon-ebs: Creating the AMI: packer-example 1415818852
    amazon-ebs: AMI: ami-6c8f1a04
==> amazon-ebs: Waiting for AMI to become ready...
==> amazon-ebs: Terminating the source AWS instance...
==> amazon-ebs: Deleting temporary security group...
==> amazon-ebs: Deleting temporary keypair...
Build 'amazon-ebs' finished.

==> Builds finished. The artifacts of successful builds are:
--> amazon-ebs: AMIs were created:

And that's all there is to it. You just created an AMI with Redis installed on it. Almost seems anti-climactic.

Proof that the AMI is there:

$ aws --profile personal ec2 describe-images --owners <your ID here>
IMAGES x86_64 xen ami-ac9e0bc4 <ID redacted>/packer-example 1415809098 machine aki-88aa75e1 packer-example 1415809098 <ID redacted> False /dev/sda1 ebs available paravirtual
BLOCKDEVICEMAPPINGS /dev/sda1 
EBS True False snap-52661fee 8 standard
BLOCKDEVICEMAPPINGS /dev/sdb ephemeral0
Cleanup

If you don't want to be billed a few cents a month for the AMI you just created, get rid of it.

$ aws --profile personal ec2 deregister-image --image-id ami-ac9e0bc4
true

In addition to the AMI, there's a snapshot associated with the AMI as well. Determine what its ID is and kill it.

$ aws --profile personal ec2 describe-snapshots --owner-ids <your ID here>
SNAPSHOTS Created by CreateImage(i-ca982d2b) for ami-ac9e0bc4 from vol-bca67ca4 False <ID redacted> 100% snap-52661fee 2014-11-12T16:19:41.000Z completed vol-bca67ca4 8
$ aws --profile personal ec2 delete-snapshot --snapshot-id snap-52661fee
true

Conclusion

This is just a taste of what I think Packer can help us do.

Imagine a single JSON file that describes producing an AMI, a VMDK and a Docker image.

Imagine that Packer JSON being called as part of a Jenkins build that watches a Git repo that holds your Puppetry.

Now imagine running a barrage of tests on your newly created images checking the state of your platform, maybe using serverspec or Guardrail to define your policies and test.

And finally, imagine that image being consumed by your code deploy Jenkins pipeline. Everything is installed and configured and tested already but the code.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment