Skip to content

Instantly share code, notes, and snippets.

@akurkin

akurkin/README.md

Forked from dergachev/README.md
Created March 10, 2014 11:02
Show Gist options
  • Save akurkin/9463064 to your computer and use it in GitHub Desktop.
Save akurkin/9463064 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
README.md

Vagrant Setup

This tutorial guides you through creating your first Vagrant project.

We start with a generic Ubuntu VM, and use the Chef provisioning tool to:

  • install packages for vim, git
  • create user accounts, as specified in included JSON config files
  • install specified user dotfiles (.bashrc, .vimrc, etc) from a git repository

Afterwards, we'll see how easy it is to package our newly provisioned VM environment into a new "Vagrant box", which can be instantly deployed for new projects, and shared with others.

Install Vagrant

For a backgrond on Vagrant, see:

Download and install VirtualBox from https://www.virtualbox.org/wiki/Downloads Download and install Vagrant from http://downloads.vagrantup.com

Setup up a Vagrant project:

Download the Precise Pangolin Ubuntu 12.04 base vagrant box, see http://vagrantbox.es/

    vagrant box add precise64 http://files.vagrantup.com/precise64.box # 323MB, faster download

Initialize a new Vagrant project in ~/code/vagrant-tutorial, based on the precise64 box we just installed:

    mkdir -p ~/code/vagrant-tutorial/ & cd ~/code/vagrant-tutorial/
    vagrant init precise64 # creates a default Vagrantfile in the current directory

Creating a VM

You can now ask Vagrant to start up a VM as configured by the default Vagrantfile:

  vagrant up

The VM is now running in Virtualbox. You can ssh into it (no password required) as follows:

  vagrant ssh # ssh in to the VM

Note that on the new VM, /vagrant is a shared directory linked with ~/code/vagrant-tutorial

  ls /vagrant #shared folder mounted to your project root

Soon, you'll start making modifications to the Vagrantfile. There are three ways to ask Vagrant to rebuild the VM.

  # Fastest method: re-runs the provisioner (eg chef-solo) without stopping the VM.
  vagrant provision 

  # Restarts VM, ru-provisions. Use this if you changed virtualbox settings (eg shared folders)
  vagrant reload 

  # Destroys the active VM, and rebuilds from the base box.
  # Slow, but guarantees stability.
  vagrant destroy --force && vagrant up # deletes the VM and rebuilds

Provision VM customizations using Chef

The main benefit of Vagrant is to easily provision VMs with your customizations, which generally include the following:

  • installing packages (apt-get install vim)
  • updating configuration files (/etc/apache2/httpd.conf)
  • setting up user accounts
  • running various scripts

Vagrant calls this process "provisioning", and supports several popular tools including Puppet and Chef.

This tutorial will use Chef-solo, a single-server version of Chef. Chef calls provisioning scripts "recipes", and related recipes are grouped into "cookbooks". (Get the idea?)

For this tutorial, we will use chef-solo to provision the following Chef cookbooks:

  • vim: basically runs 'sudo apt-get install vim'
  • git: basically runs 'sudo apt-get install git'
  • users: creates user accounts from JSON configuration files
  • homesick: installs public git repositories for each user's dotfiles (eg .vimrc, .bashrc)

The following additional cookbooks are required to support dotfiles repositories requiring SSH key authentication via agent forwarding:

  • homesick_agent
  • extend_lwrp
  • root_ssh_agent
  • ssh_known_hosts

For more information about how the custom cookbooks work (and for my notes about learning chef), see the following writeups:

Install and configure cookbooks

Create sub-directories for cookbooks (chef deployment scripts) and databags (JSON config files):

  mkdir ~/code/vagrant-tutorial/site-cookbooks
  mkdir ~/code/vagrant-tutorial/cookbooks
  mkdir ~/code/vagrant-tutorial/databags

Install the following community-maintained cookbooks to cookbooks:

  cd ~/code/vagrant-tutorial/cookbooks
  git clone git://github.com/fnichol/chef-homesick.git homesick
  git clone git://github.com/opscode-cookbooks/ssh_known_hosts.git
  git clone git://github.com/opscode-cookbooks/vim.git
  git clone git://github.com/opscode-cookbooks/git.git

In the course of creating this tutorial, I created the following custom ("site-specific") cookbooks. Install them into site-cookbooks:

  cd ~/code/vagrant-tutorial/site-cookbooks
  git clone git://github.com/dergachev/chef_homesick_agent.git homesick_agent
  git clone git://github.com/dergachev/chef_root_ssh_agent.git root_ssh_agent
  git clone git://github.com/dergachev/chef_users.git users
  git clone git://github.com/dergachev/chef_extend_lwrp.git extend_lwrp

When installing cookbooks, be mindful that cookbooks folder names must match the cookbook name.

  • Wrong: git clone git://github.com/dergachev/chef_users.git
  • Correct: clone git://github.com/dergachev/chef_users.git users

Install databags

The users, homesick, and ssh_known_hosts cookbooks require JSON configuration files to be placed in the following sub-directories:

  mkdir -p ~/code/vagrant-tutorial/databags/users
  mkdir -p ~/code/vagrant-tutorial/databags/ssh_known_hosts

For each user to be created, you'll need to provide a config file at databags/users/USERNAME.json. If a user has a dotfiles directory which should be installed (homesick calls them castles), you'll need to insert an addtional 'homesick_castles' property.

Sample databags/users/testuser.json

For example, save the following into databags/users/testuser.json, taking care to modify the following fields:

{
  "groups": [
    "sysadmin"
  ],
  "comment": "Test User Name",
  "password": "$6$BcvQ4/W1iMlP9S33$k4RmfftqRi1I5T.z113L1VrXX0K78Uwii8Ot4WC1p74m2agZHYqfp9eNYG10B6adrQIEJ4jQyagJiMt7q9MiF.",
  "ssh_keys": [
    "ssh-rsa AAA456...uvw== testuser@testdomain.com"
  ],
  "id": "alex",
  "homesick_castles": [
    {
      "name": "dotfiles",
      "source": "git://github.com/technicalpickles/dotpickles.git"
    }
  ],
  "shell": "/bin/bash",
  "email": "testuser@testdomain.com"
}

For more information about the respective databag formats, see

Sample databags/ssh_known_hosts/github.json

If any homesick_castle entry contains a SSH git URL (necessary for read-write access, generally of the form git@github.com:technicalpickles/dotpickles.git), you will need to add that host key signature into databags/ssh_known_hosts/hostname.json.

For example, if your git repository server is github.com, create databags/ssh_known_hosts/github.json as follows:

{
"id": "github",
"fqdn": "github.com",
"rsa": "AAAAB3NzaC1yc2EAAAABIwAAAQEAq2A7hRGmdnm9tUDbO9IDSwBK6TbQa+PXYPCPy6rbTrTtw7PHkccKrpp0yVhp5HdEIcKr6pLlVDBfOLX9QUsyCOV0wzfjIJNlGEYsdlLJizHhbn2mUjvSAHQqZETYP81eFzLQNnPHt4EVVUh7VfDESU84KezmD5QlWpXLmvU31/yMf+Se8xhHTvKSCZIFImWwoG6mbUoWf9nzpIoaSjB+weqqUUmpaaasXVal72J+UX2B+2RPW3RcT0eOzQgqlJL3RKrTJvdsjE3JEAvGq3lGHSZXy28G3skua2SmVi/w4yCE6gbODqnTWlg7+wC604ydGXA8VJiS5ap43JXiUFFAaQ==",
"ipaddress": "207.97.227.239"
}

If your git host is git.mycompany.com, you should create git-mycompany.json and populate it with values derived from cat ~/.ssh/known_hosts | grep git mycomapny.com. For more info, see https://github.com/dergachev/chef_homesick_agent#install-ssh_known_hosts-cookbook

Install Vagrantfile

With this done, you need specify which cookbooks and recipes Vagrant should deploy, via the chef-solo provisioner. To do this, replace the default Vagrantfile (~/code/vagrant-tutorial/Vagrantfile) with the following:

Vagrant::Config.run do |config|
  config.vm.box = "precise64"

  # Necessary for homesick_agent::data_bag
  config.ssh.forward_agent = true

  config.vm.provision :chef_solo do |chef|

    # contains "users" and "ssh_known_hosts" databags
    chef.data_bags_path = "databags"

    chef.cookbooks_path = ["cookbooks", "site-cookbooks"]

    # stuff that should be in base box
    chef.add_recipe "vim"
    chef.add_recipe "git"
    
    # setup users (from data_bags/users/*.json)
    chef.add_recipe "users::ruby_shadow" # necessary for password shadow support
    chef.add_recipe "users::sysadmins" # creates users and sysadmin group
    chef.add_recipe "users::sysadmin_sudo" # adds %sysadmin group to sudoers

    # homesick_agent and its dependencies
    chef.add_recipe "root_ssh_agent::ppid" # maintains agent during 'sudo su root'
    chef.add_recipe "ssh_known_hosts" # populates /etc/ssh/ssh_known_hosts from data_bags/ssh_known_hosts/*.json
    chef.add_recipe "homesick_agent::data_bag" # includes homesick::data_bag
    
    # instruct "homesick::data_bag" to install dotfiles for the user 'testuser'
    chef.json = {
       :users => ['testuser']
    }

    # chef.log_level = :debug

  end
end

Be sure to modify the chef.json[:users] property to reflect the user(s) you're configuring in your databags.

Launch the VM

If you did everything right, you are ready to start the VM!

  vagrant destroy --force && vagrant up

It'll take a minute to start the VM, and then several more to run chef-solo to install all the cookbooks and recipes specified in Vagrantfile.

It's entirely possible that something goes wrong. Pay close attention to chef errors, to figure out which installation step wasn't performed correctly. If you get stuck, consider uncommenting the following in Vagrantfile:

  chef.log_level = :debug

As you tweak things, you'll generally need to re-run vagrant provision to see if the errors go away. Keep in mind that chef will halt on the first error, so often you'll fix one error only to stumble upon the next one. In some cases, you'll need to run vagrant destroy --force ; vagrant up, to really start from a clean slate.

Good luck!

Connect to VM

Once you've resolved any chef errors After doing that, ssh to the VM as as one of the newly-created users

  vagrant ssh -p -- -l testuser   #replace 'testuser' with your username
  ls -al ~ # dotfiles should be symlinked to ~/.homesick

Note that vagrant also port forwards localhost:8080 to vagrant:80. So if you have a web server setup on your VM, you will be able to acess it at http://127.0.0.1:8080.

Package a customized Vagrant box

At this point, we can package our customizations to the precise64 base Vagrant box into precise64-customized, to use as the box for future projects. These steps follow the Vagrant packaging tutorial: http://vagrantup.com/v1/docs/getting-started/packaging.html

Note: before trying this for the second time, you'll need to remove previously exported boxes as follows:

  cd ~/code/vagrant-tutorial
  rm ~/code/vagrant-tutorial/package.box
  vagrant box remove precise64-customized

First, create this file: ~/code/vagrant-tutorial/Vagrantfile.pkg as follows:

Vagrant::Config.run do |config|
  config.vm.forward_port 80, 8080
  config.ssh.forward_agent = true # important for recipe[homesick_agent::data_bag]
end

Now export the currently running VM, bundled with the above Vagrantfile.pkg:

  vagrant package --vagrantfile Vagrantfile.pkg #creates ~/code/vagrant-tutorial/package.box

Now add it as precise64-customized to your user's set of available boxes (stored in ~/.vagrant.d/boxes/):

  vagrant box add precise64-customized ~/code/vagrant-tutorial/package.box

If that worked, you're now ready to test it with a new project:

  mkdir -p ~/code/new-vagrant-project ; cd ~/code/new-vagrant-project
  # generate a Vagrantfile that includes config.vm.box="precise64-customized"
  vagrant init precise64-customized 
  vagrant up

Now test that everything works correctly (including user deployment):

  cd ~/code/new-vagrant-project
  vagrant ssh -p -- -l testuser # replace 'testuser' with your username
  ls -al ~    # various dotfiles should be symlinked to ~/.homesick

That's it! (Hopefully).

For info about bundling Vagrantfiles and Vagrantfile overloading, see http://vagrantup.com/v1/docs/vagrantfile.html#vagrantfile_load_order

Building a base box from scratch

If you want to build a base box VM from scratch, see the following:

Also check out VeeWee, a tool to create a Vagrant base box from a distribution ISO image:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment