My ansible approach

ansible_approach.md

My ansible approach

Per service playbooks

Playbooks are written per service : a playbook is a collection of tasks, and eventually associated handers, templates, and files, that are required to properly install and setup a service.

Each playbook has a setup.yml entry point, which is responsible of including various necessary tasks to get the service up and running. Typically, this means installing the service and configuring it.

Here is an entry point example for a tmux playbook :

---
- hosts: debian

  tasks:
    - include: tasks/install.yml
    - include: tasks/configure.yml

Install task handles package installation and adds pair programming scripts scripts in /usr/local/bin :

---
  - name: Tmux | Installs tmux
    action: apt pkg=tmux state=latest
    action: ping
    only_if: ${tmux.install}
    tags: 
      - tmux

  - name: Tmux | Adds tmux pairing scripts
    action: copy src=files/$item dest=/usr/local/bin/ owner=root group=root mode=0777
    only_if: ${tmux.install}
    with_items:
      - tmux-attach
      - tmux-start
      - tmux-stop
    tags: 
      - tmux

Here, these actions are only ran if tmux.install is set to "yes" or True somewhere in the group/host vars. More on that later.

Then the setup tasks will push configuration files to selected users :

---

    - name: Tmux | Gets installed Tmux version
      shell: tmux -V || echo 0.0
      only_if: ${tmux.install}
      register: tmux_version

    - name: Tmux | Adds .tmux.conf for all accounts for version ${tmux_version.stdout}
      action: template src=templates/tmux.conf.j2 dest=~${item.username}/.tmux.conf owner=${item.username} group=${item.username}
      with_items: ${accounts}
      only_if: ${tmux.install} and is_set('$tmux_version')
      tags:
        - tmux

The first task tries to get the installed tmux version (ince (not so) old tmux version do not have the -V switch, the || trick will set the version number to 0.0 if not set). The version is then registered and is used in the template to add certain directives depending on tmux version.

The second task installs a tmux.conf file in several home directories (accounts, defined also somewhere in the group/host vars). The template is generated only if tmux_version is set. This prevents template generation to choke if we run in check mode (which skips the previous shell: action and thus doesn't register tmux_version). Also, tmux

Again, those two tasks are ran only if tmux.install is set to "yes" or True somewhere.

Default behaviour and variables overriding

I use ansible with the hash_behaviour=merge in ansible.cfg. This means that if you define a hash i a group, a subsequent redefinition of this hash will merge keys of both hashes. If you have identical keys in both, the keys of the last defined hash will override previously defined ones.

For instance, let's say that, by default, we don't want tmux installed. We just need to define the following in group_vars/all :

tmux:
  install: no

However, if we want to install tmux on all debian systems, we could have in group_vars/debian :

tmux:
  install: yes

In the same vein, I usually have several keys under a "service namespace" (tmux here). For instance, let's say I have this in the 'all' vars file :

tmux:
    install: no
    prefix: C-b

I could then, at a refined level (a specific host, or a specific group), have tmux installed with a different key binding just by changing few keys :

tmux:
    install: yes
    prefix: C-x

As you might guess, I try to mutualize common vars/hash keys at the highest possible level, but each host ends up with some kind of "identity card" consisting in a bunch of variables, for instance :

accounts:
  - username: gina
    name: "Ski Instructor"
    email: "gina@example.org"
  - username: root
    name: "Rooooot"
    email: "root@example.org"
  - username: caroline
    name: "Drop out"
    email: "caro@example.org"

mysql:
  bind_address: "127.0.0.1" 
  key_buffer: "16M"
  keep_backup_count: 30
  root_password: "thisisthepass"
  users:
    - name: db_user1
      password: "omg"
      privs: "some_database.*:ALL"

network_interfaces_ipv4:
  eth0:
    address: 192.168.0.2
    netmask: 255.255.255.192
    gateway: 192.168.0.1
    firewall: true

ruby:
  install: yes
  current: "1.9.3-p374"
  deploy_user: "caroline"

ssh:
  allow_users:
    - root
    - git
  local_admins:
    # this will pull out james' key from top level defined 'admins'
    # hash and add it to root's authorized_keys
    - james

For instance, DNS servers are defined in group_vars/all, but coul easily be overriden for this specific host by adding :

# List of nameservers
nameservers:
  - 10.12.13.14
  - 10.11.12.13

in the hosts playbook.

Host playbook collection

In order to run specific playbooks on specific hosts, I use a "top level" playbook per host. It's a kind of "pivot table" between hosts and service playbooks. For instance, here is a playbook dedicated for a host :

---
- include: apt/setup.yml
- include: system/setup.yml
- include: ssh/setup.yml
- include: network/setup.yml
- include: bash/setup.yml
- include: git/setup.yml
- include: htop/setup.yml
- include: tmux/setup.yml
- include: postfix/setup.yml
- include: grub/setup.yml
- include: sudo/setup.yml
- include: mysql/setup.yml
- include: ruby/setup.yml

Whan I want this host to be configured according the current policies, I just :

ansible-playbook -l host.example.org host.e.o.yml

Doing it this way ensures that playbook contains absolutely no information about specific hosts or groups (besides OS information of course). This makes the service playbooks adhere to the Single Responsability Principle : they focus on the service only, not on what is required on the target machines.

Of course, this requires extensive templating where cases regarding all hosts are accounted for. However, this is a continuous process. Templates are changed when needs arise and get more and more complete as requirements increase.

However, this might have limitations. Some machines require very specific configurations. Adding more templating might not be the be-all and end-all solution to the problem. I only deployed rather simple services for now, so I don't know how extensible this approach is to more complex services.

This "pivot playbook" could be better expressed like this :

---
- hosts: all
  include: apt/setup.yml
  include: system/setup.yml
  include: ssh/setup.yml
  include: network/setup.yml
  include: bash/setup.yml
  include: sudo/setup.yml

- hosts: http*.example.com
  include: git/setup.yml
  include: htop/setup.yml
  include: tmux/setup.yml
  include: postfix/setup.yml
  include: grub/setup.yml

- hosts: dbservers*.example.com
  include: mysql/setup.yml

- hosts: webapp[1-10].example.com
  include: ruby/setup.yml
  ...

However this is not currently possible IMHO.

Thoughts ?

extensive_variables_examples.yml

#
# accounts: accounts that will receive customization (bash, git, etc...)
#
accounts:
  - username: ginagina
    name: "Ski Instructor"
    email: "gina@exampe.org"
  - username: root
    name: "rooooot"
    email: "root@example.org"

#
# admins: all admins on the machine, along with their IP and ssh key
#
# If admins have multiple keys or multiple IPs, they must be listed 
# several times (each combination must be listed)
# NOTE: This variable should only be set in group_vars/all. Just use 
# ssh.local_admins to select who you want to allow in (see ssh)
#
admins:
  james:
    ip: 192.168.0.228
    key: ssh-dss AAAAB3NzaC1kc3MAAACBAJhZLMFYqW...HyFFZb1VaZgXAY+g== james@example.org

#
# ansible_python_interpreter: python path on the node
#
# Only useful on Arch hosts
#
# ansible_python_interpreter: /usr/bin/python

#
# backup_server: backup server ip address
#
# used in firewall rules generation
#
backup_server: 192.168.0.12

#
# bash: bash related config
# 
bash:
  # This let's you disable UTF8 symbols in bash prompt
  utf8_prompt: yes

#
# dns_master
#
# IP address of dns master in case we are DNS server
#
dns_master: 192.168.0.11

#
# firewall: firewall related rules
#
firewall:
  rules_file: "/etc/network/iptables"
  unfilter_tcp_out: no
  additional_rules:
    - "-I TCP_IN -m tcp -p tcp --dport 80 -j ACCEPT"
    - "-I TCP_IN -m tcp -p tcp --dport 443 -j ACCEPT"

#
# git: git related config
#
# Set git.install to "yes" if you want ot install it
#
git: 
  install: yes

#
# haproxy stuff
#
haproxy:
  install: yes
  stats:
    users:
    - name: james
      password: $6$AasbD..
    - name: john
      password: $6$.1Xqi..

#
# htop: htop related config
#
# Set htop.install to "yes" if you want ot install it
#
htop:
  install: yes

#
# monitoring_server: IP address of monitoring server
# 
# used in firewall rules generation
#
monitoring_server: 192.168.0.99 

#
# more_hosts_entries: additional /etc/hosts entries
# to add in /etc/hosts
#
more_hosts_entries:
  - "192.168.0.100 james james.example.org"
  - "192.168.0.101 john john.exampe.org"

#
# mysql: mysql related vars, including database dumps
#
mysql:
  bind_address: "127.0.0.1" 
  key_buffer: "16M"
  keep_backup_count: 30
  root_password: "passssss"
  dump_directory: "/backup"
  users:
    - name: dbuser1
      password: user1pass
      privs: "user1_firstdb.*:ALL/user1_other.*:ALL"
    - name: dbuser2
      password: user2pass
      privs: "user1_firstdb.*:INSERT,UPDATE/user2_other.*:ALL"

#
# nameservers: list of nameservers
#
nameservers:
  - "10.12.13.14"
  - "10.11.12.13"

#
# network_interfaces_ipv4: network configuration
#
network_interfaces_ipv4:
  eth0:
    address: 192.168.0.2
    netmask: 255.255.255.192
    gateway: 192.168.0.1
    firewall: on

#
# ntp_server: NTP clock server address
#
# used in firewall rules generation
#
ntp_server: "10.11.11.11"

#
# packages: what packages needs to be added or removed
#
# Mostly useful in group_vars with per-distribution sensible values
#
packages:
  add:
    - ethtool
    - dstat
    - libc-bin 
  remove:
    - dovecot-common
    - logwatch

#
# php: PHP related configuration
#
php:
  safe_mode: "On"
  safe_mode_gid: "On"
  disable_functions: "apache_getenv,apache_get_modules,apache_get_version,apache_note,apache_setenv,chgrp,chown,diskfreespace,disk_free_space,
dl,getmypid,getrusage,highlight_file,ini_alter,ini_restore,listen,login,openlog,phpinfo,popen,posix_mkfifo,posix_setegid,posix_seteuid,posix_s
etgid,posix_setpgid,posix_setsid,posix_setuid,proc_get_status,proc_nice,proc_open, proc_terminate,show_source,socket_bind,socket_listen,symlin
k,system,virtual"

#
# postfix: SMTP server we want to use as relay
#
postfix:
  relayhost: smtp.example.org

#
# rabbitmq: rabbitmq related config
#
rabbitmq:
  port: 5672

#
# ruby: ruby related config
# 
ruby:
  install: no
  # Hypest ruby version
  current: "1.9.3-p374"
  # Under which account we want ruby to be installed ?
  deploy_user: caroline
  # Package list of build dependencies
  deps:
    - build-essential 
    - openssl 

#
# ssh: ssh server config
#
ssh:
  # users than can be logged in as via SSH
  allow_users:
    - root
    - caroline
  x11_forwarding: "no"
  local_admins:
    - james
    - john

#
# ucarp: ucarp double setup (crossed active-active)
#
# this requires a peer configured symetrically
#
ucarp:
  mode: double
  iface: eth0
  address: 192.168.0.193
  netmask: 255.255.255.192
  peer: rns1.example.org
  vhid: 193
  password: password193
#
# Peer would be :
# ucarp:
#   mode: double
#   iface: eth0
#   address: 192.168.0.194
#   netmask: 255.255.255.192
#   peer: rns0.example.org
#   vhid: 194
#   password: password194

#
# tmux: tmux related config
#
# Set tmux.install to "yes" if you want ot install it
#
tmux:
  install: yes
  prefix: C-b

I have two comments:

1. Wouldn't it be better not to include the tmux playbook on machines that
   don't get tmux, rather than turning it into a play conditional play?

   A case could be made for tmux.install==False causing tmux to be purged from
   the system, but you don't do that.

   And even if you did, I think there are numerous cases where parametrisation
   via variables should actually affect the composition of the playbook, not
   just the behaviour.

2. Have a look at
   https://groups.google.com/forum/?fromgroups=#!topic/ansible-project/HetiQdNKOj4
   and the referenced post to the salt-users list and chime in (on the ansible
   list of course).