12-08-14-fabric-workshop.rst

Fabric Workshop

Automate everything!

http://virtualenv.readthedocs.org/en/latest/virtualenv.html#installation https://pypi.python.org/pypi/virtualenv/1.11.6 http://www.fabfile.org/

For Fabric to work we need couple things: - Python 2 - Virtualenv - PIP

Create project directory and move to it::

$ mkdir hello_world $ cd hello_world

Of course we are using git:

$ git init

To setup virtualenv with pip:

$ curl -O https://pypi.python.org/packages/source/v/virtualenv/virtualenv-1.11.6.tar.gz
$ tar -xzf virtualenv*tar.gz
$ ./virtualenv-1.11.6/virtualenv.py -p python2 venv
$ rm -rI virtualenv*
$ source venv/bin/activate 

Finally we start our work on project. As we automate everything, create dependency file called `requirements.txt`:

$ cat > requirements.txt << EOF
fabric
EOF

Then we install our only requirement in python virtual environment::

$ pip install -r requirements.txt

Now we have Fabric setup in our project. Fabric by default uses modules called fabfile.py to look for tasks. So we create one:

$ cat > fabfile.py << EOF
from fabric.api import run 

def host_type():
    run('uname -s')
EOF

To test out if our tasks works, we run:

$ fab -H localhost host_type

It should ask credentials for localhost and print out type of the system. Ignore python bytecode:

$ cat > .gitignore << EOF
*.pyc
EOF

Now we are ready to add changes to repository:

$ git add --all
$ git commit -m "Add project deployment script draft"

Execute arbitary Python function via the command line. Execute shell commands ower SSH. Looks for file called fabfile.py.

You execute tasks using syntax <task name>:<arg>,<kwarg>=<value>.

There are multiple strategies to move files between hosts. - You can tar them and move through SSH connection. Can be slow and time consuming. - You can rsync them to remote host.

Note

Triple check trailing slash meaning in source and destination argument.

To separate sensitive information, it is possible to write configuration in ~/.fabricrc or passed by specifying -c flag.

Note

By default handles failures. Be aware of commands with non standard exit codes.

Suppressing warning messages

from fabric.api import settings, local, abort from fabric.contrib.console import confirm

def test():

with settings(warn_only=True):

result = local('./manage.py test my_app', capture=True)

if result.failed and not confirm('Test failed. Continue anyway?):

abort('Aborting at user request')

Change files on remote host Run commands in parallel

Passing in arguments Use colon command:<argument> to pass in arguments from comandline.

ENV

env.hosts = ['host1', 'host2'] env.user = 'ubuntu'

You can configure your execution using ENV global variable.

Fabfile used in combination with Salt-Stack https://gist.github.com/abele/4ffbffe1871d47afa4c7

bootstrap.py

#!/usr/bin/env python2
# -*- coding: utf-8 -*-
from __future__ import print_function

import subprocess
import sys


VENV_VERSION = '1.9.1'
PYPI_VENV_BASE = 'https://pypi.python.org/packages/source/v/virtualenv'
PYTHON = 'python2'
INITIAL_ENV = 'venv'


def shellcmd(cmd, echo=True):
    """Run 'cmd' in the shell and return its standard out."""
    if echo:
        print('[cmd] {0}'.format(cmd))

    proc = subprocess.Popen(
        cmd,
        shell=True,
        stdout=subprocess.PIPE,
        stderr=sys.stderr,
    )

    if echo:
        while proc.poll() is None:
            output = proc.stdout.readline()
            print(output, end='')

    return proc


def main():
    """Download and install virtualenv."""
    dirname = 'virtualenv-' + VENV_VERSION
    tgz_file = dirname + '.tar.gz'

    # Fetch virtualenv from PyPI
    venv_url = PYPI_VENV_BASE + '/' + tgz_file
    shellcmd('curl -O {0}'.format(venv_url))

    # Untar
    shellcmd('tar xzf {0}'.format(tgz_file))

    # Create the initial env
    shellcmd('{0} {1}/virtualenv.py {2} --distribute'.format(
        PYTHON, dirname, INITIAL_ENV))

    # Install the virtualenv package itself into the initial env
    shellcmd('{0}/bin/pip install {1}'.format(INITIAL_ENV, tgz_file))

    # Cleanup
    shellcmd('rm -rf {0} {1}'.format(dirname, tgz_file))

    # Install development dependencies
    shellcmd('{0}/bin/pip install fabric'.format(
        INITIAL_ENV))


if __name__ == '__main__':
    main()