How to deploy AMY for PyData on Ubuntu 16.04 LTS.md

Introduction

AMY is a web-based workshop administration application for Software Carpentry and related projects. Its target audience is workshop coordinators, most of whom are non-programmers, who need to keep track of what workshops are being arranged, when they're supposed to occur, who's teaching what, and so on.

AMY was extended to support PyData conferences and allow PyData conference cordinators to keep track of conference sites and their speakers.

Prerequisites and Goals

In order to complete this guide, you should have a fresh Ubuntu 16.04 LTS server instance.

We will be installing AMY within a virtual environment. Installing AMY into an environment specific to your project will allow your projects and their requirements to be handled separately.

We will configure AMY to use the PyData Djano application. The PyData app works as an addon for AMY and allows AMY to communicate with the API that exists over PyData conference websites. It also adds some nice defaults and populates the database for use by PyData cordinators.

Once we have our database and application up and running, we will install and configure the Gunicorn application server. This will serve as an interface to our application, translating client requests in HTTP to Python calls that our application can process. We will then set up Nginx in front of Gunicorn to take advantage of its high performance connection handling mechanisms and its easy-to-implement security features.

Let's get started.

Install the Packages from the Ubuntu Repositories

To begin the process, we'll download and install all of the items we need from the Ubuntu repositories.

sudo apt update
sudo apt-get install python-pip python3-dev nginx nodejs nodejs-legacy npm 

This will install pip, the Python development files needed to build Gunicorn later, and the libraries needed to interact with it, the Nginx web server, Node.js JavaScript platform and it package manager NPM.

Install the Frontend Dependency Manager for AMY

AMY uses Bower to manage its JavaScript and CSS dependencies. We will install Bower using NPM.

sudo npm install -g bower

Hint: If you are logged in as the user root, allow Bower to run with sudo permission by creating a ~/.bowerrc file. Add the following configuration to ~/.bowerrc:

{ "allow_root": true }

Create a Python Virtual Environment for AMY

We will be installing our Python requirements within a virtual environment for easier management. To do this, we first need access to the virtualenv command. We can install this with pip:

sudo pip install virtualenv

Create a directory where you wish to keep your project:

mkdir ~/amy_site

Within the project directory, create a Python virtual environment running Python 3 by typing:

cd ~/amy_site
virtualenv venv --python=/usr/bin/python3

This will create a directory called venv within your amy_site directory. Inside, it will install a local version of Python 3 and a local version of pip. We can use this to install and configure an isolated Python environment for our project.

Before we install our project's Python requirements, we need to activate the virtual environment. You can do that by typing:

source venv/bin/activate

Clone and setup AMY

We will clone the AMY repository on GitHub to our server.

git clone https://github.com/swcarpentry/amy.git
cd amy

With our application cloned, we install AMY's python dependencies and Gunicorn with the local instance of pip:

pip install -r requirements.txt
pip install gunicorn

Now, we can migrate the initial database schema to our SQLite database using the management script:

python manage.py migrate

Create an administrative user for the project by typing:

python manage.py createsuperuser

You will have to select a username, provide an email address, and choose and confirm a password.

We install the frontend dependencies using Bower by typing:

make bower_components

We can collect all of the static content into the directory location we configured by typing:

python manage.py collectstatic

Finally, you can test our your project by starting up the Django development server with this command:

python manage.py runserver 0.0.0.0:8000

In your web browser, visit your server's domain name or IP address followed by :8000:

http://server_domain_or_IP:8000/workshops

You should see AMY's login page.

Install the PyData application

We proceed to make PyData-specific changes to the AMY application.

• Add pydata to the list of INSTALLED_APPS in amy/settings.py. Ensure that the pydata app is listed before the workshops app.

  INSTALLED_APPS = (
      ...
      'pydata',
      'workshops',
      ...
  )

• Include pydata.urls in the urlpatterns of amy.urls before workshops.url.

  urlpatterns = [
      ...
      url(r'^workshops/', include('pydata.urls')),
      url(r'^workshops/', include('workshops.urls')),
      ...
  ]

• Add the username and password of a superuser instance that exists across all conference sites to amy/settings.py.

  PYDATA_USERNAME_SECRET = 'username'
  PYDATA_PASSWORD_SECRET = 'password'

  You can also fetch them from the environment variables.

  PYDATA_USERNAME_SECRET = os.environ.get('PYDATA_USERNAME_SECRET')
  PYDATA_PASSWORD_SECRET = os.environ.get('PYDATA_PASSWORD_SECRET')

• Install fixtures from the pydata/fixtures/ directory.

  python manage.py loaddata pydata/fixtures/*

• Recollect static files from the pydata application.

  python manage.py collectstatic

Invoke Django system check framework to check for any errors.

python manage.py check

Create a Gunicorn Systemd Service File

We should implement a more robust way of starting and stopping the AMY server. To accomplish this, we'll make an Systemd service file to serve AMY using the Gunicorn application server.

Create and open a Systemd service file for Gunicorn with sudo privileges in your text editor:

sudo nano /etc/systemd/system/amy.service

Start with the [Unit] section, which is used to specify metadata and dependencies. We'll put a description of our service here and tell the init system to only start this after the networking target has been reached:

[Unit]
Description=gunicorn daemon
After=network.target

Next, we'll open up the [Service] section. We'll specify the user and group that we want to process to run under. We will give our regular user account ownership of the process since it owns all of the relevant files. We'll give the Nginx user group ownership so that it can communicate easily with Gunicorn.

Hint: You can also give the root account ownership of the process.

We'll then map out the working directory and specify the command to use to start the service. In this case, we'll have to specify the full path to the Gunicorn executable, which is installed within our virtual environment. We will bind it to a Unix socket within the project directory since Nginx is installed on the same computer. This is safer and faster than using a network port. We can also specify any optional Gunicorn tweaks here. For example, we specified 3 worker processes in this case:

[Unit]
Description=gunicorn daemon
After=network.target

[Service]
User=user
Group=www-data
WorkingDirectory=/home/user/amy_site/amy
ExecStart=/home/user/amy_site/venv/bin/gunicorn --workers 3 --bind unix:amy.sock amy.wsgi:application

Finally, we'll add an [Install] section. This will tell Systemd what to link this service to if we enable it to start at boot. We want this service to start when the regular multi-user system is up and running:

[Unit]
Description=gunicorn daemon
After=network.target

[Service]
User=user
Group=www-data
WorkingDirectory=/home/user/amy_site/amy
ExecStart=/home/user/amy_site/venv/bin/gunicorn --workers 3 --bind unix:amy.sock amy.wsgi:application

[Install]
WantedBy=multi-user.target

With that, our Systemd service file is complete. Save and close it now.

We can now start the Gunicorn service we created and enable it so that it starts at boot:

sudo systemctl start amy
sudo systemctl enable amy

Configure Nginx to Proxy Pass to Gunicorn

Now that Gunicorn is set up, we need to configure Nginx to pass traffic to the process.

We must ensure that Nginx has the permission to access amy.sock as the user user and group www-data. Open the Nginx config file for Gunicorn with sudo privileges in your text editor:

sudo nano /etc/nginx/nginx.conf

Replace the user directive in nginx.conf with:

user root www-data;

With the right permissions, start by creating and opening a new server block in Nginx's sites-available directory:

sudo nano /etc/nginx/sites-available/amy

Inside, open up a new server block. We will start by specifying that this block should listen on the normal port 80 and that it should respond to our server's domain name or IP address:

server {
    listen 80;
    server_name server_domain_or_IP;
}

Next, we will tell Nginx to ignore any problems with finding a favicon. We will also tell it where to find the static assets that we collected in our /amy_site/amy/static directory. All of these files have a standard URI prefix of /static, so we can create a location block to match those requests:

server {
    listen 80;
    server_name server_domain_or_IP;

    location = /favicon.ico { access_log off; log_not_found off; }
    location /static/ {
        root /home/user/amy_site/amy;
    }
}

Finally, we'll create a location / {} block to match all other requests. Inside of this location, we'll include the standard proxy_params file included with the Nginx installation and then we will pass the traffic to the socket that our Gunicorn process created:

server {
    listen 80;
    server_name server_domain_or_IP;

    location = /favicon.ico { access_log off; log_not_found off; }
    location /static/ {
        root /home/user/amy_site/amy;
    }

    location / {
        include proxy_params;
        proxy_pass http://unix:/home/user/amy_site/amy/amy.sock;
    }
}

Save and close the file when you are finished. Now, we can enable the file by linking it to the sites-enabled directory:

sudo ln -s /etc/nginx/sites-available/amy /etc/nginx/sites-enabled

Test your Nginx configuration for syntax errors by typing:

sudo nginx -t

If no errors are reported, go ahead and restart Nginx by typing:

sudo service nginx restart

You should now be able to go to your server's domain or IP address to view AMY.