Skip to content

Instantly share code, notes, and snippets.

@evansmwendwa

evansmwendwa/Python-Deployment.md

Created October 26, 2017 08:13
Show Gist options
  • Save evansmwendwa/c4ac55889029e3fd01e06a49dcddf9ed to your computer and use it in GitHub Desktop.
Save evansmwendwa/c4ac55889029e3fd01e06a49dcddf9ed to your computer and use it in GitHub Desktop.
Download ZIP
Raw
Python-Deployment.md

DEPLOYING DJANGO APPLICATION TO UBUNTU SERVER

KMD CASE - DEPLOYED IN FORGE

Warning! Intermediate experience in setting up Nginx and Python in ubuntu is required to follow this setup

This setup uses Python version 2.7.x and Ubuntu 16 setup with a non root sudo account.

This Tutorial follows the article provided by digital ocean https://www.digitalocean.com/community/tutorials/how-to-set-up-django-with-postgres-nginx-and-gunicorn-on-ubuntu-16-04 and the setup is done in a digital ocean droplet provisioned by Laravel forge. Forge's sites feature and SSL feature is used instead of manual nginx server blocks.

Install the Packages from the Ubuntu Repositories

Install python

python 2

sudo apt-get update
sudo apt-get install python-pip python-dev libpq-dev postgresql postgresql-contrib nginx

python 3

sudo apt-get update
sudo apt-get install python3-pip python3-dev libpq-dev postgresql postgresql-contrib nginx

Create the PostgreSQL Database and User

Log into an interactive Postgres session by typing:

sudo -u postgres psql

create db

all postgresql commands must end with a ;

CREATE DATABASE projectdbname;

create user

make sure to replace password and user name to your prefered

CREATE USER projectdbuser WITH PASSWORD 'secret';

setup db defaults

https://docs.djangoproject.com/en/1.9/ref/databases/#optimizing-postgresql-s-configuration

ALTER ROLE projectdbuser SET client_encoding TO 'utf8';
ALTER ROLE projectdbuser SET default_transaction_isolation TO 'read committed';
ALTER ROLE projectdbuser SET timezone TO 'UTC';

grant user access to the db

GRANT ALL PRIVILEGES ON DATABASE projectdbname TO projectdbuser;

exit postgresql

\q

Create a Python Virtual Environment for your Project

If you are using Python 3, type:

sudo pip install virtualenv

With virtualenv installed, we can start forming our project. Create and move into a directory where we can keep our project files:

Git Clone / Create your project files

git clone git@github.com:iHub/kmd-api.git

# then cd to project directory

# create virtual env (the env folder is `.virtualenv` )
virtualenv -p /usr/bin/python2.7 .virtualenv

This will create a directory called myprojectenv within your myproject directory. Inside, it will install a local version of Python and a local version of pip. We can use this to install and configure an isolated Python environment for our project.

Before we install our project's Python requirements, we need to activate the virtual environment. You can do that by typing:

source .virtualenv/bin/activate

Your prompt should change to indicate that you are now operating within a Python virtual environment. It will look something like this: (myprojectenv)user@host:~/myproject$.

With your virtual environment active, install Django, Gunicorn, and the psycopg2 PostgreSQL adaptor with the local instance of pip:

Regardless of which version of Python you are using, when the virtual environment is activated, you should use the pip command (not pip3).

pip install django gunicorn psycopg2

Adjust the Project Settings

The first thing we should do with our newly created project files is adjust the settings. Open the settings file in your text editor:

# inside the project directory
nano config/settings/production.py

Start by locating the ALLOWED_HOSTS directive. This defines a whitelist of addresses or domain names may be used to connect to the Django instance. Any incoming requests with a Host header that is not in this list will raise an exception. Django requires that you set this to prevent a certain class of security vulnerability.

. . .
# The simplest case: just add the domain name(s) and IP addresses of your Django server
# ALLOWED_HOSTS = [ 'example.com', '203.0.113.5']
# To respond to 'example.com' and any subdomains, start the domain with a dot
# ALLOWED_HOSTS = ['.example.com', '203.0.113.5']
ALLOWED_HOSTS = ['your_server_domain_or_IP', 'second_domain_or_IP', . . .]

Install Django Dependencies

# this project is using python 2.7
pip install -r requirements/production.txt

NB: You might need to install these dependencies if you are having problems finishing the above install command

https://stackoverflow.com/questions/28253681/you-need-to-install-postgresql-server-dev-x-y-for-building-a-server-side-extensi

Use these following commands, this will solve the error:

sudo apt-get install postgresql

then fire:

sudo apt-get install python-psycopg2

and last:

sudo apt-get install libpq-dev

Complete Initial Project Setup

Now, we can migrate the initial database schema to our PostgreSQL database using the management script:

# cd to inside the project directory
# Edit your .env file inside (config/settings/.env) see /.env.dist for blueprint

./manage.py makemigrations
./manage.py migrate

Create an administrative user for the project by typing:

./manage.py createsuperuser

You will have to select a username, provide an email address, and choose and confirm a password.

We can collect all of the static content into the directory location we configured by typing:

./manage.py collectstatic

Testing with the dev server

Create an exception for port 8000 by typing:

sudo ufw allow 8000

Finally, you can test our your project by starting up the Django development server with this command:

./manage.py runserver 0.0.0.0:8000

In your web browser, visit your server's domain name or IP address followed by :8000:

http://server_domain_or_IP:8000

If you append /admin to the end of the URL in the address bar, you will be prompted for the administrative username and password you created with the createsuperuser command:

After authenticating, you can access the default Django admin interface:

When you are finished exploring, hit CTRL-C in the terminal window to shut down the development server.

Testing Gunicorn's Ability to Serve the Project

The last thing we want to do before leaving our virtual environment is test Gunicorn to make sure that it can serve the application. We can do this easily by typing:

# our wsgi file is inside the config directory
gunicorn --bind 0.0.0.0:8000 config.wsgi:application

This will start Gunicorn on the same interface that the Django development server was running on. You can go back and test the app again.

Note: The admin interface will not have any of the styling applied since Gunicorn does not know about the static CSS content responsible for this.

We passed Gunicorn a module by specifying the relative directory path to Django's wsgi.py file, which is the entry point to our application, using Python's module syntax. Inside of this file, a function called application is defined, which is used to communicate with the application. To learn more about the WSGI specification, click here.

When you are finished testing, hit CTRL-C in the terminal window to stop Gunicorn.

We're now finished configuring our Django application. We can back out of our virtual environment by typing:

deactivate

Create a Gunicorn systemd Service File

We have tested that Gunicorn can interact with our Django application, but we should implement a more robust way of starting and stopping the application server. To accomplish this, we'll make a systemd service file.

Create and open a systemd service file for Gunicorn with sudo privileges in your text editor:

sudo nano /etc/systemd/system/gunicorn.service

Paste the contents in editor ensure to update user and paths

[Unit]
Description=gunicorn daemon
After=network.target

[Service]
User=forge
Group=www-data
WorkingDirectory=/home/forge/cdmadmin.meteo.go.ke
ExecStart=/home/forge/cdmadmin.meteo.go.ke/.virtualenv/bin/gunicorn --workers 3 --bind unix:/home/forge/cdmadmin.meteo.go.ke/kmd.sock config.wsgi:application

[Install]
WantedBy=multi-user.target

With that, our systemd service file is complete. Save and close it now.

We can now start the Gunicorn service we created and enable it so that it starts at boot:

sudo systemctl start gunicorn
sudo systemctl enable gunicorn

Configure Nginx to Proxy Pass to Gunicorn

Now that Gunicorn is set up, we need to configure Nginx to pass traffic to the process.

Sample forge nginx.config file SSL is set using forge let's encrypt feature proxy pass is added to the location block

# FORGE CONFIG (DOT NOT REMOVE!)
include forge-conf/cdmadmin.meteo.go.ke/before/*;

server {
    listen 443 ssl http2;
    listen [::]:443 ssl http2;
    server_name cdmadmin.meteo.go.ke;
    # root /home/forge/cdmadmin.meteo.go.ke/;

    # FORGE SSL (DO NOT REMOVE!)
    ssl_certificate /etc/nginx/ssl/cdmadmin.meteo.go.ke/201644/server.crt;
    ssl_certificate_key /etc/nginx/ssl/cdmadmin.meteo.go.ke/201644/server.key;

    ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
    ssl_ciphers 'ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA';
    ssl_prefer_server_ciphers on;
    ssl_dhparam /etc/nginx/dhparams.pem;

    add_header X-Frame-Options "SAMEORIGIN";
    add_header X-XSS-Protection "1; mode=block";
    add_header X-Content-Type-Options "nosniff";

    # index index.html index.htm index.php;

    charset utf-8;

    # FORGE CONFIG (DOT NOT REMOVE!)
    include forge-conf/cdmadmin.meteo.go.ke/server/*;

    location / {
        include proxy_params;
        proxy_pass http://unix:/home/forge/cdmadmin.meteo.go.ke/kmd.sock;
    }

    location = /favicon.ico { access_log off; log_not_found off; }
    location = /robots.txt  { access_log off; log_not_found off; }

    access_log off;
    error_log  /var/log/nginx/cdmadmin.meteo.go.ke-error.log error;

    # error_page 404 /index.php;

    location ~ /\.ht {
        deny all;
    }
}

# FORGE CONFIG (DOT NOT REMOVE!)
include forge-conf/cdmadmin.meteo.go.ke/after/*;

Finally, we need to open up our firewall to normal traffic on port 80. Since we no longer need access to the development server, we can remove the rule to open port 8000 as well:

sudo ufw delete allow 8000
sudo ufw allow 'Nginx Full'

You should now be able to go to your server's domain or IP address to view your application.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment