This GIST is for getting zero downtime deployments with your Python applications

1. Zero Downtime Deployment.md

Instructions

This is a gist containing several files needed to get Masonite automatically deploying to your servers via GitHub pushes (or releases)

This GIST uses unix sockets and uWSGI in order to get zero downtime deployment.

Requirements

• NGINX installed (may or may not be fully configured)
• Python 3 installed and everything needed to run a Masonite application (see Masonite documentation for requirements)
• Linux packages installed required to install Python packages (see Masonite documentation again)
• Git installed
• SSH credentials to connect to the server you are deploying to
• Need to have the uwsgi requirement in your requirements.txt file

Instructions

First things first. The first thing you need to do is have NGINX installed. This Gist assumes you have that installed already.

At high level there are 3 things we have to do:

1. We need to make sure our NGINX config is setup.
2. We need to make sure we have a config file for our actual application
3. We need to get the workflow in our application so GitHub will know to run it on certain actions (pushing or releasing)

Step 1 - Setting up NGINX config file

First we need to locate where our NGINX config file lives. Simply run this:

$ nginx -t
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful

When you get the output, you are looking for the configuration file. Both directories should be the same.

We can view and edit this file by doing:

$ nano /etc/nginx/nginx.conf

If you already have an NGINX config file then you may not want to override the existing file. The real important part is are these 2 lines in this block:

    ##
    # Virtual Host Configs
    ##

    include /etc/nginx/conf.d/*.conf;
    include /etc/nginx/sites-enabled/*;

This will tell NGINX to additionally load config files from these directories. We will use the bottom directory to put the app-config.conf file in this gist. If you already have these 2 lines (or lines similiar to these) then you are good. You can exit out by hitting CTRL+X. If you get some kind of prompt that says "Save modified buffer? (Y/n)" just hit "Y" and hit enter. This just tells nano to save the file. If you don't want to save it then enter n and hit enter.

Step 2 - Setting up the application config

Once done we need to then go to one of those directories. It really doesn't matter which one but let's pick the bottom one

$ cd /etc/nginx/sites-enabled

We can then make our config file. You can name this whatever you want. Maybe its the name of your application:

$ nano exampleapp.conf

This should open an empty editor. You can then paste in app-config.conf you see in this Gist.

Modifying the file

You may need to make some modifications to it.

Going from top to bottom, we should be listening on port 80.

The next line says server_name. If you put your IP address then it will work when you go to http://17.62.11.87. What you most likely want is a domain name so you can change it to a domain name instead.

The next is location /. We can leave that as is

The next line is include uwsgi_params;. leave that as well.

The next line says unix:///srv/sockets/example.com.sock;. You will just need to swap out the domain name for your own domain name. This domain will most likely be whatever you put in the server_name spot.

All said and done you might have something like looks like this:

## /etc/nginx/sites-enabled/app-config.conf
## Name this file whatever you want. Just make sure it goes in the /etc/nginx/sites-enabled/ directory

server {
     listen 80;
-    server_name 17.62.11.87;
+    server_name subdomain.example.com;


    location / {
         include uwsgi_params;
-        uwsgi_pass unix:///srv/sockets/example.com.sock;
+        uwsgi_pass unix:///srv/sockets/subdomain.example.com.sock;
         proxy_request_buffering off;
         proxy_buffering off;
         proxy_redirect off;
    }
}

Once done you can back out of nano again.

Reload NGINX

At this point you may need to reload nginx. To do that just run:

nginx -s reload

Step 3 - Adding the workflow to your project

The GitHub workflow action

The last stop now that the server is up and running is you will need to get the workflow into your project so GitHub will pick up on it and run it on various actions like pushing or releasing.

You can do this by starting in your project (or GitHub repo) and create a .github/workflows/deploy.yml file. Then copy and paste the workflow to that file (the masonite-deploy.yml file in this Gist). Commit the change and push it up to the origin repository on GitHub.

The secrets

Next we need to configure some github secrets. You can go to your repo and look in the top right of the page. You should see "Setting". Then go to "Secrets" on the left vertical navigation bar near the bottom.

You'll need to add these secrets:

HOST: the server IP address you want to deploy to

USERNAME: the username of the ssh user

PASSWORD: the password of the ssh user

PORT: The port for ssh (usually 22)

ENV: A list of environment variables for your project. This should just be a list of rows with key value pairs like this:

KEY=123
APP_DEBUG=True
ENV3=value

DOMAIN: The domain name for your project. This will be used to match the unix socket to the application.

Done

Once done the GitHub action will run when you push (or release depending on what you set at the top, see the comment there). Feel free to modify the action if you want. There are comments next to each line to explain what they are doing.

app-config.conf

server {
    listen 80;
    server_name 17.62.11.87; 
    # This should be changed to either your server IP or a domain name like:
    # server_name example.com;

    location / {
        include uwsgi_params;
        # The deployment script will fetch the domain using your domain setup.
        # So replace example.com with the domain you setup in GitHub secrets
        uwsgi_pass unix:///srv/sockets/example.com.sock;
        proxy_request_buffering off;
        proxy_buffering off;
        proxy_redirect off;
    }
}

nginx.conf

## /etc/nginx/nginx.conf

user root;
worker_processes auto;
pid /run/nginx.pid;
include /etc/nginx/modules-enabled/*.conf;

events {
    worker_connections 768;
    # multi_accept on;
}

http {

    ##
    # Basic Settings
    ##

    sendfile on;
    tcp_nopush on;
    tcp_nodelay on;
    keepalive_timeout 65;
    types_hash_max_size 2048;
    server_tokens on;

    # server_names_hash_bucket_size 64;
    # server_name_in_redirect off;

    include /etc/nginx/mime.types;
    default_type application/octet-stream;

    ##
    # SSL Settings
    ##

    ssl_protocols TLSv1 TLSv1.1 TLSv1.2; # Dropping SSLv3, ref: POODLE
    ssl_prefer_server_ciphers on;

    ##
    # Logging Settings
    ##

    access_log /var/log/nginx/access.log;
    error_log /var/log/nginx/error.log;

    ##
    # Gzip Settings
    ##

    gzip on;

    # gzip_vary on;
    # gzip_proxied any;
    # gzip_comp_level 6;
    # gzip_buffers 16 8k;
    # gzip_http_version 1.1;
    # gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss;


    ##
    # Virtual Host Configs
    ##

    include /etc/nginx/conf.d/*.conf;
    include /etc/nginx/sites-enabled/*;
}


#mail {
# # See sample authentication script at:
# # http://wiki.nginx.org/ImapAuthenticateWithApachePhpScript
#
# # auth_http localhost/auth.php;
# # pop3_capabilities "TOP" "USER";
# # imap_capabilities "IMAP4rev1" "UIDPLUS";
#
# server {
# listen localhost:110;
# protocol pop3;
# proxy on;
# }
#
# server {
# listen localhost:143;
# protocol imap;
# proxy on;
# }
#}

zero-downtime-deployment.yml

name: Deployment

on: [push]

jobs:
  build:
    name: Deploy
    runs-on: ubuntu-latest
    steps:
    - name: executing remote ssh commands using password
      uses: appleboy/ssh-action@master
      with:
        host: ${{ secrets.HOST }}
        username: ${{ secrets.USERNAME }}
        password: ${{ secrets.PASSWORD }}
        port: ${{ secrets.PORT }}
        script: |
          mkdir -p /srv/sites/basic  # Make the directory if it doesnt exist
          mkdir -p /srv/sockets # Make directory where we will store our sockets
          cd /srv/sites/basic # Go to that directory
          sudo chmod -R 0777 /srv/sockets # Make sure nginx has access to the socket directory
          git clone https://github.com/${{ github.repository }}.git ${{ github.sha }} # Clone the current repo
          cd ${{ github.sha }} # Go to the directory we just made (current deployment)
          python3 -m venv /venvs/basic # Make our virtual environment if it doesn't exist
          source /venvs/basic/bin/activate # Activate our virtual Environment
          pip install -r requirements.txt # Install our dependencies
          echo "${{ secrets.ENV }}" >> .env # Create an environment file (if you use .env files)
          set -m; nohup uwsgi --socket /srv/sockets/${{ secrets.domain }}.sock --wsgi-file wsgi.py --chmod-socket=777 &> /dev/null & # run gunicorn as a daemon
          cd ../ # Go Back 1 directory to the directory list of deployments
          sleep 5 # Just wait 5 seconds to make sure the socket has enough time to detect the new deployment changes
          shopt -s extglob # enable the command below this one
          rm -rfv !("${{ github.sha }}") # Delete all directories except for the one we just created (old deployments)
          set -m; nohup uwsgi --stop /srv/sockets/$(cat /srv/sites/basic/.last-deployment).pid &> /dev/null & # Kill last uwsgi deployment completely
          echo "${{ github.sha }}" > /srv/sites/basic/.last-deployment # Put this commit as the last deployment