Skip to content

Instantly share code, notes, and snippets.

@cspanring

cspanring/ubuntu-django-setup.md

Created September 12, 2012 12:26
Show Gist options
  • Star 2 You must be signed in to star a gist
  • Fork 1 You must be signed in to fork a gist
  • Save cspanring/3706287 to your computer and use it in GitHub Desktop.
Save cspanring/3706287 to your computer and use it in GitHub Desktop.
Download ZIP
A few quick steps to setup Ubuntu 10.04 as Django application server
Raw
ubuntu-django-setup.md

Basic Ubuntu 10.04.3 LTS server setup to deploy Django appplications

This document covers basic steps to setup

...on a default Ubuntu 10.04 server installation.

Firewall

See the UFW Community docs and Ubuntu 10.04 server UFW docs for more details.

Allow SSH port (default: 22) and activate:

$ sudo ufw allow ssh
$ sudo ufw enable

Allow http service (port 80):

$ sudo ufw allow http

Check which rules are active:

$ sudo ufw status

Users

Add a group wheel that will contain users with sudo privileges:

$ sudo groupadd wheel
$ sudo visudo
	## Allows people in group wheel to run all commands
	%wheel ALL=(ALL) ALL

Add new user, with home directory, bash and part of group wheel:

$ sudo useradd myusername -m -g wheel -s /bin/bash
$ sudo passwd myusername

The user django will serve Django applications, without having sudo privileges.

$ sudo useradd django -m -g www-data -s /bin/bash
$ sudo passwd myusername

Configure remote login permissions:

$ sudo nano /etc/ssh/sshd_config
	# Disable remote login for django user
	Match User django
	    PasswordAuthentication no
$ sudo /etc/init.d/ssh reload

The WWW directory

As django user create a www directory with the structure below:

$ su - django
$ sudo mkdir -p /home/django/www/domain.name/{public,log,backup,private}

Software installations

Basic system software

Some basic tools that are of general use for developing and deploying Python based applications.

Build tools and package manager helper:

$ sudo apt-get install build-essential python-software-properties

Python tools

$ sudo apt-get install python-dev python-setuptools
$ sudo easy_install pip
$ sudo pip install virtualenvwrapper

$ nano /home/django/.profile
export WORKON_HOME=~/virtualenvs
source /usr/local/bin/virtualenvwrapper.sh

Git distributed version control

$ sudo apt-get install git-core

See GitHub's Linux docs for detailed information about how to configure git and a GitHub account.

Nginx web server

Add the stable repository and install Nginx 1.0 (default Ubuntu 10.04 repository has older version, might not be required for newer Ubuntu version):

$ sudo add-apt-repository ppa:nginx/stable
$ sudo apt-get update
$ sudo apt-get install nginx

Nginx configuration

Edit nginx.conf to with a basic configuration:

$ sudo nano /etc/nginx/nginx.conf

	user www-data;
	worker_processes 2;
	# check maximum file descriptors available at /proc/sys/fs/file-max
	worker_rlimit_nofile 197587;
	pid /var/run/nginx.pid;

	events {
		# 2GB available memory
		worker_connections 1024;
		# multi_accept on;
		use epoll;
	}

	http {
		index index.html;
		client_max_body_size 20m;
		##
		# Basic Settings
		##

		sendfile on;
		tcp_nopush on;
		tcp_nodelay on;
		keepalive_timeout 65;
		types_hash_max_size 2048;
		# server_tokens off;

		# server_names_hash_bucket_size 64;
		# server_name_in_redirect off;

		include /etc/nginx/mime.types;
		default_type application/octet-stream;

		##
		# Logging Settings
		##
		
		# uncomment if access logs are needed
		# access_log /var/log/nginx/access.log;
		error_log /var/log/nginx/error.log;

		##
		# Gzip Settings
		##

		gzip on;
		gzip_disable "msie6";

		gzip_vary on;
		gzip_proxied any;
		gzip_comp_level 6;
		gzip_buffers 16 8k;
		gzip_http_version 1.1;
		gzip_types text/plain text/css application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript;

		##
		# Virtual Host Configs
		##

		include /etc/nginx/conf.d/*.conf;
		include /etc/nginx/sites-enabled/*;
	}

Some interesting posts about Nginx configuration optimizations:

Test and reload Nginx's updated configuration:

$ sudo nginx -t
$ sudo nginx -s reload

PostgreSQL database server

Add the PostgreSQL 9.0 repository (default version Ubuntu 10.04 is 8.4, might not be required on newer Ubuntu system) and install the PostgreSQL server and development libraries:

$ sudo add-apt-repository ppa:pitti/postgresql
$ sudo apt-get update
$ sudo apt-get install postgresql-9.0 postgresql-server-dev-9.0 postgresql-contrib-9.0 libpq-dev

Create the PostGIS template (note: PostGIS must be installed, see below for details):

$ sudo su - postgres
$ POSTGIS_SQL_PATH=`pg_config --sharedir`/contrib
# Creating the template spatial database.
$ createdb -E UTF8 --lc-collate en_US.UTF-8 --lc-ctype en_US.UTF-8 -T template0 template_postgis
$ createlang -d template_postgis plpgsql # Adding PLPGSQL language support.
# Allows non-superusers the ability to create from this template
$ psql -d postgres -c "UPDATE pg_database SET datistemplate='true' WHERE datname='template_postgis';"
# Loading the PostGIS SQL routines
$ psql -d template_postgis -f $POSTGIS_SQL_PATH/postgis-1.5/postgis.sql
$ psql -d template_postgis -f $POSTGIS_SQL_PATH/postgis-1.5/spatial_ref_sys.sql
$ psql -d template_postgis -f $POSTGIS_SQL_PATH/postgis_comments.sql
# Enabling users to alter spatial tables.
$ psql -d template_postgis -c "GRANT ALL ON geometry_columns TO PUBLIC;"
$ psql -d template_postgis -c "GRANT ALL ON geography_columns TO PUBLIC;"
$ psql -d template_postgis -c "GRANT ALL ON spatial_ref_sys TO PUBLIC;"
# Garbage-collect and freeze.
$ psql -d template_postgis -c "VACUUM FULL;"
$ psql -d template_postgis -c "VACUUM FREEZE;"

Create a new PostgreSQL user and set password:

$ createuser django
	Shall the new role be a superuser? (y/n) n
	Shall the new role be allowed to create databases? (y/n) y
	Shall the new role be allowed to create more new roles? (y/n) n
$ psql
	# ALTER ROLE django WITH password 'mypassword';
	ALTER ROLE
	# \q

Create a new database from a template and assign an owner

$ createdb test -T template_postgis -O django

Delete database

$ dropdb test

PostgreSQL configuration

Each sudoer is also a PostgreSQL superuser. If that's not sufficient, you can use the postgres user to perform database related tasks:

$ sudo su - postgres

Config directory: /etc/postgresql/9.0/main
Data directory: /var/lib/postgresql/9.0/main

Allow remote access, if required by editing the config files pg_hba.conf, postgresql.conf, adjusting the firewall rules and finally restarting the server to propagate all changes:

$ sudo nano /etc/postgresql/9.0/main/pg_hba.conf
	...
	# All of an intranet
	host    all             all             10.10.10.0/24           md5
	...
$ sudo nano /etc/postgresql/9.0/main/postgresql.conf
	...
	listen_addresses = '*'
	...
# open PostgreSQL's port in firewall
$ sudo ufw allow from 10.10.10.0/24 to any port 5432
$ sudo /etc/init.d/postgresql restart

Performance tweaks for postgresql.conf:

shared_buffers: set to 10-20% of RAM
effective_cache_size: set to 75% of RAM

Memcached caching system

Memcached will help to reduce load on Django applications (and therefore database queries).

Install and start the memcached server:

$ sudo apt-get install memcached libmemcached-dev
$ sudo memcached -d -u www-data -p 11211 -m 64

Memcached startup configuration is located at /etc/memcached.conf.

PostGIS

Add repository with last 1.5 version (might not be required on newer Ubuntu system):

$ sudo add-apt-repository ppa:ubuntugis/ubuntugis-unstable
$ sudo add-apt-repository ppa:pi-deb/gis
$ sudo apt-get update
$ sudo apt-get install postgresql-9.0-postgis \
proj libproj-dev libgeos-3.2.2 libgeos-c1 libgeos-dev \
libgdal1-1.8.0 libgdal1-dev libxml2 libxml2-dev \
checkinstall

Django application deployment

Django applications are deployed by the Gunicorn WSGI server and cached with memcached. Nginx is proxying the WSGI server and also serving Django's static files.

Additional Python modules (Gunicorn, Memcached)

Additional dependencies that need to be installed on the system or better, the individual virtual environment:

$ pip install gunicorn python-memcached

Test Gunicorn installation with Django:

$ cd /home/django/domains/MYPROJECT.ORG/private/MYPROJECT
$ gunicorn_django -b 0.0.0.0:8000

Create a gunicorn script /home/django/domains/MYPROJECT.ORG/private/gunicorn.sh and make it executable:

#!/bin/bash
set -e
LOGFILE=/home/django/domains/MYPROJECT.ORG/log/gunicorn.log
NUM_WORKERS=1
# user/group to run as
USER=django
GROUP=www-data
cd /home/django/domains/MYPROJECT.ORG/private/MYPROJECT
source /home/django/virtualenvs/MYPROJECT/bin/activate
exec /home/django/virtualenvs/MYPROJECT/bin/gunicorn MYPROJECT.wsgi:application \
	--workers=$NUM_WORKERS \
	--bind=127.0.0.1:8001 \
	--user=$USER --group=$GROUP --log-level=error \
	--log-file=$LOGFILE 2>>$LOGFILE

$ chmod ug+x gunicorn.sh

Additional Django local_settings.py configuration parameters:

# static files served by Nginx
STATIC_URL = '/static/'
STATIC_ROOT = '/home/django/domains/MYPROJECT.ORG/public/static'

# uncomment and/or change if GDAL can't be found
# GDAL_LIBRARY_PATH = '/usr/lib/libgdal1.8.0.so'

# memcached integration (https://docs.djangoproject.com/en/1.3/topics/cache/)
CACHES = {
	'default': {
		'BACKEND': 'django.core.cache.backends.memcached.MemcachedCache',
		'LOCATION': '127.0.0.1:11211',
	}
}
MIDDLEWARE_CLASSES = (
	'django.middleware.cache.UpdateCacheMiddleware',
	'django.middleware.common.CommonMiddleware',
	'django.middleware.cache.FetchFromCacheMiddleware',
)
CACHE_MIDDLEWARE_ALIAS = 'default'
CACHE_MIDDLEWARE_SECONDS = 3600
CACHE_MIDDLEWARE_KEY_PREFIX = 'MYPROJECT'

Ubuntu Upstart integration

Add Ubuntu Upstart service script in /etc/init/MYPROJECT.conf to start app on system boot:

description "MYPROJECT Django instance"
start on runlevel [2345]
stop on runlevel [06]
respawn
respawn limit 10 5
exec /home/django/domains/MYPROJECT.ORG/private/gunicorn.sh

# start and stop app
$ sudo service MYPROJECT [start/stop]

Nginx virtual host

Nginx serves static files and proxies Gunicorn WSGI applications.

A virtual host configuration with a proxy to a Django project looks like:

$ nano /etc/nginx/sites-available/MYPROJECT.ORG
server {
	listen   80;
	server_name MYPROJECT.ORG;

	root /home/django/domains/MYPROJECT.ORG/public;

	access_log /home/django/domains/MYPROJECT.ORG/log/access.log;
	error_log /home/django/domains/MYPROJECT.ORG/log/error.log;

	# serve staticfiles
	location /static/ {
	root   /home/django/domains/MYPROJECT.ORG/public/;
	expires 1d;
	}

	# proxy
	location / {
		proxy_pass_header Server;
		proxy_set_header Host $http_host;
		proxy_redirect off;
		proxy_set_header X-Real-IP $remote_addr;
		proxy_set_header X-Scheme $scheme;
		proxy_connect_timeout 10;
		proxy_read_timeout 10;
		proxy_pass http://127.0.0.1:8001/;
	}

	# what to serve if upstream is not available or crashes
	# error_page 500 502 503 504 /media/50x.html;
}

Enable the virtual host with a symbolic link from available to enabled:

$ sudo ln -s /etc/nginx/sites-available/MYPROJECT.ORG /etc/nginx/sites-enabled/MYPROJECT.ORG
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment