In this tutorial, I briefly explain how to set up a webserver using nginx, openssl and uwsgi on Arch Linux. The tutorial is applicable to other Linux distributions and goes through the required configuration step by step. It is, I believe, beginner friendly.
I updated, changed and tested the tutorial on Dec. 8th, 2013. All commands and
configuration files relate to the software versions that are currently available
in the Arch package repositories. Especially nginx and uwsgi change often and
quickly, so check out new cool features from time to time and follow there releases.
Arch Linux is an amazing Linux distribution. Among other great things, it is extremely lightweight, ships with pacman, the best package manager I have seen so far, and has a great wiki with all you need to know to get it up and running.
After having used Arch on my laptop and desktop machine at work for a couple of years now, I decided to give it a try running on my webserver. Because it has worked reasonably stable for almost a year now and there are only very few tutorials for running Arch as a server, I decided to write a comprehensive tutorial for setting up a full web-server stack on Arch Linux.
Although written for Arch, this tutorial is in general applicable to any Linux distribution. Since Arch uses mostly standard locations for configuration files, the only Arch specific commands will be the installation of packages.
One important thing must be said first. Arch is a rolling release distribution. Its repositories always contain the newest upstream versions of software and the linux kernel. This is usually not needed for server setups and there are a lot more stable and well-tested Linux distributions, like for example Debian Linux. Things may break! The more often you change or update software, especially when it is not thoroughly tested, the higher is the risk of rendering something unuseful, introducing security vulnurabilities or even causing boot- or runtime errors. For this reason, do only use Arch as a production server, when you are sure what you are doing, are able to fix things in some kind of rescue system and are not hosting extremely important web services or -sites.
In the (almost) ten months that I have been running Arch as a server, I experienced only a few critical problems and all of them could be fixed quickly. However, I did have more downtime than for instance Debian would have shown. So act on you own risk!
The tutorial is more or less easily applicable to other Linux distributions.
You do need to know how to use the corresponding package manager to install
software packages. Also, some newer software may not be in your repositories
in which case you need to compile it yourself. Currently, this applies for instance to
uwsgi on Debian.
Arch switched to the systemd system daemon, a rather new init system. Therefore the startup of daemons or services may be different for your distribution. For example, starting the reverse proxy nginx requires executing
# systemctl start nginx
on Arch, but you would need to run
# /etc/init.d/nginx start
on Debian based distributions.
This tutorial is about webservers. By webserver I mean the following stack of applications:
- nginx: A webserver to serve static files and a reverse proxy for our application-specific servers. I will include the inclusion of ngx_pagespeed, the pagespeed module by google.
- uwsgi: A stack of servers (or servlets) for our web applications. I will show how to set up the uwsgi emperor to make deployment of additional apps as easy as possible.
- OpenSSL: I will spend some time explaining how to create SSL certificates for secure browsing.
- the application: Be it written in PHP, Python, Lua, etc., this is the actual program we want to serve.
All parts of this setup can be replaced by other software. Feel free to skip chapters to read only the part on a single application. They are mostly decoupled anyway.
You should have a ready to use Arch Linux system set up. It does not need a graphical user interface. You need root access and a working network connection. Before you start, update your system once again by executing
# pacman -Syu
We will use the user http with its home directory /srv/http for the webserver.
For some reason, that directory is not owned by the user http on a fresh Arch install,
so that we have to adjust permissions:
# chown http:http /srv/http
Also, no default shell is set for http, so set one. (Replace zsh with whichever shell
you like most)
# chsh -s /usr/bin/zsh http
Before we begin, some notes on how to read the tutorial:
- I will most of the times say Arch instead of Arch Linux and you may replace that with the name of your preferred Linux distribution in your head.
- CLI commands are prefixed with a #, when executed as root and with $ when
executed as some other user. The user is, in all cases,
http! Messing that up may break file permissions! - Typos are intended!
Especially when hosting multiple sites that are served by multiple servlets,
it is helpful to have a smart and consistend file and folder structure for the
different projects. Both uwsgi and
nginx, the servers we are using, support globbing in
their configuration files. That means we can have them look for config files
in, say, /path/to/projects/*/*.conf, where * is the name of different
projects.
Arch ships with a folder /srv/http, which is the home directory of the
http user. This is the perfect place for our projects to reside. I suggest
and use the following structure for my projects:
/srv/http/project1nginx.conf: The nginx configuration fileuwsgi.ini: The uwsgi configuration filelog/: All the logs for this project are stored hererun/: pid files and sockets are created herehtdocs/: This is where the application files reside.
/srv/http/project2...
This setup is portable and keeps everything belonging to a single project in
the corresponding folder. It also separates application logic (htdocs/) from
run files and so on.
Throughout the tutorial, we will work with an example project. Let us first
create its files and folders which we will edit later. Following the project
structure explained above, these commands will create the necessary structure.
Please note, that in future I will skip the su (set user) to http. For
commands prefixed with a $ sign, I assume the current user to be http.
# su http
$ cd
$ mkdir -p example/log example/run example/htdocs
$ touch example/nginx.conf example/uwsgi.ini
$ touch example/htdocs/index.html example/htdocs/index.py
These are all files we'll need.
Let us begin by installing and setting up nginx to serve static files. Later, we will also configure it as a reverse proxy to serve our dynamic web applications.
nginx (pronounced "engine x") is a webserver and reverse proxy for the HTTP protocol. It can also serve email stuff, but we will focus on serving websites. nginx is a bit more than ten years old and was written by a russian guy named Igor Sysoev. Recently, nginx was turned into a company and received a good amount of funding. It is among the three most popular webservers, together with Apache and Microsoft-IIS.
Other than its big and famous brother Apache, it only serves static files, like html, css and javascript and can act as a reverse proxy. It is NOT capable of handling dynamic stuff itself, i.e. spawning PHP interpreters or managing fCGI servers. Instead, it routes traffic to such servers through sockets (or via local network) and serves their responses to the client. This is the reverse proxy part of nginx.
I use nginx, because it has a very low memory footprint, is extremely fast and very actively developed. Considering the recent funding, at least within the next few years it probably will only become better and more famous, so it is a somewhat future-proof choice.
Install nginx by executing
# pacman -S nginx
Easy.
Let us clean up the default configuration file shipped with nginx and change some stuff. My nginx configuration looks like this at the moment:
user http;
worker_processes 1;
events {
worker_connections 1024;
}
http {
include mime.types;
default_type application/octet-stream;
sendfile on;
keepalive_timeout 65;
gzip on;
include /srv/http/*/nginx.conf;
}
The differences to the default configuration are the following:
- I removed all commented lines.
- I set
user http;because I want the webserver to run under this uid - I uncommented
gzip on; - I added
include /srv/http/*/nginx.conf;to the bottom of thehttpblock.
nginx will now look in /srv/http/*/nginx.conf for additional configuration
files. Each time you add an application in /srv/http with an nginx.conf file
and reload the webserver, this will be read and applied. This implies, that a mistake
in any of the files will prevent the server from running at all, so be careful!
Let us now try to serve a Hello World with nginx. Put some string into
/srv/http/example/htdocs/index.html:
$ cd ~/example
$ echo "Hello World\!" > htdocs/index.html
Now, we edit /srv/http/example/nginx.conf so that nginx knows about
our project:
server {
listen 80;
server_name example.YOURDOMAIN;
error_log /srv/http/example/log/nginx.error.log;
access_log /srv/http/example/log/nginx.access.log;
root /srv/http/example/htdocs;
index index.html;
}
Don't forget to replace YOURDOMAIN with your domain, of which example.
will be the subdomain to our project. The server_name directive should
contain the domains under which your project should to be accessible.
It may contain multiple domains separated by spaces, i.e.,
server_name www.example.com example.com subd1.example.com;
Now, simply reload the nginx config, pray that everything works fine and visit your projects address:
# systemctl start nginx
You should now be able to type the project domain (example.YOURDOMAIN above)
in the browsers address field and be greated with
Hello World!
nginx is working now, so let's go on by setting up uwsgi to serve some dynamic pages like PHP, Python, Lua and so on.
uWSGI is a protocol that can be used to
deploy dynamic web applications with webservers like nginx. To use it, you
need to have a uWSGI server, of which the most popular one is
uwsgi, developed by the italian company
unbit. It was originally created to serve Python
applications, hence the WSGI in the name, but it now includes many more
components, i.e., CGI, PHP, Lua, Rack, Go, etc.
uwsgi is highly configurable and follows a one-app-per-server approach. Each application will have its own uwsgi configuration file (and on or multiple uwsgi server threads), in which the configuration can be adjusted to match the purpose of the application as nicely as possible.
uwsgi is perfect when you want to serve all sorts of different applications. I am using it (among other stuff) to serve django (Python), cgit (CGI) and Drupal (PHP) sites. In our setup, we will make use of the uwsgi emperor. The emperor is a process that looks for uwsgi configuration files in certain directories, called vassals, and manages the corresponding uwsgi servers. It watches over the vassals and is capable of heal the servers after a crash.
I recommend installing uwsgi following a modular approach.
# pacman -S uwsgi-plugin-common
You might need to install some dependencies first. As an example, we will serve a python app, so we also need the Python plugin of uwsgi.
# pacman -S uwsgi-plugin-python2
The packages do not include any init scripts or systemd services. Therefore,
create the file /etc/systemd/system/uwsgi.service with the following content:
[Unit]
Description=uWSGI Server
[Service]
ExecStart=/usr/bin/uwsgi --emperor "/srv/http/*/uwsgi.ini" --uid=http --gid=http --vassals-inherit /srv/http/base.uwsgi.ini
SuccessExitStatus=30
ExecReload=/bin/kill -HUP $MAINPID
KillSignal=SIGINT
Restart=always
Type=notify
NotifyAccess=all
[Install]
WantedBy=multi-user.target
The unit spawns the uwsgi emperor who looks in /srv/http/*/uwsgi.ini for
application-specific uwsgi configuration files. The processes are started as
the http user and group. All configuration files will share the content in
/srv/http/base.uwsgi.ini. This is to save typing in additional config files.
First, set up the base uwsgi config file /srv/http/base.uwsgi.ini mentioned above:
[uwsgi]
uid = http
gid = http
Though these two lonely directives seem funny, this would be the place to put config
entries, that are shared by all the vassals, i.e., the actual applications in /srv/http.
An option in a vassal's uwsgi.ini always overrides the value in the base.uwsgi.ini.
Now, let us configure our project to be served via uwsgi. Edit the
/srv/http/example/uwsgi.ini file and insert the following:
[uwsgi]
plugin = python2
socket = %drun/%n.sock
chdir = %dhtdocs
wsgi-file = index.py
master = True
pidfile = %drun/%n.pid
logto = %dlog/%n.log
The variables %d and %n contain the project path (with trailing slash,
unfortunately) and the filename of the uwsgi config file without the
extension, respectively. In the config, we specify the plugin to use
(python2), the socket and pidfile location, the location of the logfile and
the user and group for this vassal.
With chdir= we change into our projects htdocs/ directory and wsgi-file
is the python file that contains the application callable (see next
section).
In the section above, we defined a socket location for uwsgi. We will now
configure nginx to use that socket when it wants to talk to uwsgi. Edit
/srv/http/example/example.nginx.conf and change it to the following snippet:
server {
listen 80;
server_name example.YOURDOMAIN;
error_log /srv/http/example/log/error.log;
access_log /srv/http/example/log/access.log;
location / {
include uwsgi_params;
uwsgi_modifier1 30;
uwsgi_pass unix:/srv/http/example/run/uwsgi.sock;
}
}
uwsgi_modifier1 defines the type of uWSGI request. You can read about its
values here.
With uwsgi_pass we tell nginx where to look for the socket. The inclusion
of uwsgi_params sets some default values for the variables.
We can drop the root and index directives from the nginx.conf file, because
the behaviour and paths are configured in our uwsgi.ini file and taken care of by
uwsgi.
The last step for our running python app is the application itself. We will
simply use the example from
Wikipedia.
Edit /srv/http/example/htdocs/index.py and insert the following lines of
python code:
def application(env, start_response):
start_response('200 OK', [('Content-Type','text/html')])
return "Hello World from python!\n"
This is a minimal example for a WSGI app. It simply defines a callable
(application()) that initializes a HTTP response and returns some content.
Of course, you will want to use some web framework for more sophisticated
applications. Great examples are flask,
webpy, Django etc.
If you made no mistakes during the configuration, you should now be able to restart the servers and visit your app.
# systemctl start uwsgi
# systemctl reload nginx
You can inspect the uwsgi emperor process using systemctl:
# systemctl status uwsgi
uwsgi.service - uWSGI Server
Loaded: loaded (/etc/systemd/system/uwsgi.service; disabled)
Active: active (running) since Sun 2013-12-08 19:39:23 UTC; 2min 45s ago
Main PID: 529 (uwsgi)
Status: "The Emperor is governing 1 vassals"
CGroup: /system.slice/uwsgi.service
├─529 /usr/bin/uwsgi --emperor /srv/http/*/uwsgi.ini --uid=http --gid=http --vassals-inherit /srv/http/base.uwsgi.ini
├─546 /usr/bin/uwsgi --ini /srv/http/example/uwsgi.ini --inherit /srv/http/base.uwsgi.ini
└─552 /usr/bin/uwsgi --ini /srv/http/example/uwsgi.ini --inherit /srv/http/base.uwsgi.ini
Looks good, hm? Now, by visiting
http://example.YOURDOMAIN with your browser, you
should see the python generated Hello World from python! greeting. Success!
In this section, I will briefly explain how to enable SSL support for your nginx projects. We will create a self-signed SSL certificate and configure nginx to use it for your sites.
It is not that easy to explain SSL (Secure Sockets Layer) in just a few sentences. Basically the communication between the client and the server is encrypted. The authentication is determined once per request by handshaking during which the servers certificate must be accepted by the client. This happens automatically, when the client already knows about the CA (certificate authority) that signed the servers certificate. Several root certificates of such CAs are built into OSes, Browsers and so on, but it is usually quite expensive to have your certificate signed by those CAs. We will therefore sign our certificates ourselves. The downside of that is that each client must accept our certificate manually before he can talk to our server.
You can read more about SSL on Wikipedia.
In case you don't have OpenSSL installed (it's a dependency of OpenSSH, so you probably have it already), do it now:
# pacman -S openssl
The following steps are shamelessly stolen from here. Sorry for that!
First, let us create some temporary folder to work in:
$ cd $(mktemp -d)
Now, we create our private key. A passphrase must be given, but we can remove it later.
$ openssl genrsa -des3 -out myssl.key 1024
For our key, we create a certificate signing request. In commercially signed
certificates, this is what we would send to the CA. You will be asked some
information about your certificate. For the common name insert the domain
you want to secure.
$ openssl req -new -key myssl.key -out myssl.csr
This is the example output of my CSR:
Enter pass phrase for myssl.key: *******
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [AU]:DE
State or Province Name (full name) [Some-State]:Hessen
Locality Name (eg, city) []:Marburg
Organization Name (eg, company) [Internet Widgits Pty Ltd]:
Organizational Unit Name (eg, section) []:
Common Name (e.g. server FQDN or YOUR name) []:example.YOURDOMAIN
Email Address []:YOU@YOURDOMAIN
Please enter the following 'extra' attributes
to be sent with your certificate request
A challenge password []:
An optional company name []:
Now we remove the passphrase from our private key:
$ cp myssl.key myssl.key.org
$ openssl rsa -in myssl.key.org -out myssl.key
And finally, we can create our SSL certificate:
$ openssl x509 -req -days 365 -in myssl.csr -signkey myssl.key -out myssl.crt
Done! In your temporary folder there should be a myssl.key and myssl.crt
file. Those two files are all we need. Let us move them to some more
appropriate location.
# cp myssl.crt /etc/ssl/certs/example.YOURDOMAIN.crt
# cp myssl.key /etc/ssl/private/example.YOURDOMAIN.key
Now, we need to configure nginx for SSL encryption. Change your
/srv/http/example/example.nginx.conf to the following snippet:
server {
listen 443 ssl;
server_name example.YOURDOMAIN;
error_log /srv/http/example/log/error.log;
access_log /srv/http/example/log/access.log;
ssl_certificate /etc/ssl/certs/example.YOURDOMAIN.crt;
ssl_certificate_key /etc/ssl/private/example.YOURDOMAIN.key;
location / {
include uwsgi_params;
uwsgi_modifier1 30;
uwsgi_pass unix:/srv/http/example/run/uwsgi.sock;
}
}
That's it! Restart nginx...
# systemctl reload nginx
... and visit https://example.YOURDOMAIN. If you have to accept some certificate, you succeeded. Congratulations!
As Christian thankfully pointed out in the comments, we want our log files to be rotated so
they don't grow too large. This is as simple as adding the file /etc/logrotate.d/webserver
with the following content:
/srv/http/*/log/*.log {
compress
rotate 4
size 1024k
notifempty
missingok
noolddir
}
Now, once a day the tool logrotate checks all our logfiles and, if they grew larger than
1024k, rotates them.
What is usually part of dynamic web apps, is a database. Recently, SQLite received much attention, but of course there are others. Since databases are so decoupled from the contents of this tutorial, I will not go through the steps of setting one up.
I myself have an instance of MariaDB (replacement of MySQL) running on my server. I usually create for each project a single database and its own database user. This enhances portability. Another popular and probably better choice than MariaDB is PostgreSQL.
The last part of the tutorial is some further reading suggestions. The following links a worth reading and will help you setting up additional stuff in web- servers.
- uwsgi docs. All about configuration of uwsgi and more.
- how to configure cgit, django, trac, piwik, ownCloud, Roundcube with uwsgi
- nginx configuration. Read here how to fine tune nginx.
- How to use django with uwsgi
- uwsgitop, a small tool to monitor your uwsgi vassals AUR package
- SSL Wikipedia Article
I really hope that this tutorial is helpful for somebody. I enjoy working with nginx and uwsgi and hope to spread this joy. Also, Arch rules! I am glad to answer questions in the comments!