@credit Yan Zhu (https://github.com/nina-zhu)
Flask is a microframework for Python based on Werkzeug, Jinja 2 and good intentions, it can help you get your Python application or website off the ground. Flask includes a simplified development server for testing your code locally, but for anything even slightly production related, a more secure and powerful web server is required.
In this guide, we will demonstrate how to install and configure some components on Ubuntu 14.04 to support and serve Flask applications. We will configure the uWSGI application container server to interface with our applications. We will then set up Nginx to reverse proxy to uWSGI, giving us access to its security and performance features to serve our apps.
In order to complete this guide, you should have a fresh Ubuntu 14.04 server instance with a non-root user with sudo privileges configured.
We will be installing Flask within a virtual environment. This will allow your projects and their requirements to be handled separately. We will be creating a sample project, you can run through the steps to create as many projects as you want, it is intended to be a multi-project environment.
Once we have our applications, we will install and configure the uWSGI application server. This will serve as an interface to our applications which will translate client requests using HTTP to Python calls that our application can process. We will then set up Nginx in front of uWSGI to take advantage of its high performance connection handling mechanisms and its easy-to-implement security features.
Let's get started.
We will be installing our Flask projects in their own virtual environments to isolate the requirements for each. To do this, we will be installing virtualenv, which can create Python virtual environments, and virtualenvwrapper, which adds some usability improvements to the virtualenv work flow.
We will be installing both of these components using pip, the Python package manager. This can be acquired from the Ubuntu repositories:
sudo apt-get update
sudo apt-get install python-pip
In this guide, we are using Python version 2. If your code uses Python 3, you can install the python3-pip package. You will then have to substitude the pip commands in this guide with the pip3 command when operating outside of a virtual environment.
Now that you have pip installed, we can install virtualenv and virtualenvwrapper globally by typing:
sudo pip install virtualenv virtualenvwrapper
With these components installed, we can now configure our shell with the information it needs to work with the virtualenvwrapper script. Our virtual environments will all be placed within a directory in our home folder called .virtualenvs for easy access. This is configured through an environment variable called WORKON_HOME. We can add this to our shell initialization script and can source the virtual environment wrapper script.
If you are using Python 3 and the pip3 command, you will have to add an additional line to your shell initialization script as well:
echo "export VIRTUALENVWRAPPER_PYTHON=/usr/bin/python3" >> ~/.bashrc
Regardless of which version of Python you are using, you need to run the following commands:
echo "export WORKON_HOME=~/.virtualenvs" >> ~/.bashrc
echo "source /usr/local/bin/virtualenvwrapper.sh" >> ~/.bashrc
Now, source your shell initialization script so that you can use this functionality in your current session:
source ~/.bashrc
You should now have directory called .virtualenvs in your home folder which will hold virtual environment information.
Now that we have our virtual environment tools, we will create a virtual environment, install Flask in it, and start the project.
We can create a virtual environment easily by using some commands that the virtualenvwrapper script makes available to us.
Create your first virtual environment with the name of your first site or project by typing:
mkvirtualenv firstflask
This will create a virtual environment, install Python and pip within it, and activate the environment. Your prompt will change to indicate that you are now operating within your new virtual environment. It will look something like this: (firstflask)user@hostname:~$. The value in the parentheses is the name of your virtual environment. Any software installed through pip will now be installed into the virtual environment instead of on the global system. This allows us to isolate our packages on a per-project basis.
Our first step will be to install Flask itself. We can use pip for this without sudo since we are installing this locally in our virtual environment:
pip install Flask
With Flask installed, we can create our first sample project by typing:
cd ~
mkdir firstflask
cd firstflask/
vi firstflask.py
Type following code in the firstflask.py:
from flask import Flask
application = Flask(__name__)
@application.route("/")
def index():
return "Hello World!"
if __name__ == "__main__":
application.run()Save and close the file when you are finished. With all of that out of the way, we can test our project by temporarily starting the development server. Type:
cd ~
python firstflask/firstflask.py
This will start up the development server on port 5000. Now head over to http://127.0.0.1:5000/ in your browser, you should see a page that displays "Hello World!".
After testing this functionality out, stop the development server by typing CTRL-C in your terminal.
Since we are now done with the Flask portion of the guide, we can deactivate our virtual environment:
deactivate
If you need to work on your Flask site again, you should reactivate its respective environment. You can do that by using the workon command:
workon firstflask
Again, deactivate when you are finished working on your sites:
deactivate
Now that we have a Flask project set up and ready to go, we can configure uWSGI. uWSGI is an application server that can communicate with applications over a standard interface called WSGI.
Before we jump in, we should address some confusing terminology associated with the interrelated concepts we will be dealing with. These three separate terms that appear interchangeable, but actually have distinct meanings:
- WSGI: A Python spec that defines a standard interface for communication between an application or framework and an application/web server. This was created in order to simplify and standardize communication between these components for consistency and interchangeability. This basically defines an API interface that can be used over other protocols.
- uWSGI: An application server container that aims to provide a full stack for developing and deploying web applications and services. The main component is an application server that can handle apps of different languages. It communicates with the application using the methods defined by the WSGI spec, and with other web servers over a variety of other protocols. This is the piece that translates requests from a conventional web server into a format that the application can process.
- uwsgi: A fast, binary protocol implemented by the uWSGI server to communicate with a more full-featured web server. This is a wire protocol, not a transport protocol. It is the preferred way to speak to web servers that are proxying requests to uWSGI.
The WSGI spec defines the interface between the web server and application portions of the stack. In this context, "web server" refers to the uWSGI server, which is responsible for translating client requests to the application using the WSGI spec. This simplifies communication and creates loosely coupled components so that you can easily swap out either side without much trouble.
The web server (uWSGI) must have the ability to send requests to the application by triggering a defined "callable". The callable is simply an entry point into the application where the web server can call a function with some parameters. The expected parameters are a dictionary of environmental variables and a callable provided by the web server (uWSGI) component.
In response, the application returns an iterable that will be used to generate the body of the client response. It will also call the web server component callable that it received as a parameter. The first parameter when triggering the web server callable will be the HTTP status code and the second will be a list of tuples, each of which define a response header and value to send back to the client.
With the "web server" component of this interaction provided by uWSGI in this instance, we will only need to make sure our applications have the qualities described above. And before you ask, Flask applications have those qualities because Flask depends on the Werkzeug WSGI toolkit. Werkzeug is a WSGI utility library for Python, started as a simple collection of various utilities for WSGI applications and has become one of the most advanced WSGI utility modules.
In this tutorial, we'll be installing uWSGI globally. This will create less friction in handling multiple Flask projects. Before we can install uWSGI, we need the Python development files that the software relies on. We can install this directly from Ubuntu's repositories:
sudo apt-get install python-dev
Now that the development files are available, we can install uWSGI globally through pip by typing:
sudo pip install uwsgi
We can quickly test this application server by passing it the information for our site. For instance, we can tell it to serve our first project by typing:
uwsgi --http :8080 --home /home/user/.virtualenvs/firstflask --chdir /home/user/firstflask --manage-script-name --mount /=firstflask:application
Here, we've told uWSGI to use our virtual environment located in our ~/.virtualenvs directory, to change to our project's directory. The --manage-script-name will move the handling of SCRIPT_NAME to uwsgi, since it's smarter about that. It is used together with the --mount directive which will make requests to / be directed to firstflask:application. application is the callable inside of your application (usually the line reads application = Flask(__name__)). For our demonstration, we told it to serve HTTP on port 8080. If you go to server's domain name or IP address in your browser, followed by :8080, you will see your site again. When you are finished testing out this functionality, type CTRL-C in the terminal.
Running uWSGI from the command line is useful for testing, but isn't particularly helpful for an actual deployment. Instead, we will run uWSGI in "Emperor mode", which allows a master process to manage separate applications automatically given a set of configuration files.
Create a directory that will hold your configuration files. Since this is a global process, we will create a directory called /etc/uwsgi/sites to store our configuration files. Move into the directory after you create it:
sudo mkdir -p /etc/uwsgi/sites
cd /etc/uwsgi/sites
In this directory, we will place our configuration files. We need a configuration file for each of the projects we are serving. The uWSGI process can take configuration files in a variety of formats, but we will use .ini files due to their simplicity.
Create a file for your first project and open it in your text editor:
sudo vi firstflask.ini
Inside, we must begin with the [uwsgi] section header. All of our information will go beneath this header. We are also going to use variables to make our configuration file more reusable. After the header, set a variable called project with the name of your first project. Add a variable called base with the path to your user's home directory:
[uwsgi]
project = firstflask
base = /home/user
Next, we need to configure uWSGI so that it handles our project correctly. We need to change into the root project directory by setting the chdir option. We can combine the home directory and project name setting that we set earlier by using the %(variable_name) syntax. This will be replaced by the value of the variable when the config is read.
In a similar way, we will indicate the virtual environment for our project. By setting the module, we can indicate exactly how to interface with our project (by importing the "application" callable from the firstflask.py file within our project directory). The configuration of these items will look like this:
[uwsgi]
project = firstflask
base = /home/user
chdir = %(base)/%(project)
home = %(base)/.virtualenvs/%(project)
module = %(project):application
We want to create a master process with 5 workers. We can do this by adding this:
[uwsgi]
project = firstflask
base = /home/user
chdir = %(base)/%(project)
home = %(base)/.virtualenvs/%(project)
module = %(project):application
master = true
processes = 5
Next we need to specify how uWSGI should listen for connections. In our test of uWSGI, we used HTTP and a network port. However, since we are going to be using Nginx as a reverse proxy, we have better options.
Instead of using a network port, since all of the components are operating on a single server, we can use a Unix socket. This is more secure and offers better performance. This socket will not use HTTP, but instead will implement uWSGI's uwsgi protocol, which is a fast binary protocol designed for communicating with other servers. Nginx can natively proxy using the uwsgi protocol, so this is our best choice.
We will also modify the permissions of the socket because we will be giving the web server write access. We'll set the vacuum option so that the socket file will be automatically cleaned up when the service is stopped:
[uwsgi]
project = firstflask
base = /home/user
chdir = %(base)/%(project)
home = %(base)/.virtualenvs/%(project)
module = %(project):application
master = true
processes = 5
socket = %(base)/%(project)/%(project).sock
chmod-socket = 664
vacuum = true
uWSGI provides the ability to dynamically scale the number of running workers via pluggable algorithms. Use uwsgi --cheaper-algos-list to get the list of available algorithms.
To enable cheaper mode (adaptive process spawning) add the cheaper = N option to the uWSGI configuration file, where N is the minimum number of workers uWSGI can run. The cheaper value must be lower than the maximum number of configured workers (workers or processes option).
Change the file content to:
[uwsgi]
project = firstflask
base = /home/user
chdir = %(base)/%(project)
home = %(base)/.virtualenvs/%(project)
module = %(project):application
master = true
processes = 10
cheaper = 2
cheaper-initial = 5
cheaper-step = 1
cheaper-algo = spare
cheaper-overload = 5
socket = %(base)/%(project)/%(project).sock
chmod-socket = 664
vacuum = true
This configuration will tell uWSGI to run up to 10 workers under load. If the app is idle uWSGI will stop workers but it will always leave at least 2 of them running. With cheaper-initial you can control how many workers should be spawned at startup. If your average load requires more than minimum number of workers you can have them spawned right away and then "cheaped" (killed off) if load is low enough. When the cheaper algorithm decides that it needs more workers it will spawn cheaper-step of them. This is useful if you have a high maximum number of workers - in the event of a sudden load spike it would otherwise take a lot of time to spawn enough workers one by one.
The cheaper-algo option sets cheaper algorithm to use. As our load is very low, here we use the default algorithm spare. If all workers are busy for cheaper-overload seconds then uWSGI will spawn new workers. When the load is gone it will begin stopping processes one at a time. Other algorithms include spare2, backlog, busyness, please refer to the doc.
With this, our first project's uWSGI configuration is complete. Save and close the file.
We now have the configuration files we need to serve our Flask projects, but we still haven't automated the process. Next, we'll create an init script to automatically start uWSGI at boot.
We will create an init script in the /etc/init.d directory, where these files are checked:
sudo vi /etc/init.d/uwsgi
With following contents (substitute user with yours):
#!/bin/sh
#
# Simple uWSGI init.d script
PATH=/sbin:/usr/sbin:/bin:/usr/bin
EXEC=/usr/local/bin/uwsgi
CONFDIR=/etc/uwsgi/sites
PIDFILE=/var/run/uwsgi.pid
LOGFILE=/var/log/uwsgi.log
UID=user
GID=www-data
case "$1" in
start)
echo "Starting uWSGI server..."
$EXEC --emperor $CONFDIR --pidfile $PIDFILE --daemonize $LOGFILE --uid $UID --gid $GID
echo "uWSGI started"
;;
stop)
echo "Stopping ..."
$EXEC --stop $PIDFILE --vacuum
echo "uWSGI stopped"
rm $PIDFILE
;;
*)
echo "Please use start or stop as first argument"
;;
esac
We need to start uWSGI in Emperor mode and pass in the directory where we stored our configuration files. uWSGI will read the files and serve each of our projects. We also specify the pidfile so that we know which instance to stop. We daemonize the instance and output the log to the specified file.
We also need to set the username and group that the process will be run as. We will run the process under our own username since we own all of the files. For the group, we need to set it to www-data group that Nginx will run under. Our socket settings from the uWSGI configuration file should then allow the web server to write to the socket. Change the username above to match your username on the server.
When stop, we use the pidfile, and set the vacuum option to automatically clean up the socket files.
When you are finished, save and close the file. Then make the file executable:
sudo chmod +x /etc/init.d/uwsgi
We won't start uWSGI yet since we will not have the www-data group available until after we install Nginx.
With uWSGI configured and ready to go, we can now install and configure Nginx as our reverse proxy. This can be downloaded from Ubuntu's default repositories:
sudo apt-get install nginx
Once Nginx is installed, we can create 2 directories sites-available and sites-enabled in /etc/nginx directory, in order to enable and disable virtual hosts very easily by symlinking and unsymlinking:
sudo mkdir /etc/nginx/sites-available
sudo mkdir /etc/nginx/sites-enabled
Then open /etc/nginx/nginx.conf:
sudo vi /etc/nginx/nginx.conf
Add this line in the http block:
include /etc/nginx/sites-enabled/*;
Meanwhile comment out this line in the http block:
#include /etc/nginx/conf.d/*.conf;
If you remember from earlier, we set the group to www-data in uWSGI init script to allow Nginx to write to the socket, here we need to set Nginx user group to www-data too. The default setting user nginx; in /etc/nginx/nginx.conf omits group, a group whose name equals that of user is used, i.e. the group is nginx, that's not we want, we have to specify the group explicitly. Use this line to replace the original user nginx; line:
user nginx www-data;
Once you are finished, save and close /etc/nginx/nginx.conf.
Now we can go ahead and create a server block configuration file for each of our projects. Start with the first project by creating a server block configuration file:
sudo vi /etc/nginx/sites-available/firstflask
Inside, we can start our server block by indicating the port number and domain name where our first project should be accessible. We can use localhost if we don't have a domain name. We will also tell Nginx not to worry if it can't find a favicon:
server {
listen 80;
server_name localhost;
location = /favicon.ico { access_log off; log_not_found off; }
}
After that, we can use the uwsgi_pass directive to pass the traffic to our socket file. The socket file that we configured was called firstflask.sock and it was located in our project directory. We will use the include directive to include the necessary uwsgi parameters to handle the connection:
server {
listen 80;
server_name localhost;
location = /favicon.ico { access_log off; log_not_found off; }
location / {
include uwsgi_params;
uwsgi_pass unix:/home/user/firstflask/firstflask.sock;
}
}
That is actually all the configuration we need. Save and close the file when you are finished.
Next, link your new configuration file to Nginx's sites-enabled directory to enable it:
sudo ln -s /etc/nginx/sites-available/firstflask /etc/nginx/sites-enabled
Check the configuration syntax by typing:
sudo service nginx configtest
If no syntax errors are detected, you can restart your Nginx service to load the new configuration:
sudo service nginx restart
If you remember from earlier, we never actually started the uWSGI server. Do that now by typing:
sudo service uwsgi start
You should now be able to reach your project by going to its respective domain names, here is http://localhost/.
Once Nginx is installed using Ubuntu's default repositories, i.e. via sudo apt-get install nginx, it becomes a service by default, and will start automatically when the server is powered on. This is good, and suitable for a production server. But now we'll try a manual way, disable automatically starting, instead we start Nginx and uWSGI by executing a shell script file called start.sh, and stop them by executing stop.sh.
To disable Nginx automatically starting at boot, run following command:
sudo update-rc.d nginx disable
This will change S20nginx to K80nginx in /etc/rc[2-5].d directories, thus won't start Nginx at boot time.
For uWSGI, we created an init script for it by ourselves, it won't be automatically started at boot until you run:
sudo update-rc.d uwsgi defaults
Now restart your server, you'll find Nginx and uWSGI are not started by running:
ps -ef | grep nginx
ps -ef | grep uwsgi
And of course you can not access your site in http://localhost/.
You can definitely skip this step, automatically starting and manually starting can both exist. Anyway, let's get them started manually now.
Create a shell script called start.sh:
vi start.sh
Put these lines in it:
sudo service nginx start
sudo service uwsgi start
Save and close it. Make it executable:
chmod +x start.sh
Run it:
./start.sh
You can access your site http://localhost/ again.
Now that we can start Nginx and uWSGI manually, we can also stop them manually.
Create a shell script called stop.sh:
vi stop.sh
Put these lines in it:
sudo service nginx stop
sudo service uwsgi stop
Save and close it. Make it executable:
chmod +x stop.sh
Run it:
./stop.sh
You can not access your site http://localhost/ now.