Setting environment variables for Apache/mod_wsgi hosted Python application.

gistfile1.md

Django documentation says to use:

WSGIScriptAlias / /path/to/mysite.com/mysite/wsgi.py
WSGIPythonPath /path/to/mysite.com

<Directory /path/to/mysite.com/mysite>
<Files wsgi.py>
Require all granted
</Files>
</Directory>

To set environment variables which would be inherited by the Django application referred to by this, one can set them in the envvars file created as part of the Apache installation. There are a few problems with that though.

The first is that Linux distributions created a completely different way of configuring Apache and ignored in large part how the Apache Software Foundations default Apache distribution works. How you would do the equivalent varies between Linux distributions but generally requires setting some other special file alongside the system init scripts. Unfortunately some Linux distributions didn't make this a separate file and instead require you to modify the init scripts themselves.

The second is that the envvars file applies to the whole of Apache and cannot be made to apply to specific mod_wsgi daemon processes. This makes it unsuitable where hosting multiple Python web applications under the same Apache.

The much simpler way is to create a separate WSGI script file independent of the wsgi.py that Django generates with its startproject template. You might call this django.wsgi, or you might have separate production.wsgi and development.wsgi files if have multiple environments. Place the WSGI script file in the same directory as the wsgi.py file. In the file add something like:

import os

os.environ['DJANGO_SETTINGS_MODULE'] = 'mysite.settings'

os.environ['DJANGO_CONFIGURATION'] = 'Production'
os.environ['DJANGO_SECRET_KEY'] = 'RANDOM_SECRET_KEY_HERE'
os.environ['DJANGO_AWS_ACCESS_KEY_ID'] = 'YOUR_AWS_ID_HERE'
os.environ['DJANGO_AWS_SECRET_ACCESS_KEY'] = 'YOUR_AWS_SECRET_ACCESS_KEY_HERE'
os.environ['DJANGO_AWS_STORAGE_BUCKET_NAME'] = 'YOUR_AWS_S3_BUCKET_NAME_HERE'
os.environ['SENDGRID_USERNAME'] = 'YOUR_SENDGRID_USERNAME'
os.environ['SENDGRID_PASSWORD'] = 'YOUR_SENDGRID_PASSWORD'

from django.core.wsgi import get_wsgi_application
application = get_wsgi_application()

If you are concerned that you might have sensitive settings in a file which is part of your source code repository. Then have just the environment variables being set in a separate Python code file which you manually copy into place on the host in a directory such as /etc.

For example, on OpenShift, you would place the file in ${OPENSHIFT_DATA_DIR} and then use:

import os

OPENSHIFT_DATA_DIR = os.environ['OPENSHIFT_DATA_DIR']

os.environ['DJANGO_SETTINGS_MODULE'] = 'mysite.settings'

execfile(os.path.join(OPENSHIFT_DATA_DIR, 'mysite-production-envvars.py'))

from django.core.wsgi import get_wsgi_application
application = get_wsgi_application()

Alternatively, you could put the WSGI script file in a completely different location outside of the source code for your application, even putting it in a subdirectory under /etc. In other words, tell your system adminstrator what needs to go in it and let them create it where they want to per their policies.

With the modified WSGI script file in place, change the name of the WSGI script file in the Apache configuration snippet to refer to it instead of the original wsgi.py.

WSGIScriptAlias / /path/to/mysite.com/mysite/production.wsgi
WSGIPythonPath /path/to/mysite.com

<Directory /path/to/mysite.com/mysite>
<Files production.wsgi>
Require all granted
</Files>
</Directory>

Change the WSGIScriptAlias and Directory directive paths if you do put it under /etc somewhere, but leave the WSGIPythonPath setting alone as that still needs to refer to where the source code for the project resides.

If using mod_wsgi daemon mode as you should be, you would instead also have something like:

WSGIDaemonProcess example.com python-path=/path/to/mysite.com:/path/to/venv/lib/python2.7/site-packages
WSGIProcessGroup example.com

So you would be setting the python-path option there to the location of your your source code instead. If using a modern version of mod_wsgi, use python-home instead.

• http://blog.dscpl.com.au/2014/09/python-module-search-path-and-modwsgi.html

This method can be used for any environment variables with the exception of those which affect the operation of the processes as a whole at a system level. That is, you cannot set LANG and LC_ALL in this way. Modern versions of mod_wsgi though have lang and locale options on WSGIDaemonProcess directive which achieve the same result. If not using daemon mode, you should be as embedded mode isn't generally recommended.

• http://blog.dscpl.com.au/2014/09/setting-lang-and-lcall-when-using.html

The only other problematic environment variable is LD_LIBRARY_PATH. This is only a problem where shared libraries aren't installed in a proper system location. If this is unavoidable, set LD_RUN_PATH to the library directory when installing any Python modules using the library and the location will be embedded in the Python module extension so that the shared library will be found whether or not LD_LIBRARY_PATH is set properly.

In summary, ignore the Django generated wsgi.py file. This never used to exist and that it started to be generated has caused problems with understanding how to set up Apache/mod_wsgi ever since. The first problem was that it used os.environ.setdefault() which causes problems when running multiple Django applications in the same Apache process. Secondly, that wsgi.py was auto generated, it gave users the impression that it would always be correct, must be used and wouldn't need modifying. Consequence was that people stopped seeing what one of the main roles of the WSGi script file was for, which was to set environment variables before the Django application got loaded.

@GrahamDumpleton Thanks fo this! One question, why not just add the env variables to the auto generated django wsgi.py?

The original suggestion was from many many years ago. Gist doesn't show the true date, it wasn't when the first comment was. The information would have been posted at a time when people were still using Django versions which didn't include wsgi.py.

So the convention was based on not having a wsgi.py file to begin with, but also to avoid changing what was a generated code file. Also because wsgi.py was being generated, it was usually under version control in a Git repo or other source code control system. You don't want to put sensitive information like database credentials in a Git repo, especially if a public repo. The entry point file being separate means it could be manually added to a host system.

The contents of wsgi.py also isn't always correct for use with mod_wsgi. This is because they changed to using os.environ.setefault() for accessing DJANGO_SETTINGS_MODULE which breaks setups where have multiple Django applications in one deployment. Changing things to suit mod_wsgi when you were changing what was preferred for runserver. See "Leaking of process environment variables" in http://blog.dscpl.com.au/2012/10/requests-running-in-wrong-django.html

Thanks Graham, true wsgi.py is under version control where sensitive data would cause trouble. But as I do not host several Django applications I will try my luck by just reading the vars in existing mod_wsgi.py from an external path:

execfile('/etc/mysite-production-envvars.py'))

If I'm right it should work that way without having to modify apache conf. This article, even old, is still very useful!

Hi @GrahamDumpleton, is there a way to use apache PassEnv to pass the system env variables to the wsgi file ?
The goal is to have the exact same machine in test and production and just play with the env variables.

If have tried the following but still get KeyError when loading the env vars in settings.py :

from django.core.wsgi import get_wsgi_application 
_application = get_wsgi_application()

def application(environ, start_response):
    for key in environ:
        if key.startswith('APP_'):
            os.environ[key] = environ[key]

    return _application(environ, start_response)

I tried to load the vars with SetEnv or PassEnv in apache without success

Thank you

You should not update os.environ per request as it is bad practice and can have unintended consequences if running multiple Python WSGI applications in the same setup, so don't do that.

As to PassEnv, provided the environment variables are truly set in the environment of the script that starts Apache, they would appear in os.environ, not the per request environ dictionary.

Where are you setting the system environment variables for Apache?

In a standard Apache distribution there is an envvars file you would set them in. Unfortunately, Linux distributions ignored that approach and do their own custom startup mechanisms, so I have no idea where a specific Linux distro may require you set environment variables to have a specific system service inherit them.

It is because Linux distros break the normal way of setting things up in Apache, that the originally described approach is recommended.

Thank you for the article. I ran into a similar problem with Flask, Python, and WSGI. I was able to use the Apache "SetEnv" directive to set what I needed. (In my case, FLASK_ENV="development" so that flask would automatically pick up code changes) in the relevant apache virtual host file.)

https://httpd.apache.org/docs/2.4/mod/mod_env.html#setenv