Skip to content

Instantly share code, notes, and snippets.

@GrahamDumpleton
Last active August 29, 2015 14:08
Show Gist options
  • Save GrahamDumpleton/dca226dcb67518a9454a to your computer and use it in GitHub Desktop.
Save GrahamDumpleton/dca226dcb67518a9454a to your computer and use it in GitHub Desktop.
Delegate only sub URL to embedded mode.
# This is example configuration of how a specific sub URL of a Python
# web application can be delegated to run in mod_wsgi embedded mode
# where the remainder of the site is delegated to run in mod_wsgi
# daemon mode.
#
# The particular example in this case is to work around the current
# issue that the optional enabling of non WSGI handling of chunked
# request content doesn't work for mod_wsgi daemon mode. A fix for
# that has been developed but hasn't yet been released. As Linux
# distributions ship older mod_wsgi versions and do not update them,
# that fix will will not help anyway.
#
# The example works on the idea that one can vertically partition a
# Python web application, having parts run under different mod_wsgi
# daemon process groups, or even in Apache child worker processes
# themselves, based on requirements and characteristics of each
# part of the application. For more details see:
#
# * http://blog.dscpl.com.au/2014/02/vertically-partitioning-python-web.html
# First up, where mod_wsgi daemon mode is used for an entire Python
# web application, the typical configuration will be.
WSGIDaemonProcess mysite
WSGIProcessGroup mysite
WSGIScriptAlias / /some/path/wsgi.py
<Directory /some/path>
<Files wsgi.py>
Order deny,allow
Allow from all
</Files>
</Directory>
# One hopes that if this is the only WSGI application being hosted
# that it is also forced to run in the main (first) Python interpreter
# context, but that is quite often not the case, resulting in a
# small amount of memory being wasted, but worse is that some third
# party packages will not work properly in sub interpreters and can
# crash and/or deadlock the process. The preferred configuration if
# running one Python web application per mod_wsgi daemon process group
# is to also have:
WSGIApplicationGroup %{GLOBAL}
# There are now two choices one can use to have just a specific sub
# URL of a site run in a different mod_wsgi daemon process group or
# in embedded mode. The first is to delegate a specific subset of URLs
# from the main application to run in the different context. Here we
# are going to have it run in embedded mode. Because we also want it
# to be able to handle chunked request content, we enabled that non
# WSGI extension as well.
<Location /suburl-1>
WSGIProcessGroup %{GLOBAL}
WSGIApplicationGroup %{GLOBAL}
WSGIChunkedRequest On
</Location>
# The other option is where the handler for that specific suburl is
# actually a different WSGI application altogether. In that case you
# can use the following. The only trick here is that this use of the
# WSGIScriptAlias MUST be placed before that above for '/' as sub URL
# must come first due to the precedence rules. So make sure you move
# this up. It is included at this point here purely so can explain it
# after the above in this description.
WSGIScriptAlias /suburl-2 /some/other/path/wsgi.py
<Directory /some/other/path>
<Files wsgi.py>
Order deny,allow
Allow from all
WSGIProcessGroup %{GLOBAL}
WSGIApplicationGroup %{GLOBAL}
WSGIChunkedRequest On
</Files>
</Directory>
# As soon as you start to run anything in embedded mode, there are some
# risks, especially if you don't control the overall Apache installation
# and configuration.
#
# The first problem is what Apache MPM is being used. If it is prefork MPM
# then immediately you will have a problem with having lots of copies of
# this special WSGI application or sub URL of the bigger site. This is
# because prefork MPM uses many single thread processes. Worse is if the
# MPM has been set up with PHP in mind as PHP defaults are really bad for
# Python web applications. You can see more about this problem in:
#
# * http://lanyrd.com/2013/pycon/scdyzk/
#
# Ideally for a Python web application worker or event MPMs will be used
# and it will be configured to keep a high percentage of processes always
# persistent and not fallback on Apache dynamic process scaling which can
# cause problems when used with Python web applications as explained in
# the conference talk above.
#
# A second problem if trying to do this in a hosting environment where
# they set most things up for you is if they have disabled the ability to
# use embedded mode. Although this is actually good practice if only using
# daemon mode, most of the time they don't even realise they should have
# been doing it and so embedded mode isn't disabled. Not disabling
# embedded mode if not using it is bad as it means the Python interpreter
# is initialised in the Apache child worker processes when there is no
# need. This slows down spin up of new child worker processes and uses more
# memory for no reason.
#
# If however they did actually disable it, you would have to undo that. For
# that you would use the following, which would need to be at global scope
# outside of any VirtualHost.
WSGIRestrictedEmbedded Off
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment