Transparently split python package (for binary extensions)

README.md

Summary

This is a demo of how a python package can be transparently split between two locations. This setup is particularly useful when your package contains binary extension modules that are built out of the source tree, i.e. with cmake, but your package also contains pure python source files. With the setup described below all modules will appear as a single unified package, while retaining their separate file locations. All that is required is that root/build/ is on the python path and that root/build/mypkg/__init__.py has information about the absolute location of root/src. With this strategy any errorprone file copying is avoided.

Directory Structure

root/
    build/
        test.py
        mypkg/
            __init__.py
            b.py
    src/
        mypkg/
            __init__.py
            a.py

build-mypkg-__init__.py

mport sys
from pkgutil import extend_path

PKG_SRC_PATH = '@PACKAGE_SRC@'.split(';')
for path in reversed(PKG_SRC_PATH):
    sys.path.insert(0, path)
    del path

__path__ = extend_path(__path__, __name__)
del PKG_SRC_PATH
del extend_path
del sys

build-mypkg-b.py

def hello():
    print "Hello from", __name__, "located at", __file__

build-test.py

# mypkg is split between a src and a build folder
from mypkg import a, b

a.hello()
b.hello()

src-mpkg-a.py

def hello():
    print "Hello from ", __name__, "located at", __file__

src-mypkg-__init__.py

# Just a regular package init file