Inspired from: https://chrisyeh96.github.io/2017/08/08/definitive-guide-python-imports.html
There are situations where you need to use a Python package, but you cannot just pip install it.
It could be because of complex dependencies, or you need to maintain a fork of it and be able to edit the code, or the code is private and you don't have or want to run a private package index.
Regardless, one way to deal with this is to add the package as a subdirectory in your own project.
The way this works is usually by using Git submodules if the package is a repository on Git.
The problem now is how to import it.
Python 3 has made implicit relative imports illegal. And historically the recommendation is to use
absolute imports with the package name as the importing name. So if the package was called package.
This usually means the repository has a package directory, and inside that directory is __init__.py
and other module files. These modules now import things via from package import module.
There is actually a better alternative. It is in my opinion one should be using explicit relative imports
instead. So instead of from package import module, modules should be using from . import module or
from .module import function. This can be used in the __init__.py to further export internal modules
as exposed constructs for the package itself. (Remember all packages are module!).
However while you can follow this advice for code under your control. If you don't want to maintain an
alternative fork, or try to push it upstream, you will have to perform a sys.path hack.
Basically inside your own package, your top-level __init__.py should have something like this:
import sys, os.path as path
from inspect import getsourcefile
parent_dir = path.dirname(path.dirname(path.abspath(getsourcefile(lambda: 0))))
sys.path.append(path.join(parent_dir, 'RepositoryName'))
# make sure to hack the sys.path
# before importing your own modules
from . import module
from .module import *Just replace RepositoryName with the name of the repository of the package. Now you can use
from package import module within your own modules.
Best thing about this, is that this will work even when the package does become pip installable.
In that case, you just need to remove the embedded directory, and also the above snippet in your
__init__.py.
Beware that doing things this way allows you to easily use Git submodules as a dependency management system. However it doesn't work as nicely if the subdirectory package has its own dependencies that are managed outside Git submodules. Unless your top-level project has all the dependencies of the subdirectory package, you still have to bring in the subdirectory package's dependencies. It may be better to try to reduce the complexity of this and only use one dependency management tool.