|"""IPython startup script to detect and inject VIRTUAL_ENV's site-packages dirs.|
|IPython can detect virtualenv's path and injects it's site-packages dirs into sys.path.|
|But it can go wrong if IPython's python version differs from VIRTUAL_ENV's.|
|This module fixes it looking for the actual directories. We use only old stdlib|
|resources so it can work with as many Python versions as possible.|
|Author: Henrique Bastos <firstname.lastname@example.org>|
|from warnings import warn|
|virtualenv = os.environ.get('VIRTUAL_ENV')|
|version = os.listdir(os.path.join(virtualenv, 'lib'))|
|site_packages = os.path.join(virtualenv, 'lib', version, 'site-packages')|
|lib_dynload = os.path.join(virtualenv, 'lib', version, 'lib-dynload')|
|if not (os.path.exists(site_packages) and os.path.exists(lib_dynload)):|
|msg = 'Virtualenv site-packages discovery went wrong for %r' % repr([site_packages, lib_dynload])|
|i = sys.path.index("") + 1|
|i = 0|
May 5, 2017
Nice! Done! I'm trusting you've tested it #lol
May 5, 2017
Yes. I'm using it for a few days and it's working.
Oct 7, 2017
I was following your Medium post to get something setup on my machine. I started running into some issues with packages that used
setup tools) in their
__init__.py files. The error would look like:
Python 3.6.1 (default, Oct 6 2017, 10:41:43) Type 'copyright', 'credits' or 'license' for more information IPython 6.2.1 -- An enhanced Interactive Python. Type '?' for help. In : from pkg_resources import get_distribution In : __version__ = get_distribution('numpy').version --------------------------------------------------------------------------- DistributionNotFound Traceback (most recent call last) <ipython-input-2-3b18f3abdfeb> in <module>() ----> 1 __version__ = get_distribution('numpy').version ~/.pyenv/versions/3.6.1/envs/jupyter3_install/lib/python3.6/site-packages/pkg_resources/__init__.py in get_distribution(dist) 560 dist = Requirement.parse(dist) 561 if isinstance(dist, Requirement): --> 562 dist = get_provider(dist) 563 if not isinstance(dist, Distribution): 564 raise TypeError("Expected string, Requirement, or Distribution", dist) ~/.pyenv/versions/3.6.1/envs/jupyter3_install/lib/python3.6/site-packages/pkg_resources/__init__.py in get_provider(moduleOrReq) 434 """Return an IResourceProvider for the named module or requirement""" 435 if isinstance(moduleOrReq, Requirement): --> 436 return working_set.find(moduleOrReq) or require(str(moduleOrReq)) 437 try: 438 module = sys.modules[moduleOrReq] ~/.pyenv/versions/3.6.1/envs/jupyter3_install/lib/python3.6/site-packages/pkg_resources/__init__.py in require(self, *requirements) 979 included, even if they were already activated in this working set. 980 """ --> 981 needed = self.resolve(parse_requirements(requirements)) 982 983 for dist in needed: ~/.pyenv/versions/3.6.1/envs/jupyter3_install/lib/python3.6/site-packages/pkg_resources/__init__.py in resolve(self, requirements, env, installer, replace_conflicting, extras) 865 if dist is None: 866 requirers = required_by.get(req, None) --> 867 raise DistributionNotFound(req, requirers) 868 to_activate.append(dist) 869 if dist not in req: DistributionNotFound: The 'numpy' distribution was not found and is required by the application
It turned out that by the time that the script in this gist was called,
pkg_resources had already been imported (at least once) before.
Its documentation states that:
Add a path item to the entries, finding any distributions on it. You should use this when you add additional items to sys.path and you want the global working_set to reflect the change. This method is also called by the WorkingSet() constructor during initialization.
This method uses find_distributions(entry,True) to find distributions corresponding to the path entry, and then add() them. entry is always appended to the entries attribute, even if it is already present, however. (This is because sys.path can contain the same value more than once, and the entries attribute should be able to reflect this.)
Thus, the solution that I found (not sure how "correct" it is...) is to:
import pkg_resources at the top, then add:
along with the
sys.path.insert statements at the end.
Nov 19, 2018
When using this script, I found that it ignored packages that were installed in 'develop' mode via
python setup.py develop or
pip install -e .. It seems like when something is installed in develop mode, an egg-link is added to the site-packages which points to the active package directory. I'm not sure how this egg-link is resolved normally in python, but at some point that directory is directly added to
sys.path. To mimic this behavior, I added the following anywhere after the variable
site_packages is defined:
egg_links = [i for i in os.listdir(site_packages) if 'egg-link' in i] for egg in egg_links: with open(os.path.join(site_packages,egg), 'r') as fin: sys.path.insert(0, fin.readline().rstrip('\n'))
This simply looks for anything that has 'egg-link' in the name, then opens the file and adds the first line to the path. No idea if you could have an edge case in which there could be multiple paths or lines in the egg-link, but this works for my needs.
I don't know if there's a way to send pull requests for a gist. Anyway, there's a suggestion below.
Keeping the default Python behavior to search for modules in current directory first, change lines 34 and 35 to this:
It allows local custom modules override their version even inside the virtualenv.
Good work. Keep walking.