Skip to content

Instantly share code, notes, and snippets.

@ramsey

ramsey/pip-cache-notes.md

Last active April 4, 2017 20:30
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save ramsey/37485c99aedf990bb42515c3b27ef542 to your computer and use it in GitHub Desktop.
Save ramsey/37485c99aedf990bb42515c3b27ef542 to your computer and use it in GitHub Desktop.
Download ZIP
pip cache issue
Raw
pip-cache-notes.md

Thinking this to be a powerline issue related to powerline/powerline#1494, I originally posted the following at powerline/powerline#1494 (comment)

Thanks so much to Trent Harvey and @ZyX-I for their help working through this. (Twitter thread.)

I just encountered this problem. I'm not sure what causes it, since I've been using my terminal configuration for years without this issue. I did not install or upgrade any software today. Rebooting my machine did not fix it.

I'm running:

  • Mac OS X 10.11.6
  • iTerm2, but I also experience this issue with Apple Terminal
  • zsh 5.2 (Homebrew version)
  • powerline-status 2.3.dev9999+git.d611cc2e439e02a123b4f208767c8dc5f72ce567

My shell configuration is available here: https://github.com/ramsey/dotfiles

I did noticed two things that I can do to work around the issue:

  • Turn off wi-fi, launch terminal; prompt successfully appears, and I can turn wi-fi back on, or
  • Comment out the following line in my ~/.zsh/config file:
export POWERLINE_LOCATION=`pip show powerline-status | grep "^Location" | awk '{ print $2 }'`

The problem appears to be a network-related issue with pip. For example, when I type pip show powerline-status, I see:

$ pip show powerline-status
Name: powerline-status
Version: 2.3.dev9999+git.d611cc2e439e02a123b4f208767c8dc5f72ce567
Summary: The ultimate statusline/prompt utility.
Home-page: https://github.com/powerline/powerline
Author: Kim Silkebaekken
Author-email: kim.silkebaekken+vim@gmail.com
License: MIT
Location: /usr/local/lib/python2.7/site-packages
Requires:

And then it just hangs at that point. When I Ctrl+c to cancel, I receive the following backtrace:

Python pip backtrace (click to expand) 
Traceback (most recent call last):
  File "/usr/local/bin/pip", line 11, in <module>
    load_entry_point('pip==9.0.1', 'console_scripts', 'pip')()
  File "/usr/local/lib/python2.7/site-packages/pip/__init__.py", line 233, in main
    return command.main(cmd_args)
  File "/usr/local/lib/python2.7/site-packages/pip/basecommand.py", line 252, in main
    pip_version_check(session)
  File "/usr/local/lib/python2.7/site-packages/pip/utils/outdated.py", line 126, in pip_version_check
    headers={"Accept": "application/json"},
  File "/usr/local/lib/python2.7/site-packages/pip/_vendor/requests/sessions.py", line 488, in get
    return self.request('GET', url, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/pip/download.py", line 386, in request
    return super(PipSession, self).request(method, url, *args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/pip/_vendor/requests/sessions.py", line 475, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python2.7/site-packages/pip/_vendor/requests/sessions.py", line 628, in send
    r.content
  File "/usr/local/lib/python2.7/site-packages/pip/_vendor/requests/models.py", line 755, in content
    self._content = bytes().join(self.iter_content(CONTENT_CHUNK_SIZE)) or bytes()
  File "/usr/local/lib/python2.7/site-packages/pip/_vendor/requests/models.py", line 676, in generate
    for chunk in self.raw.stream(chunk_size, decode_content=True):
  File "/usr/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/response.py", line 357, in stream
    data = self.read(amt=amt, decode_content=decode_content)
  File "/usr/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/response.py", line 314, in read
    data = self._fp.read(amt)
  File "/usr/local/lib/python2.7/site-packages/pip/_vendor/cachecontrol/filewrapper.py", line 63, in read
    self._close()
  File "/usr/local/lib/python2.7/site-packages/pip/_vendor/cachecontrol/filewrapper.py", line 50, in _close
    self.__callback(self.__buf.getvalue())
  File "/usr/local/lib/python2.7/site-packages/pip/_vendor/cachecontrol/controller.py", line 297, in cache_response
    self.serializer.dumps(request, response, body=body),
  File "/usr/local/lib/python2.7/site-packages/pip/download.py", line 289, in set
    return super(SafeFileCache, self).set(*args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/pip/_vendor/cachecontrol/caches/file_cache.py", line 99, in set
    with self.lock_class(name) as lock:
  File "/usr/local/lib/python2.7/site-packages/pip/_vendor/lockfile/__init__.py", line 197, in __enter__
    self.acquire()
  File "/usr/local/lib/python2.7/site-packages/pip/_vendor/lockfile/mkdirlockfile.py", line 57, in acquire
    time.sleep(wait)
KeyboardInterrupt

This may not be a problem with powerline at all but, rather, with some pip networking issue. It's frustrating, though.

Curiouser and curiouser... If I switch to any virtualenv, pip works fine there, but when not in a virtualenv, pip hangs as described.

Using --disable-pip-version-check does the trick. Now, if I can just figure out what's going on with pip... 😄

A few more details, in case this helps others who find this issue on similar searches...

This issue appears to be related to caching, not network connections, as I originally thought. When I use -vvv, I can see that the command hangs when it tries caching the response:

Output of pip show -vvv powerline-status 
Name: powerline-status
Version: 2.3.dev9999+git.d611cc2e439e02a123b4f208767c8dc5f72ce567
Summary: The ultimate statusline/prompt utility.
Home-page: https://github.com/powerline/powerline
Author: Kim Silkebaekken
Author-email: kim.silkebaekken+vim@gmail.com
License: MIT
Location: /usr/local/lib/python2.7/site-packages
Requires:
Metadata-Version: 1.1
Installer:
Classifiers:
  Development Status :: 4 - Beta
  Environment :: Console
  Environment :: Plugins
  Intended Audience :: End Users/Desktop
  License :: OSI Approved :: MIT License
  Natural Language :: English
  Operating System :: Microsoft :: Windows
  Operating System :: POSIX
  Programming Language :: Python :: 2.6
  Programming Language :: Python :: 2.7
  Programming Language :: Python :: 3.2
  Programming Language :: Python :: 3.3
  Programming Language :: Python :: 3.4
  Programming Language :: Python :: Implementation :: CPython
  Programming Language :: Python :: Implementation :: PyPy
Entry-points:
Looking up "https://pypi.python.org/pypi/pip/json" in the cache
No cache entry available
Starting new HTTPS connection (1): pypi.python.org
"GET /pypi/pip/json HTTP/1.1" 200 72986
Updating cache with response from "https://pypi.python.org/pypi/pip/json"
Caching b/c date exists and max-age > 0

Running the same command with --no-cache-dir succeeds without any problems. This issue appears related: pypa/pip#3245

It seems that, by clearing the cache directory, you can solve this problem. See also: http://stackoverflow.com/a/31807659/446932

Running rm -rf ~/Library/Caches/pip did the trick for me on OS X. Now, I can run pip commands without --no-cache-dir or --disable-pip-version-check.

To speed things up on launching terminal sessions, I've modified my POWERLINE_LOCATION line to:

export POWERLINE_LOCATION=`pip show --disable-pip-version-check --no-cache-dir powerline-status | grep "^Location" | awk '{ print $2 }'`

Best of luck!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment