zahlman/question.md Secret

## question.md

      
    Raw
  

              question.md
            
          
    Why can't I use the third-party library I just installed using pip? Why do I get an ImportError or ModuleNotFoundError when pip says it's there?

I installed the package-installation-test package from PyPI, using the command pip install package-installation-test. I know (from reading the documentation, both on PyPI and the project's GitHub repo) that this should make it possible to import example_package, for example, from the interpreter prompt.
But when I try it, I get an error:
>>> import example_package
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'example_package'

Furthermore, I can't seem to run the command-line tool that the package promised to install:
$ demo-example-package
demo-example-package: command not found

But I know that the package is installed because pip will complain if I try to install it again:
$ pip install package-installation-test
Requirement already satisfied: package-installation-test in /path/to/python/site-packages (1.0.0)

and also because pip list shows the installed package, with the expected version number:
$ pip list | grep example
example-package           1.0.0

I did some research and I don't think this problem is specific to my operating system or to any Python version, but it seems like it might have something to do with having multiple copies of Python on my computer. I also saw some advice about using "virtual environments" or "venvs" to manage the installations, but that seems counterintuitive. If the problem is because of multiple copies of Python, why would I want more? Shouldn't I be trying to remove some instead?
Anyway, how exactly does this work, and how can I solve the problem?

Dealing with third-party libraries and multiple Python installations

Many people have more than one installation of Python on their computer. While this is sometimes unintentional, there are many practical benefits to doing it intentionally. Most installations of Python provide their own copy of pip, which is a command-line tool for installing third-party libraries.
Before we get into the details of troubleshooting problems like this, there are two very important caveats:


If a copy of Python came with your computer, or with an operating system that you installed, do not ever attempt to uninstall that Python. Ideally, do not try to install anything to it, either.

Many non-Windows operating systems have key components implemented as Python scripts, and they may require a specific version of Python to work properly. Messing with this installation  can seriously damage your operating system - the Internet is full of horror stories. This was an especially big problem in the late 2010s, with operating systems dependent on Python 2.7 and users naively trying to get rid of it to avoid accidentally running their 3.x code on a 2.x interpreter.

To avoid this problem, some Linux distributions may deliberately exclude pip from the system Python. Please do not try to install pip for a Python that came with your system, even if the operating system helpfully offers the option. Just use virtual environments (venvs), as explained further down. That way, if a malicious package is installed accidentally, it at least won't have an opportunity to damage the operating system.


Do not ever use sudo on Linux/Mac to try to solve an installation problem, without an extremely clear, good reason that you are able to explain by yourself.

If a guide suggests using sudo for the pip program - whether for troubleshooting or for the normal installation of a specific package - please disregard it. If everyone talking about a specific package says to use sudo to install it, consider whether the package might be malicious.

The only reason pip would need sudo is in order to allow installing to the system Python (but this is a bad idea, as already established) or because using sudo would alter the PATH (but there are much better ways to do that).
If for some reason it is necessary to install to a system Python, at least strongly consider the --user option for pip.


How does pip actually work?

pip is not an ordinary program. It is a Python package, which comes with a wrapper executable also named pip. (There may also be a wrapper executable named pip3, or even one named after the Python minor version number.)
On Linux, the wrapper is a .py file with a shebang line and an executable bit set, which adjusts the command-line arguments, imports the package and runs the appropriate code. The wrapper will be in the same folder as python itself.
On Windows, the wrapper is a stub .exe that loads a Python DLL and uses that to run the appropriate code. It will be in a related folder.
Each installation of Python may have the pip package installed. The corresponding wrapper will hard-code which Python to use, and packages will then be installed for that Python.
Which pip am I using? Which Python does that correspond to?

On Mac and Linux, which pip will tell you the path to the pip that will run using the pip command. For Windows, please see https://stackoverflow.com/questions/304319. Because of the path relationships described above, this also tells us where the corresponding python executable is.
Of course, it is always possible to override this by providing a path explicitly.
Why isn't that the python that's running?

Normally, running python at the command line should ensure that the same Python is used as the one that runs for pip. Some things that can go wrong:


It's common on Windows systems for python.exe not to be on the PATH at all, because the installer offers that option. The idea is that py (see below) can manage Python installations, so the PATH setting isn't necessary. However, this comes with a few drawbacks.


Because Windows doesn't put python.exe and pip.exe (i.e., the wrapper) in the same folder, it's possible to configure the PATH so that one of these is included and the other isn't.


It's possible to configure systems so that python will only run a 2.x version, or only run a 3.x version. (Sometimes this is combined with separate python2 and/or python3 commands.) Say for example that a 2.x installation and 3.x installation are both listed on the PATH, but python is configured to run only 3.x, while the 2.x installation appears first. It could happen that python would run the 3.x installation, but pip would install for the 2.x installation.


What about if I use py (on Windows) rather than python?

The Python Launcher for Windows (py.exe) can be installed along with any Python installation. Only one copy is needed per system, and it's installed to the same place: directly into the Windows folder. The idea is that this will always be on the PATH, and therefore always findable. When py is used to run a Python script, it will attempt to choose a Python installation on the computer based on command-line arguments and/or the script's shebang line. Thus, by associating the .py file extension with py.exe, Windows users can double-click scripts and have them run with the correct Python - even though Windows lacks the Unix concept of executable text files and doesn't natively assign any meaning to shebang lines.
This works well for many users, but it isn't perfect. Old versions of py (installed along with 3.3 or 3.4) might not know about virtual environments; and the pip wrapper executable is an actual stub .exe, so it won't use py's logic to choose a Python.
What about the Python used by my IDE?

Two options:


Configure the IDE so that it uses the same Python that was used for pip.


Use the IDE to run the corresponding pip (equivalently, to run-as-a-module the pip package for the Python it's using).


IDE-specific instructions will be covered in separate answers.
What about other Python implementations, such as PyPy, Anaconda and IronPython?

These will be covered in a separate answer.
What is pip3? How does it relate to the problem?

Exactly what it sounds like: another wrapper for the pip package, but one which is only included in 3.x Python installations. This makes it possible to configure the system so that pip installs to one specific 2.x installation, and pip3 installs to one specific 3.x installation.
However, the other management techniques in this answer are much more foolproof. pip3 isn't really necessary.
What is a virtual environment (venv)? How does it help solve the problem?

A virtual environment is functionally a separate installation of Python that is designed to swap in for what python finds at the command line.
How can I be absolutely sure that pip and Python match up in my terminal?

Two options:


Explicitly use Python to run pip: python -m pip (or py -m pip, where applicable). The -m flag tells Python to run the package as a module. This way bypasses the wrapper script entirely; the pip code will be found directly using Python's module search path, rather than the system PATH environment variable.


Set up and activate a virtual environment first. This will automatically reconfigure the PATH environment variable for as long as the virtual environment is active, such that python means the Python in the venv and pip installs for that Python as well. (Of course, it is possible to break this intentionally.) This does not impact py on Windows.


How do I fix command-line tools (executable wrapper scripts)?

If the tool doesn't run (and its name doesn't tab-complete on Linux), it's because the path to the tool isn't on the PATH. These tools are wrapper executables that will be put into the same folder as the pip wrapper executable, and which work the same way.
It's possible to modify PATH explicitly to include the folder that contains the wrappers. It's also possible to specify this path explicitly. However, if the package was installed to a venv, simply activating the venv will make the necessary change to PATH. This approach is recommended as it is simple and foolproof.