Skip to content

Instantly share code, notes, and snippets.

@MichaelCurrin
Last active November 1, 2024 11:18
Show Gist options
  • Save MichaelCurrin/3a4d14ba1763b4d6a1884f56a01412b7 to your computer and use it in GitHub Desktop.
Save MichaelCurrin/3a4d14ba1763b4d6a1884f56a01412b7 to your computer and use it in GitHub Desktop.
Set up Python 3 and new virtual environment

Set up Python 3 and new virtual environment

A beginner's guide to installing Python 3 and setting up a virtual environment

This guide covers how to install and upgrade Python 3 and how to create and an install into a Python virtual environment.

This is best practice in Python for both local and production code, as it isolates the scope where your python commands and pip commands run, protecting your global environment and allowing your to manage multiple virtual environments each with their own set of unique Python packages.

TL;DR

A summary of steps you'll learn about in this guide:

  • How to install Python in macOS, Linux or Windows.
  • Create virtual env.
    $ python3 -m venv venv
  • Activate virtual env.
    $ # Linux/macOS
    $ source venv/bin/activate  
    $ # Windows
    $ venv\Scripts\activate    
  • Install packages in the virtual env.
    (venv) $ pip install package-name
    (venv) $ pip install -r requirements.txt

Table of Contents

  1. Check Python version
  2. Setup Python
  3. Setup virtual environment

1. Check Python Version

If you already have Python installed, you can check the version you have. Or you'll get a message saying the command could not be found.

$ python --version

If Python 2 is your system's default, you'll have to specify Python 3 instead.

$ python3 --version

2. Set up Python

If you have a Unix-like system (Mac or Linux), you probably already have Python installed maybe not a new version of Python 3. Then skip to Upgrade Python below.

Install Python

Follow instructions below to update the package manager then install the package.

For more detailed instructions or how to install for Windows or a Linux OS not listed below, I recommended reading this Real Python - Installing Python article. See also the Install Python 3 on a Mac article.

Mac OS

$ brew install python3

Or more specifically, use a formula like python@3.9.

$ brew install python@3.9

Linux

Ubuntu/Debian
$ sudo apt-get update
$ sudo apt-get install python3
Fedora

From the Fedora Project article, which also covers using multiple Python versions and virtual environments.

$ sudo dnf install python39  # 3.9
CentOS
$ sudo yum update
  • From Real Python.
    $ sudo yum install yum-utils
    $ sudo yum install https://centos7.iuscommunity.org/ius-release.rpm
  • From the article on How to install Python3 on CentOS 7.
    $ sudo yum install centos-release-scl
    $ sudo yum install rh-python39  # 3.9
ArchLinux

From Real Python.

$ packman -S python

Upgrade Python

Proceed with caution. You may break your machine's environment (some programs may stop working), or make your existing virtual environments point to an inactive minor version of Python (this happened to me on a Mac).

Mac OS

$ brew upgrade python3

Linux

Ubuntu/Debian
$ sudo apt-get update && sudo apt-get upgrade python3

Note that for Debian, the major versions are locked for a Debian version. Therefore you should upgrade to a newer Debian version to get a newer Python version. Or install it from a different repository or from source, but this is a custom installation which may be hard to get working or maintain, so I don't recommend it. For instance, when you install C headers or choose a Python version for your IDE, you system will not by default know to look in the custom location you've installed Python to.

3. Setup virtual environment

How to create a virtual environment and install python packages into it. Only Unix-based systems are covered here.

Note that recent versions of Python 2 and 3 include pip already. And while the virtualenv library works for Python 2 and 3, I've read that venv is the recommended choice (it is actually a builtin for Python 3). So I use that here.

Create

This guide uses Python's venv tool to manage virtual environments. When you create a virtual environment, it is setup to use a specific version of Python - you need to recreate it if you want to upgrade it.

Examples:

  • Use system default for PY 3.
    $ python3 -m venv <NEW_ENV_NAME> 
  • Specify target version, provided it is installed.
    $ python3.7 -m venv <NEW_ENV_NAME>
    $ python3.6 -m venv <NEW_ENV_NAME>
  • For any old projects where you really need use Python 2.
    $ python2 -m venv <NEW_ENV_NAME>

Avoid using just python without a version number as you might get PY 2 or 3.

Note: The best tool for managing Python virtual environments is debateable. In short, virtualenv tool was popular but venv tool is supposed to be preferred over it. In fact, venv is now standard in Python3. Alternatives are pipenv and Virtual Env Wrapper.

Follow the steps below to add a virtual environment to your project directory under the name venv. With the first option that will be a directory and with the second option that will be a symbolic link to a uniquely name virtual environment in a chose central location.

The first option is simpler, however, it's is less maintainable since if you delete the project directory (or a repo), you'll also lose the venv. And if you move/rename the project, the links in the venv will break (though, there are special steps needed to make a venv directory relocatable).

Either way, make sure to update your project so the directory is excluded from version control.

  • .gitignore
    __pycache__/
    
    venv
    

Option A - Create inside the project

How to install a virtual environment in the same directory as your project or repo.

$ cd <PATH/TO/PROJECT>
$ python3 -m venv venv

That used the module flag to run the "venv" python module. It created a directory called "venv", though you could call it "virtualenv" or anything you like.

Option B - Create inside a central directory

Instructions for Linux and macOS

Choose a location for install all your virtual envs across projects, then create symlinks (symbolic links to them as needed.

I've seen some people use the ~/.virtualenvs directory. However, this is already used sometimes such as if virtualenvwrapper is installed. I prefer to create a different directory that I saw recommended.

$ mkdir -p ~/.local/virtualenvs
$ cd ~/.local/virtualenvs
$ python3 -m venv my-project-name
$ cd <PATH/TO/PROJECT>

Create soft (-s) symbolic link (ln) to the virtual environment directory. Note that if you move or the delete the directory, this link will no longer be valid.

$ ln -s ~/.local/virtualenvs/my-project-name venv

Check the symlink.

$ ls -l venv
lrwxr-xr-x   1 ...  ....    47B  7 Feb 19:33 venv -> <USER_DIR>/.local/virtualenvs/my-project-name/

If you ever need to point the symlink file to a different path, run this to delete and create the symlink. If you omit the force flag, you'll get an error that the file already exists.

$ ln -sf ~/.local/virtualenvs/my-project-name venv

Install into the environment

Within the project directory, activate the environment. You'll need to do this each time you want to work on the project. Your IDE can be configured to recognize it for your automatically.

  • Linux or macOS
    $ source venv/bin/activate
  • Windows
    > venv\Scripts\activate

Note how the prefix changed to show active status.

Upgrade pip.

$ pip install pip --upgrade

Install Python packages from project requirements file.

$ pip install -r requirements.txt

Install a Python package by name. Here are some examples.

$ pip install pandas
$ pip install requests==22.0.2

Check scope

Check to see how an activated virtual environment isolates your commands:

(venv) $ which python
<USER_DIR>/.local/virtualenvs/my-project-name/bin/python
(venv) $ which pip
<USER_DIR>/.local/virtualenvs/my-project-name/bin/pip

Deactivate

Use deactivate to go back to your global environment and see your prompt change. But, install outside an virtual environment with caution, as conflicts can arise from different projects.

(venv) $ deactivate
$

Notes

The reason this exists here as a gist is to make is easy to maintain a single set of Python setup instructions which I can link to from many projects. This can be embeded in Jekyll site using the jekyll-gist plugin and {% gist 3a4d14ba1763b4d6a1884f56a01412b7 %}.

@darkato42
Copy link

darkato42 commented May 14, 2022

This gist is fantastic and well maintained! Many thanks!

I've opted for the symbolic link option. and by adding a function to my ~/.zshrc. I was able to quickly create a venv in the same directory with pyve my-project, and run source venv/bin/activate/ afterwards.

# Add a python venv with symbolic link
function pyve {
    mkdir -p ~/.local/virtualenvs
    python3 -m venv ~/.local/virtualenvs/$1
    ln -s ~/.local/virtualenvs/$1 venv
}

It's working fine apart from Pylance in VS Code showing the modules as not resolved. Not sure how to use Select Python interpreter and allow Pylance find those modules. Any ideas?

@MichaelCurrin
Copy link
Author

MichaelCurrin commented May 20, 2022

Neat. You can add -f to ln to force creating the symlink even it exists. So if you changed it, then you effectively delete it and make the new one. And if its identical then you no changes and still get a success status.

I also like to install and upgrade basic packages inside an new env

echo 'Upgrading pip version'
venv/bin/pip install pip --upgrade --force

echo 'Installing setuptools and wheel'
venv/bin/pip install setuptools wheel --upgrade

--

Regarding VS Code, I set my Python interpreter in a project to point to venv in the repo root. It doesn't matter whether that is a directory or a symlink to a directory, it will work the same.

https://github.com/MichaelCurrin/py-project-template/blob/cfc6be115f1c2d7b71caf93865c9b9134f7a160f/.vscode/settings.json#L7

See also

https://github.com/MichaelCurrin/py-project-template/blob/master/.env#L5

@SUVENDUC789
Copy link

Hello

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment