Changing python interpreter from within a virtual env

changing-virtual-env-interpreter.md

I isolate dependencies using python virtual environments. The base system's os python version can be leaverage inside of these environments when tools clash. Like for certbot or ansible password files where the base configurations doesn't change. But not to muddy the waters here's exactly what I mean. The azure.azcollection for ansible has a requirements.txt file. They requirements are only necessary for one of our ansible repos. There are clashing dependencies between this collection and our ansible vault_pass file. So how do you have the vault_pass file called by ansible from within a virtual environment utilise a different version of ansible. The answer is not as simple as you might think, or at least not as simple as I first thought.

If I setup a virtual environment the calling interpreter of ansible (the interpreter of my environment) will call the /usr/bin/az command, which is fine, if it has the dependencies in PATH to run it, which it doesn't.

# ansible.cfg
vault_password_file     = vault_pass

and create a vault_pass file that pulls secrets from azure

#!/usr/bin/env bash
/usr/bin/az keyvault secret show --name <secret-name> --vault-name <vault-name> | jq -r ".value"

If I edit the /usr/bin/az command with the following python -c line

#!/usr/bin/env bash
python -c "import sys;print(sys.version)"
bin_dir=`cd "$(dirname "$BASH_SOURCE[0]")"; pwd`
AZ_INSTALLER=RPM PYTHONPATH="$bin_dir"/../lib64/az/lib/python3.6/site-packages python3 -sm azure.cli "$@"

and run the vault_pass file outside of ansible from the shell but from within an active virtual environment named "ansible" I get

(ansible) [b0bu@host ansible]$ python -V
Python 3.9.2
(ansible) [b0bu@host ansible]$ ansible --version
ansible [core 2.11.0]
  config file = /home/b0bu/dir/ansible/ansible.cfg
  configured module search path = ['/usr/share/ansible']
  ansible python module location = /home/b0bu/.pyenv/versions/3.9.2/envs/ansible/lib/python3.9/site-packages/ansible
  ansible collection location = /home/b0bu/.ansible/collections:/usr/share/ansible/collections
  executable location = /home/b0bu/.pyenv/versions/ansible/bin/ansible
  python version = 3.9.2 (default, Mar  2 2021, 17:23:39) [GCC 4.8.5 20150623 (Red Hat 4.8.5-44)]
  jinja version = 3.0.1
  libyaml = True

I'm clearly in a virtual env called "ansible" and I'm clearly using python 3.9.2 and when I run the vault pass file I get

(ansible) [user@host ansible]$ ./vault_pass
/home/b0bu/.pyenv/versions/ansible/bin/python3: No module named azure.cli.__main__; 'azure.cli' is a package and cannot be directly executed

By appending PYENV_VERSION=system to the command I can force the command to run with a specific python version where the dependencies for az are installed via yum and managed system-wide.

(ansible) [b0bu@host ansible]$ ./vault_pass
2.7.5 (default, Nov 16 2020, 22:23:17)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-44)]
<secret-output>

But what happens when I run ansible?

(ansible) [b0bu@host ansible]$ ansible-playbook playbook.yml -i inventory/
3.9.2 (default, Mar  2 2021, 17:23:39)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-44)]
/home/b0bu/.pyenv/versions/ansible/bin/python3: No module named azure.cli.__main__; 'azure.cli' is a package and cannot be directly executed

So PYENV_VERSION=system isn't honored and the vault_pass file is shared scope it's used in multiple pipelines so there's no need to set it up as a dependency per pipeline, especially if you're going to do things that'd break it. The answer is you set PATH in the vault_pass file. Now when you run ansible, you're pass file won't inherit the interpreter of the virtual env.

PATH=/usr/lib/python2.7/site-packages:/usr/bin/
/usr/bin/az keyvault secret show --name <secret-name> --vault-name <vault-name> | jq -r ".value"