Skip to content

Instantly share code, notes, and snippets.

@MuhsinFatih
Last active March 26, 2024 16:02
Star You must be signed in to star a gist
Save MuhsinFatih/ee0154199803babb449b5bb98d3475f7 to your computer and use it in GitHub Desktop.
How to recover from messed up python installation on mac, and never have to mess with apple's shitty python confusion factory

I am assuming you are here because like me, you installed a bazillion different python interpreters on mac and the whole thing is a spagetti. Today, I finally fixed my python installation. Whatever I install for python2 or python3 using pip JUST.WORKS.. My god! finally.

What the hell?

Here is what I had messed up, which you also probably did:

  • I had too many different python interpreters
  • Too many different symlinks which I lost track of
  • almost no package I installed with pip worked without a headache
  • any attempt to fix using online resources made it worse.

Let's set things right

  • First, remove all the python packages you installed. I am not sure if this is really necessary, but I did it just so I get a fresh start
pip uninstall -y -r <(pip freeze)

(source: https://stackoverflow.com/a/49200334/2770195)

  • Also remove pip cache because why not:
~/Library/Caches/pip
  • Now remove pip
pip uninstall pip
  • Remove all the third party python installations: Careful! Don't delete anything under the /System/ folder. Otherwise you WILL have to reinstall macos, it WILL crash, don't do it.
sudo rm -rf /Library/Frameworks/Python.framework
sudo rm -rf "/Applications/Python 2.7"
cd /usr/local/bin/
ls -l /usr/local/bin | grep '../Library/Frameworks/Python.framework/Versions/2.7' | awk '{print $9}' | tr -d @ | xargs rm

(source: https://stackoverflow.com/a/3819829/2770195 but I changed it a bit to remove python3 too)

  • Remove python path stuff
    Edit whatever shell profile you have, like .bash_profile or .zshrc and .zprofile or whichever shell you are using (If you don't know what I'm talking about then it's .bash_profile):
    Delete all lines that have Python.framework in it. In other words, delete old paths related to python.
nano .bash_profile

(source: https://stackoverflow.com/a/3819829/2770195 again)

  • If you have anaconda installed, get rid of that too:
rm -rf ~/anaconda*

(source: https://docs.anaconda.com/anaconda/install/uninstall/)

So far all this was standard. What's important is how you install the new version. Here's the fun part: if you already messed up your python environment, then there is a good chance you messed up permissions with some packages too. Thus any attempt to install new python versions will seem to work fine for now, but will be a brat later on. That's why the next step is so important:

  • Fix old package permissions:
sudo chown -R $USER /usr/local/bin/*

Basically, everything in this folder is supposed to NOT require root priviliges, which if they do they give pip a hard time later on. That's why we made everything belong to us

Let's install the one and only python2 we will use!:

(don't worry we will get to python3 as well)
Now that you fixed the permissions, you can go ahead and install a python2 interpreter that will NOT ever require sudo with pip and hence will install your packages just fine: Here is the trick: We are NOT using easy_install. Got it? No easy_install. It's the devil. brew is love, use brew:

  • If you don't have brew installed (firstly you should be ashamed of yourself):
/usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

Go to your .profile file again

nano .bash_profile

and add these lines:

⚠ (If you're using macos on arm, add these two lines instead):

export PATH="/usr/local/bin:/usr/local/sbin:$PATH"
export PATH="/opt/homebrew/opt/python@2/libexec/bin:$PATH"
export PATH="/usr/local/bin:/usr/local/sbin:$PATH"
export PATH="/usr/local/opt/python@2/libexec/bin:$PATH"

[UPDATE: crossed python2 installation as it is no longer relevant]
now install python2 using brew: brew install python@2 (this will also install pip)
(source: https://docs.python-guide.org/starting/install/osx/)

Install python3

brew install python

(this installs latests python, which is right now python3. Also installs pip)

Ok, if you haven't screwed up anything so far, you should be able to use pip with a peace of mind. I just tried a bunch of packages like numpy and plotly they all worked right away

Golden rule

If you want to maintain the serenity that you have just gained, you have to remember one rule: Never ever use sudo pip install. sudo is the biggest source of confusion with python packages as it screws up the permissions. This is not only true for python, but also nodejs for example, and other environments. There are a lot of posts online that use sudo pip to install packages in their instructions. Always omit the sudo part of that instruction. If you ever accidentally use sudo pip, come back here and apply the steps related to permissions again

gg wp. Now do me a favor and end aging before I die. Just saying. Have fun.

@svahora
Copy link

svahora commented May 6, 2022

I'm trying to fix my install as well. Most of the time when I'm using VS Code to install packages it gives me an error and I end up spending hours trying to fix it all just to install a simple python library. Afterwards I usually give up :(
The following is what I have for those queries requested to others:

SV$ which python
/usr/bin/python
SV$ which python2
/usr/bin/python2
SV$ which python3
/usr/bin/python3
SV$ which pip
/usr/local/bin/pip
SV$ which pip2
/
SV$ which pip3
/usr/bin/pip3
SV$ pip2 -V
-bash: pip2: command not found
SV$ pip3 -V
pip 19.0.3 from /Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/site-packages/pip (python 3.7)
SV$ pip -V
-bash: /usr/local/bin/pip: /usr/local/opt/python/bin/python3.7: bad interpreter: No such file or directory

@AllPinkyNoBrain
Copy link

thankyou thankyou thankyou thankyou !!!

I was going mad with all this (and I'd stupidly thrown in an upgrade to Monterey in the midst of my madness). This has made my day!
:-)

@crukundo
Copy link

Just incase anyone finds this in 2022, and can't install python2 with homebrew, some of the threads here (https://stackoverflow.com/questions/60298514/how-to-reinstall-python2-from-homebrew) might help. I used pyenv and reinstalled pip independently on python2 and 3

@Tyler-Pickett
Copy link

After running pip uninstall -y -r <(pip freeze)

I get this error code, everything up to gets deleted. PermissionError: [Errno 13] Permission denied: '/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/site-packages/__pycache__/six.cpython-37.pyc'

Even though I am admin, I come up as staff when I run ls -la

@dasheck0
Copy link

dasheck0 commented Oct 3, 2022

Thought my system was beyond repair. This actually fixed it. Thank you so much!

@gpmilliken
Copy link

Thanks - got some traceback errors along the way but this worked and the madness of installed modules not being found has stopped.

Thank you.

@cwilby
Copy link

cwilby commented Nov 21, 2022

Some considerations for macOS on ARM:

and add these lines:

export PATH="/usr/local/bin:/usr/local/sbin:$PATH"
export PATH="/usr/local/opt/python@2/libexec/bin:$PATH"

translates as

and add these lines:

export PATH="/usr/local/bin:/usr/local/sbin:$PATH"
export PATH="/opt/homebrew/opt/python@2/libexec/bin:$PATH"

@MuhsinFatih
Copy link
Author

Thanks @cwilby, I just checked and the path is as you said on my mac (arm). Updated the gist

@armouti
Copy link

armouti commented Jan 23, 2023

this is outdated
brew install python@2 will return
Error: No available formula with the name "python@2"
because it has been removed by homebrew Homebrew/homebrew-core#49796

@MuhsinFatih
Copy link
Author

this is outdated brew install python@2 will return Error: No available formula with the name "python@2" because it has been removed by homebrew Homebrew/homebrew-core#49796

@armouti replace that with just python and you should be good. Noone uses python2 anymore

@yin-ori
Copy link

yin-ori commented Feb 10, 2023

I installed pyenv virtualvenv after I had to completely resetup my setup. which pip/pip3/python/python3 all direct to the correct environment path, though, once I try to install packages it will still create packages in a path of homebrew library for python3.10 (which I don't even have anymore) - I am not sure if its something wrong along with my .zshcr zprofile bash or something else and cannot find anything that works for me on the internet either. $PATH will give me
zsh: no such file or directory: /Users/MYUSERNAME/.pyenv/shims:/Users/MYUSERNAME/.pyenv/bin:/opt/homebrew/bin:/opt/homebrew/sbin:/usr/local/bin:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/TeX/texbin:/usr/local/go/bin:/Library/Apple/usr/bin

see also this: python packages installed to wrong path

btw even if I delete the folder it will recreate itself upon installation :(

@UncleBob2
Copy link

Thank you so much for sharing your knowledge. I had lot of issues with "pip install TTS" and was getting lots of warnings

WARNING: Skipping /usr/local/lib/python3.11/site-packages/PyYAML-6.0-py3.11.egg-info due to invalid metadata entry 'name'
WARNING: Skipping /usr/local/lib/python3.11/site-packages/numpy-1.24.2-py3.11.egg-info due to invalid metadata entry 'name'
WARNING: Skipping /usr/local/lib/python3.11/site-packages/PyYAML-6.0-py3.11.egg-info due to invalid metadata entry 'name'
WARNING: Skipping /usr/local/lib/python3.11/site-packages/numpy-1.24.2-py3.11.egg-info due to invalid metadata entry 'name'
WARNING: Skipping /usr/local/lib/python3.11/site-packages/PyYAML-6.0-py3.11.egg-info due to invalid metadata entry 'name'
WARNING: Skipping /usr/local/lib/python3.11/site-packages/numpy-1.24.2-py3.11.egg-info due to invalid metadata entry 'name'"
WARNING: Skipping /usr/local/lib/python3.11/site-packages/PyYAML-6.0-py3.11.egg-info due to invalid metadata entry 'name'
WARNING: Skipping /usr/local/lib/python3.11/site-packages/numpy-1.24.2-py3.11.egg-info due to invalid metadata entry 'name'
WARNING: Skipping /usr/local/lib/python3.11/site-packages/PyYAML-6.0-py3.11.egg-info due to invalid metadata entry 'name'
WARNING: Skipping /usr/local/lib/python3.11/site-packages/PyYAML-6.0-py3.11.egg-info due to invalid metadata entry 'name'

Everything seems to be working now. Many thanks

@MaxistheSpy
Copy link

this is outdated brew install python@2 will return Error: No available formula with the name "python@2" because it has been removed by homebrew Homebrew/homebrew-core#49796

@armouti replace that with just python and you should be good. Noone uses python2 anymore

any chance you could update the gude then, since homebrew doesnt even support this installation its confusing

@MuhsinFatih
Copy link
Author

this is outdated brew install python@2 will return Error: No available formula with the name "python@2" because it has been removed by homebrew Homebrew/homebrew-core#49796

@armouti replace that with just python and you should be good. Noone uses python2 anymore

any chance you could update the gude then, since homebrew doesnt even support this installation its confusing

done.

@dwqualia
Copy link

I followed this exactly on Ventura 13.3 but I keep getting this error?

WARNING: Skipping /usr/local/lib/python3.11/site-packages/PyYAML-6.0-py3.11.egg-info due to invalid metadata entry 'name'

How can I fix this?

@brownj47
Copy link

I am having the same issue as @dwqualia. Could it be an issue with the PyYAML-6.0 package?

@MuhsinFatih
Copy link
Author

@brownj47 @dwqualia probably an issue with pyyaml. you can install a different version of python. 3.11 is the latest python so some packages might be unstable.

brew install python@3.10

@Drjacky
Copy link

Drjacky commented Jun 4, 2023

@brownj47 @dwqualia probably an issue with pyyaml. you can install a different version of python. 3.11 is the latest python so some packages might be unstable.

brew install python@3.10

I'm on the latest 3.11.3 and still have the issue.

@alessandraluz
Copy link

Oh. my. god. thank you for avoiding my impending life ending, by helping me with this incredible step by step!!!

@victorazzam
Copy link

To fix the PyYAML issue:

  1. Uninstall it with pythonXXX -m pip uninstall -y pyyaml
  2. Erase its path with rm -r /opt/homebrew/lib/pythonXXX/site-packages/PyYAML*

For both commands, replace XXX with the correct Python version for which the problem occurs.

@MichaelDerrenbacher
Copy link

Thank you for this! I had a patchwork python env, and this helped me get everything freshly installed

@gumbolastima
Copy link

I had a problem following these instructions, where removing the frameworks gave lots of "permission denied" errors. To solve it, I changed the command

ls -l /usr/local/bin | grep '../Library/Frameworks/Python.framework/Versions/2.7' | awk '{print $9}' | tr -d @ | xargs rm

to

sudo sh -c "ls -l /usr/local/bin | grep '../Library/Frameworks/Python.framework/Versions/2.7' | awk '{print \$9}' | tr -d @ | xargs rm"

@rudraksh97
Copy link

This thread saved me. I had to complete a python assignment for an interview and libraries weren't just installing. Big thanks!!

@mshonichev
Copy link

mshonichev commented Feb 8, 2024

I'm not sure how I got here, but this is huge!

gg indeed!

@drumstick90
Copy link

Thank you so much!
I also think installing packages by VSC isn't a good idea, I'd rather stick to pip install

@jdbohrman
Copy link

You are a fucking champ.

@franco-stuart
Copy link

With pip freeze, I get this error:

Traceback (most recent call last):
  File "/opt/homebrew/bin/pip", line 5, in <module>
    from pip._internal.cli.main import main
ModuleNotFoundError: No module named 'pip'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment