Fix broken readline in python and pdb interactive prompts in virtualenv with MacPorts python

fix-virtualenv-readline.md

Problem: borked readline in virtualenv

Python 3.7

$ mkvirtualenv test
$ python
Python 3.7.7 (default, Mar 18 2020, 09:44:23)
[Clang 9.1.0 (clang-902.0.39.2)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> ^[[A  # Up arrow does not scroll history
>>> import readline
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: dlopen(.../virtualenvs/test/lib/python3.7/readline.so, 2): no suitable image found.  Did find:
	.../virtualenvs/test/lib/python3.7/readline.so: file too short
	/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/readline/readline.py: file too short

The fix

$ workon <venv_name>
$ mv $VIRTUAL_ENV/lib/python3.7/readline.{so,py}

Generate mv commands to fix them all in one go

$ ls -l $WORKON_HOME/*/lib/python3.7/readline.so \
    | grep -Ev '\.so$' \
    | sed -E 's/.* (\/.*)\.so -> .*/mv -v \1.{so,py}/'

If that causes another error...

$ python
Python 3.7.7 (default, Mar 18 2020, 09:44:23)
[Clang 9.1.0 (clang-902.0.39.2)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import readline
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/dimagi/code/virtualenvs/cte/lib/python3.7/readline.py", line 1, in <module>
    from gnureadline import *
ModuleNotFoundError: No module named 'gnureadline'

How to fix that (maybe?)

# define function
function fix_venv_readline() {
    set -x
    vpath=$WORKON_HOME/$1/lib/python3.7
    opt_readline=/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/readline
    mv $vpath/readline.so{,.bak}
    ln -s $opt_readline/readline.py $vpath/
    ln -s $opt_readline/gnureadline.*.so $vpath/site-packages/
    set +x
}

# list virtualenvs needing repair
ls $WORKON_HOME/*/lib/python3.7/readline.so

# for each of those virtualenvs
fix_venv_readline <venv_name>

Python <= 3.6

Sample python interactive session:

$ python
Python 3.5.1 (default, Dec 29 2015, 10:15:41)
[GCC 4.2.1 Compatible Apple LLVM 7.0.2 (clang-700.1.81)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> "type an expression and press enter"
>>> 'type an expression and press enter'
_

Notice the prompt printed before the evaluated expression and the caret is on a blank line

For Python 3.6

https://trac.macports.org/ticket/48807#comment:121

sudo port install python36 +readline

For python < 3.6

Assuming virtuanenv created with --python=/opt/local/bin/pythonX.Y, replace X and Y with your python major/minor versions:

sudo port install pyXY-readline

deactivate

If it's easy to rebuild the virtualenv then just delete and rebuild it.

If you prefer to live dangerously you can try this instead to create a New virtualenv and then monkey-patch it with the old virtualenv contents

mv $WORKON_HOME/venv-name $WORKON_HOME/venv-name-borked-readline
mkvirtualenv --python=/opt/local/bin/pythonX.Y venv-name
cd $WORKON_HOME/venv-name
# may want to do this with --dry-run (-n) first
rsync -av --ignore-existing ../venv-name-borked-readline/ .