If you're requirements.txt file is the output of pip freeze
then it's really
a lock file, not a list of project dependencies. To get just the actual
dependencies:
find . -name \*.py -type f | xargs sed -En '/^(import|from)/ {s/\S+ //; s/ .*//; s/\..*//; p}' | sort -u >reqs
This extracts foo
from all import foo.bar...
and from foo.bar import...
lines, dropping
local imports (starting with '.').
To filter out standard library entries:
# testreqs.py
"""Test names on stdin to see if they're importable.
Useful for refining requirements / filtering standard library modules.
"""
import sys
from importlib import import_module
for line in filter(None, map(str.strip, sys.stdin)):
try:
import_module(line)
except ImportError:
print(line)
python testreqs.py < reqs
Or use docker for a clean environment
docker run --rm -it -v $PWD:/data python sh -c 'python /data/testreqs.py < /data/reqs'
You may still need to cross-reference the output and your requirements.txt
lock file for
packages with names that don't match their module name, e.g. git
comes from GitPython
.
Another approach would be to use pipdeptree
, but that will report things that are
installed but not used.