Skip to content

Instantly share code, notes, and snippets.

@tonyfast
Created November 11, 2020 22:52
Show Gist options
  • Save tonyfast/f74eb42f2a998d8e428a752ceb0cb1d1 to your computer and use it in GitHub Desktop.
Save tonyfast/f74eb42f2a998d8e428a752ceb0cb1d1 to your computer and use it in GitHub Desktop.

Use depfinder to discover the packages in a repo.

import depfinder
discovered = {}
for deps in [
    x[-1].describe()
    for x in depfinder.iterate_over_library('.')
]:
    for key, value in deps.items():
        discovered[key] = discovered.get(key, set()).union(value)
discovered = discovered.get('required', set()).union(
    discovered.get('questionable', set()))
discovered.discard('dgaf')

Save the packaged files to disk.

pathlib.Path('requirements-discovered.txt').write_text('\n'.join(discovered))

Try to install the discovered packages with conda.

action = doit.tools.CmdAction(['conda install --file requirements-discovered.txt'])
action.execute()

If the action fails then action.err is populated and we can discover the packages that couldn't be solve. we can assume those to be "pip" dependencies.

missing = [
    x.lstrip().lstrip('-').strip() for x in action.err.partition('Current channels:')[0].splitlines()
    if x.lstrip().startswith('- ')
]

reqs = set(filter(bool, pathlib.Path('requirements-discovered.txt').read_text().splitlines()))

split the requirements into separate files and install from those.

if action.err:
    pathlib.Path('requirements-conda.txt').write_text('\n'.join(reqs.difference(missing)))
    pathlib.Path('requirements-pip.txt').write_text('\n'.join(missing))

write a canonical environment file for folks to use later.

pathlib.Path('environment.yml').write_text(anyconfig.dumps(dict(
    name="dgaf",
    channels=["conda-forge"],
    dependencies=list(reqs.difference(missing)) + ['pip'] + [dict(pip=list(missing))]
), 'yaml'))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment