Skip to content

Instantly share code, notes, and snippets.

@vsoch
Created April 24, 2022 01:32
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save vsoch/41b4559d8f87eb9d6e62945e02689428 to your computer and use it in GitHub Desktop.
Save vsoch/41b4559d8f87eb9d6e62945e02689428 to your computer and use it in GitHub Desktop.
Example of CiteLang render (post render)

Summary

Portability and reproducibility of complex software stacks is essential for researchers to perform their work. High Performance Computing (HPC) environments add another level of complexity, where possibly conflicting dependencies must co-exist. Although container technologies like Singularity @conda{name=singularity} make it possible to "bring your own environment," without any form of central strategy to manage containers, researchers who seek reproducibility via using containers are tasked with managing their own container collection, often not taking care to ensure that a particular digest or version is used. The reproducibility of the work is at risk, as they cannot easily install and use containers, nor can they share their software with others.

Singularity Registry HPC (shpc) @pypi{name=singularity-hpc} is the first of its kind to provide an easy means for a researcher to add their research software for sharing and collaboration with other researchers to an existing collection of over 200 popular scientific libraries @github{name=autamus/registry} @github{name=spack/spack, release=0.17}. The software installs containers as environment modules that are easy to use and read documentation for, and exposes aliases for commands in the container that the researcher can add to their pipeline without thinking about complex interactions with a container. The simple addition of an entry to the registry maintained by shpc comes down to adding a yaml file, and after doing this, another researcher can easily install the same software, down to the digest, to reproduce the original work.

References

Manager Name Credit
conda singularity 0.12
pypi singularity-hpc 0.12
github autamus/registry 0.12
github spack/spack 0.12
pypi pytest 0.02
pypi requests 0.02
pypi ruamel.yaml 0.02
pypi jsonschema 0.02
pypi Jinja2 0.02
pypi spython 0.02
github actions/checkout 0.02
github actions/upload-artifact 0.02
github docker/build-push-action 0.02
github docker/login-action 0.02
github docker/setup-buildx-action 0.02
github docker/setup-qemu-action 0.02
conda libgcc-ng 0.02
github autamus/artifact-dl 0.01
github autamus/binoc 0.01
github autamus/buildconfig 0.01
github autamus/builder 0.01
github autamus/librarian 0.01
github autamus/merge-commits 0.01
github autamus/smuggler 0.01
github NextThought/sphinxcontrib-programoutput 0.01
github actions/download-artifact 0.01
github actions/setup-python 0.01
github codecov/codecov-action 0.01
github dorny/paths-filter 0.01
github readthedocs/sphinx_rtd_theme 0.01
github sphinx-doc/sphinx 0.01
github ztane/python-Levenshtein 0.01
conda openssl 0.01
conda libuuid 0.01
conda libstdcxx-ng 0.01
conda libseccomp 0.01
conda libarchive 0.01
conda cni-plugins 0.01

Note that credit values are rounded and expanded (so shared dependencies are represented as one record) and may not add to 1.0. Rounded values that hit zero are removed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment