Skip to content

Instantly share code, notes, and snippets.

@stefanvangastel
Last active May 7, 2020 20:35
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save stefanvangastel/b7ce560816e3a913d52bd9313120a7fd to your computer and use it in GitHub Desktop.
Save stefanvangastel/b7ce560816e3a913d52bd9313120a7fd to your computer and use it in GitHub Desktop.
Download Conda repo's for offline use (including .conda files). First edit links.txt, then run and finish download.sh and end with downloadConda.sh
#!/bin/bash
for link in `cat links.txt`; do
wget -r -l1 -H -t1 -N -np -nH -P "pkgs/" --cut-dirs=1 -e robots=off -R "index.html*" $link
done
# -r recursive
# -l1 maximum recursion depth (1=use only this directory)
# -H span hosts (visit other hosts in the recursion)
# -t1 Number of retries
# -N turn on timestamping
# -np dont ascend to parent dir
# -nH Dont create host directories
# -P prefix saving dir
# --cut-dirs=1 cut 1 directory off saving path
# -erobots=off execute "robots.off" as if it were a part of .wgetrc
# -R reject file extensions
#!/bin/bash
wget -e robots=off https://repo.continuum.com/pkgs/main/channeldata.json -O pkgs/main/channeldata.json
wget -e robots=off https://repo.continuum.com/pkgs/r/channeldata.json -O pkgs/r/channeldata.json
echo '' > commands.txt
for f in $(find pkgs -name '*.tar.bz2'); do
name="${f%.tar.bz2}"
echo wget -nc -e robots=off "https://repo.continuum.io/$name.conda" -O "$name.conda" >> commands.txt
done
export LC_ALL=en_US.UTF-8
cat commands.txt | parallel -j 32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment