Skip to content

Instantly share code, notes, and snippets.

@benlk
Last active December 15, 2016 18:55
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save benlk/92b11d65f2488d993193b114fba16d89 to your computer and use it in GitHub Desktop.
Save benlk/92b11d65f2488d993193b114fba16d89 to your computer and use it in GitHub Desktop.

The full Paleoclimatology archive is now just under 200GB. A tar file of the entire Paleoclimatology FTP site weighs in at 121 GB, and when gzipped, the resulting .gz file is about 75 GB. We periodically create these .tar.gz files to be permanently archived on tape media, and as it turns out I created the current version yesterday. It will be in the Paleoclimatology FTP folder until the tape ingest is confirmed, so that would be an easy way to download the entire archive if you wish.

It looks like the address for that zip is ftp://ftp.ncdc.noaa.gov/pub/data/paleo/paleo_climate_qual_r20161214.tar.gz and has an md5 at ftp://ftp.ncdc.noaa.gov/pub/data/paleo/paleo_climate_qual_r20161214.tar.gz.md5

The backups are internal to NOAA/NCEI, maintained by the Archive Branch in Asheville. It is intended to be permanent storage.


context:

Twitter thread: https://twitter.com/benlkeith/status/809450954579910656

If you wanted to back up the NOAA Paleoclimatology archive, how much disk space would it take up? 200 GB, ish.

Why would you want to? https://www.washingtonpost.com/news/energy-environment/wp/2016/12/13/scientists-are-frantically-copying-u-s-climate-data-fearing-it-might-vanish-under-trump/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment