Skip to content

Instantly share code, notes, and snippets.

@LeeBergstrand
Last active August 17, 2017 18:13
Show Gist options
  • Save LeeBergstrand/f8369b83c8010908de6373ac2b309422 to your computer and use it in GitHub Desktop.
Save LeeBergstrand/f8369b83c8010908de6373ac2b309422 to your computer and use it in GitHub Desktop.
Take this part of the config for example:
      - restore_cache:
          keys:
          - v1-dependencies-{{ checksum "requirements.txt" }}
          - v1-dependencies-
          
      - save_cache:
          key: v1-dependencies-{{ checksum "requirements.txt" }}
          paths:
            - ./venv

{{ checksum "requirements.txt" }} - Takes a unique hash of the requirments.txt file. the {{}} is a templateing engine varible insertion marker.

So if the checksum "requirements.txt" (the hash of that file from some cli program) generated the hash AD45633DF543. The key name would then be v1-dependencies-AD45633DF543.

So during the restore:
  1. Check for the key v1-dependencies-{{ checksum "requirements.txt" }}.
  2. This becomes v1-dependencies-AD45633DF543 after proccesing.
  3. If v1-dependencies-AD45633DF543 does not exist then look for any hash called starting with v1-dependencies- (this is confusing it should really be written as v1-dependencies-*).
  4. Retore this cache.
  5. Any paths originally stored under the key v1-dependencies-AD45633DF543 ( or v1-dependencies-* if v1-dependencies-AD45633DF543 is not found) are restored. In this case it would be the path ./venv.

Notes:

  • Remember the hash is only being create for file requirements.txt thus we will start building new depencancies from scratch (or at least a preveous cached version) if requirements.txt is changed (which generates a new checksum hash).
  • This will trikle over the the next cache save step.
So during the save:
  1. Store the cache under the key v1-dependencies-{{ checksum "requirements.txt" }}
  2. If the requirments.txt changes then a new this will result in a new cache being saved.

Notes:

Summary

  • We actually have two files that indcate a new cache update is requried. These are requirements.txt and base_install.sh.
  • We may need two restore and save steps. one for the python deps and one for the base_install.sh.
  • If we can't do that we can md5 has each and put that into a file and checksum that third file.a
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment