Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
Caching with Github Actions + S3
name: Python Tests
jobs:
build:
runs-on: ubuntu-18.04
strategy:
max-parallel: 2
matrix:
python-version: [3.5, 3.6, 3.7]
steps:
- uses: actions/checkout@v1
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v1
with:
python-version: ${{ matrix.python-version }}
fetch-depth: 1
- name: Fetch cached dependencies
run: |
set -x
pip install --upgrade pip
pip install awscli
CACHE_PATH=s3://BUCKET/PATH/PROJECT/WORKFLOW/JOB/py${{ matrix.python-version }}
for key in venv; do
if [[ $(aws s3 ls ${CACHE_PATH}/${key}.tar.gz | wc -l) > 0 ]]; then
echo "Restoring cache for ${key}"
aws s3 cp ${CACHE_PATH}/${key}.tar.gz ${key}.tar.gz --no-progress
tar -pzxf ${key}.tar.gz ${key}
fi
done
env:
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
- name: Install dependencies
run: |
set -x
sudo apt-get -qq install -y gcc openjdk-8-jdk
python -m pip install --upgrade pip
python -m venv venv
venv/bin/pip install -r requirements.txt
- name: Cache dependencies
run: |
set -x
CACHE_PATH=s3://BUCKET/PATH/PROJECT/WORKFLOW/JOB/py${{ matrix.python-version }}
for key in venv; do
if [[ -e $key ]]; then
echo "Caching ${key}"
tar -pzcf ${key}.tar.gz ${key}
aws s3 cp ${key}.tar.gz ${CACHE_PATH}/${key}.tar.gz --no-progress
fi
done
env:
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment