Skip to content

Instantly share code, notes, and snippets.

  • Star 12 You must be signed in to star a gist
  • Fork 1 You must be signed in to fork a gist
Star You must be signed in to star a gist
Save gh640/233a6daf68e9e937115371c0ecd39c61 to your computer and use it in GitHub Desktop.
Sample: Use Python Poetry cache on GitHub Actions workflow
name: Use Python Poetry cache on GitHub Actions workflow
on:
push:
branches:
- main
env:
PYTHON_VERSION: "3.11"
POETRY_VERSION: "1.4.2"
POETRY_URL: https://install.python-poetry.org
jobs:
use_cache:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v3
- name: Set up Python ${{ env.PYTHON_VERSION }}
uses: actions/setup-python@v4
with:
python-version: ${{ env.PYTHON_VERSION }}
id: setup_python
# Poetry cache depends on OS, Python version and Poetry version.
- name: Cache Poetry cache
uses: actions/cache@v3
with:
path: ~/.cache/pypoetry
key: poetry-cache-${{ runner.os }}-${{ steps.setup_python.outputs.python-version }}-${{ env.POETRY_VERSION }}
# virtualenv cache should depends on OS, Python version and `poetry.lock` (and optionally workflow files).
- name: Cache Packages
uses: actions/cache@v3
with:
path: ~/.local
key: poetry-local-${{ runner.os }}-${{ steps.setup_python.outputs.python-version }}-${{ hashFiles('**/poetry.lock') }}-${{ hashFiles('.github/workflows/*.yml') }}
- name: Install Poetry ${{ env.POETRY_VERSION }}
run: |
curl -sSL ${{ env.POETRY_URL }} | python - --version ${{ env.POETRY_VERSION }}
echo "$HOME/.local/bin" >> $GITHUB_PATH
- name: Install Dependencies
run: poetry install
# Write actual actions here.
@gh640
Copy link
Author

gh640 commented Aug 8, 2021

This sample is for Poetry >= 1.1.7. Poetry <= 1.1.6 doesn't use ~/.local for virtualenvs.

@gh640
Copy link
Author

gh640 commented Aug 8, 2021

Cache dir:

  • macOS: ~/Library/Caches/pypoetry
  • Windows: C:\Users\<username>\AppData\Local\pypoetry\Cache
  • Unix: ~/.cache/pypoetry

https://python-poetry.org/docs/master/configuration/#cache-dir

@gh640
Copy link
Author

gh640 commented Aug 8, 2021

@alexpovel
Copy link

alexpovel commented Jul 24, 2022

See also: https://github.com/actions/setup-python#caching-packages-dependencies

It's a bit tricky though, I had to do this (poetry env use <version> specifically).

@gh640
Copy link
Author

gh640 commented Jul 25, 2022

As @alexpovel mentioned, setup-python@v3 introduced caching Poetry dependencies.

steps:
- uses: actions/checkout@v3
- name: Install poetry
  run: pipx install poetry
- uses: actions/setup-python@v4
  with:
    python-version: '3.9'
    cache: 'poetry'
- run: poetry install
- run: poetry run pytest

Implementation:

@hongbo-miao
Copy link

Thanks for sharing! ❤️

@rockandska
Copy link

Not sure why and I still didn't find a clear reason, but I have for sure a CI who fail due to the use of setup-python and cache: poetry.
I prefer to use the solution exposed in this gist (even without the bug I identified) because we could install poetry after setup-python with pip install and it will use the python version installed by setup-python instead of the system one.

@hongbo-miao
Copy link

Hi @raineorshine here is my way of using https://github.com/Hongbo-Miao/hongbomiao.com/blob/4aae07937510b870496a992f231f38f2e9f0c242/.github/workflows/lint.yml#L260-L270

Hope it helps! 😃

    steps:
      - name: Checkout
        uses: actions/checkout@v3
      - name: Install Poetry
        run: |
          pipx install poetry
      - name: Set up Python
        uses: actions/setup-python@v4
        with:
          python-version: '3.11'
          cache: poetry
      - name: Set Poetry environment
        run: |
          poetry env use 3.11
      - name: Install dependencies
        run: |
          poetry install --no-root
      - name: Do something
        run: |
          poetry run xxx

@tedmiston
Copy link

There are also 2 different maintained actions you can use to install / setup Poetry:

https://github.com/Gr1N/setup-poetry

https://github.com/snok/install-poetry

(They are pretty similar in functionality.)

@gh640
Copy link
Author

gh640 commented Apr 7, 2023

I made a gist of a composite action to install Python and Poetry (that I use):

https://gist.github.com/gh640/317865611e39b93442579ed3cd4491ce

I hope this helps.

@hongbo-miao
Copy link

hongbo-miao commented Apr 7, 2023

Here is my current way with some small improvements 😃

    steps:
      - name: Checkout
        uses: actions/checkout@v3
      - name: Install Poetry
        run: |
          pipx install poetry
      - name: Set up Python
        uses: actions/setup-python@v4
        with:
          python-version: '3.11'
          cache: poetry
          cache-dependency-path: poetry.lock
      - name: Set Poetry environment
        run: |
          poetry env use 3.11
      - name: Install dependencies
        run: |
          poetry install --no-root
      - name: Run
        run: |
          poetry run xxx

@radzek15
Copy link

radzek15 commented Feb 28, 2024

@hongbo-miao thank you for sharing. I got an error when trying your code:
Installing dependencies from lock file pyproject.toml changed significantly since poetry.lock was last generated. Run poetry lock [--no-update] to fix the lock file.

So i just needed to add this small change:

        run: |
          poetry lock --no-update
          poetry install --no-root

@hongbo-miao
Copy link

You are welcome, @radzek15 ! However, you should run poetry lock --no-update locally and commit the change in your git instead of running in the CI.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment