Skip to content

Instantly share code, notes, and snippets.

@sungchun12

sungchun12/Dockerfile

Created July 16, 2023 20:30
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save sungchun12/70a0bbf2bdb6366b958839a4cd176496 to your computer and use it in GitHub Desktop.
Save sungchun12/70a0bbf2bdb6366b958839a4cd176496 to your computer and use it in GitHub Desktop.
Download ZIP
quick setup for airflow using astro cli
Raw
Dockerfile
FROM quay.io/astronomer/astro-runtime:7.6.0
# Install python packages
RUN pip install -r requirements.txt
# Switch to root user for installing git
USER root
# Install git
RUN apt-get update && apt-get install -y git
# Clean up APT cache
RUN apt-get clean && rm -rf /var/lib/apt/lists/*
# Switch back to the original user
USER astro
# Create a Python virtual environment and install dbt-bigquery
ENV DBT_VENV_PATH /home/astro/dbt_venv
RUN python -m venv $DBT_VENV_PATH && \
. $DBT_VENV_PATH/bin/activate && \
pip install dbt-bigquery==1.5.0
Raw
requirements.txt
apache-airflow-providers-dbt-cloud==1.0.2
polygon-api-client==1.10.1
apache-airflow-providers-google
@sungchun12
Copy link
Author

The main issue with the DAG was that the polygon api client was NOT installed in the docker container hosting airflow. This should fix that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment