Skip to content

Instantly share code, notes, and snippets.

@tjwaterman99
Created March 20, 2021 06:22
Show Gist options
  • Save tjwaterman99/abf0fda18ef7ea94f796cbf4ed9213f3 to your computer and use it in GitHub Desktop.
Save tjwaterman99/abf0fda18ef7ea94f796cbf4ed9213f3 to your computer and use it in GitHub Desktop.
Example CI/CD process for dbt on Snowflake
# For workflow syntax reference, please see the documentation available on Github
# https://docs.github.com/en/actions/reference/workflow-syntax-for-github-actions
name: CI
on:
push:
schedule:
- cron: '50 11 * * *' # Run the `main` branch daily at 11:50am UTC
workflow_dispatch: # Allow the workflow to be triggered manually
env:
SNOWSQL_USER: github
SNOWSQL_PWD: ${{ secrets.SNOWSQL_PWD }}
SNOWSQL_DATABASE: acme
SNOWSQL_SCHEMA: staging
SNOWSQL_WAREHOUSE: ci
SNOWSQL_ROLE: transformer
SNOWSQL_ACCOUNT: ${{ secrets.SNOWSQL_ACCOUNT }}
LC_ALL: C.UTF-8
LANG: C.UTF-8
jobs:
build:
runs-on: ubuntu-latest
container:
image: docker.pkg.github.com/tjwaterman99/snowflake_dbt/snowflake_dbt:0.1.5
credentials:
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
steps:
- name: Checkout
uses: actions/checkout@v2
- name: pip install
run: pip install -r requirements.txt
- name: dbt debug
run: dbt debug --profiles-dir $PWD
- name: dbt deps
run: dbt deps --profiles-dir $PWD
# Reset the staging database by cloning production
- name: Reset staging database
run: snowsql -f scripts/reset_staging_db.sql
# Load seed files
- name: dbt seed
run: "dbt seed --full-refresh --profiles-dir $PWD"
# Build's the tables and views on the DWH's `staging` schema with dbt
- name: dbt run
run: dbt run --profiles-dir $PWD
# Test the resulting tables to identify any issues before the data lands in production
- name: dbt test
run: dbt test --profiles-dir $PWD
# Clone the staging schema into the production schema once the tests have
# validated the data
- name: Promote staging
run: snowsql -f scripts/promote_staging_db.sql
if: github.ref == 'refs/heads/main'
@tjwaterman99
Copy link
Author

tjwaterman99 commented Oct 16, 2023

Hey @francisaddae - yes, this was intended as an illustration for running a CI/CD process for dbt on Snowflake.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment