Skip to content

Instantly share code, notes, and snippets.

@sgoley
Last active September 14, 2022 19:35
Show Gist options
  • Save sgoley/af3e1f857251786afb1d739ada2e2a03 to your computer and use it in GitHub Desktop.
Save sgoley/af3e1f857251786afb1d739ada2e2a03 to your computer and use it in GitHub Desktop.
creates a github action workflow for a dbt repository to automatically sqlfmt the files in the current PR and then kickoff a CI/CD dbtCloud run job with a schema override. Requires a few secret variables to be set on the repo for the dbt cloud action to run.
# GitHub Action that uses sqlfmt to reformat the sql in an incoming pull request.
# If all sql in the pull request is compliant with sqlfmt then this Action does nothing.
# Otherwise, sqlfmt is run and its changes are committed back to the incoming pull request.
name: fmtdeploy
on: [pull_request]
jobs:
fmt:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v1
- name: Set up Python 3.7
uses: actions/setup-python@v1
with:
python-version: 3.7
- name: install sqlfmt
run: pip install shandy-sqlfmt[jinjafmt]
- name: Get changed files
id: get_file_changes
uses: trilom/file-changes-action@v1.2.4
with:
output: ' '
- name: Get changed .sql files in /models to lint
id: get_files_to_lint
shell: bash -l {0}
run: |
# Set the command in the $() brackets as an output to use in later steps
echo "::set-output name=lintees::$(
# Issue where grep regular expressions don't work as expected on the
# Github Actions shell, check dbt/models/ folder
echo \
$(echo ${{ steps.get_file_changes.outputs.files_modified }} |
tr -s ' ' '\n' |
grep -E '^models.*[.]sql$' |
tr -s '\n' ' ') \
$(echo ${{ steps.get_file_changes.outputs.files_added }} |
tr -s ' ' '\n' |
grep -E '^models.*[.]sql$' |
tr -s '\n' ' ')
)"
- name: if needed, commit sqlfmt changes to the pull request
run: |
for changed_file in ${{ steps.get_files_to_lint.outputs.lintees }}; do
echo "formatting ${changed_file}"
sqlfmt ${changed_file} || continue
done
git config --global user.name 'autosqlfmt'
git config --global user.email 'autosqlfmt@github.com'
git remote set-url origin https://x-access-token:${{ secrets.GITHUB_TOKEN }}@github.com/$GITHUB_REPOSITORY
git checkout $GITHUB_HEAD_REF
git add -A
git diff-index --quiet HEAD || git commit -m "fixup: format sql with sqlfmt"
git push
deploy:
needs: fmt
runs-on: ubuntu-latest
steps:
- uses: jwalton/gh-find-current-pr@v1
id: findPr
with:
# Can be "open", "closed", or "all". Defaults to "open".
state: open
# This will echo "Your PR is 7", or be skipped if there is no current PR.
- run: echo "Your PR is ${PR}"
if: success() && steps.findPr.outputs.number
env:
PR: ${{ steps.findPr.outputs.pr }}
- name: dbtCloud CI/CD job kickoff
id: dbt_cloud_run
uses: sgoley/dbt-cloud-action@sgoley-override-inputs
with:
dbt_cloud_token: ${{ secrets.DBT_CLOUD_API_TOKEN }}
dbt_cloud_account_id: ${{ secrets.DBT_CLOUD_ACCOUNT_ID }}
dbt_cloud_job_id: ${{ secrets.DBT_CLOUD_JOB_ID }}
git_branch: ${{ github.head_ref }}
target_name_override: dev
schema_override: dbt_cloud_pr_${{ secrets.DBT_CLOUD_JOB_ID }}_${{ steps.findPr.outputs.PR }}
@sgoley
Copy link
Author

sgoley commented Sep 14, 2022

Make sure to set the following secrets on the repository:

DBT_CLOUD_ACCOUNT_ID
DBT_CLOUD_API_TOKEN
DBT_CLOUD_JOB_ID

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment