Dynamic Manual Workflow

dynamic-manual-workflow.md

How to create a Dynamic Manual Workflow with Github Actions

Overview

GitHub Actions makes it easy to automate all your software workflows, now with world-class CI/CD. Build, test, and deploy your code right from GitHub. Make code reviews, branch management, and issue triaging work the way you want.

Context

With all this feature, it's still missing some feature like making a request before a the workflow is created.

For understand what is the issue, let me explain my use-case

• We deploy our build package on ECR
• We want de deploy a given package version on a given environment

So our need was to have a dropdow menu with all our build packages(dynamic) and also a dropdown menu for all our environment.

To actually fixed this, I decided to dynamically create the file on a given event, for our cas I choice on Merge on our default branch.

So this is the final flow :

• On Merge
  • Get all images from our Docker Repository
  • Format the list images and add them to the manual workflow template
  • Commit my changes and put the file to the .github/workflows directory
  • Add the token to the actions/checkout checkout to forced Github to copy on the workflow directory.

Solutions

Check out this sample file to actually understand our I deal with that

.github/workflows/deployment.template.yml: This is the file with the manual workflow created but without our list of packages version.

name: Deployment

on:
 workflow_dispatch:
   # Inputs the workflow accepts.
   inputs:
     version:
       description: 'Package version to deployed'
       required: true
       type: choice
       options:
         $IMAGES_OPTIONS
     environment:
       description: 'Target Environment'
       required: true
       type: choice
       options:
         - DEV
         - PROD
jobs:
 deployment:
   runs-on: ubuntu-latest
   name: "Deploying on : ${{ inputs.environment }} 🌍"
   ... your normal workflow

The file is below is the template for the final Manual Workflow and the $IMAGES_OPTIONS variables will be replace on the merge event.

name: Generate ECR Image

on:
  pull_request_target:
    types:
      - closed
    branches:
      - develop
jobs:
  get-images-list:
    runs-on: ubuntu-latest
    environment: ${{ inputs.environment }}
    steps:
      - name: checkout code
        uses: actions/checkout@v3
        with:
          token: ${{ secrets.accessToken }}

      - name: Configure AWS Credentials
        uses: aws-actions/configure-aws-credentials@v1
        with:
          aws-access-key-id: ${{ secrets.awsAccessKeyId }}
          aws-secret-access-key: ${{ secrets.awsSecretKey }}
          aws-region: ${{ secrets.awsRegionName }}

      - name: Get All Images On ECR
        run: |
          responses=$(aws ecr list-images --repository-name ${{ secrets.awsEcrRepository }} | (jq '.imageIds'))
          output=""
          for release in $(echo $responses | jq '.[] | .imageTag')
          do
             output+="- $release\n          "
          done
          sed "s/\$IMAGES_OPTIONS/$output/g" $GITHUB_WORKSPACE/.github/workflows/deployment.template.yml > $GITHUB_WORKSPACE/.github/workflows/deployment.yml

      - uses: stefanzweifel/git-auto-commit-action@v4
        with:
          commit_message: Generated images
    

And this last step file just get the images from ECR and format it to the new deployment file deployment.yml and with stefanzweifel/git-auto-commit-action we commit the changed and Github can handled it if it's on the default branch.

I hope this Gist help you. Do not hesitate to let comment or any questions you have.

Thanks for sharing. It's a clever solution for a serious limitation on Github Actions today.
For the sed command, you could use the -i option for the same thing and have a shorter command:

sed -i "s/\$IMAGES_OPTIONS/$output/g" $GITHUB_WORKSPACE/.github/workflows/deployment.template.yml

I have 2 questions. The 1st one is: after you execute this once, your file .github/workflows/deployment.template.yml has some images on the version parameter and you have replaced the $IMAGES_OPTIONS placeholder. How/when do you restore it to be able to execute this more than once?

And the 2nd one is more like a concern. Do you end up with several commits with the message 'Generated images' on your default branch? Is that a problem with the project you're working?

Thank you again for taking the time of sharing this.

For the first concern, I save the result of the images on a new file deployment.yml, so the template is still the same. And for the 2nd, yes We have several "Generated images" commit on our default branch.

Crap. You're right, it's a different file. Disregard my sed comment. Thanks again!

That's a great idea and solution. I tried to adopt it.
Unfortunately, I can not use the provided ad-m/github-push-action because, my code is in a company related repo, where not allowed to use a lot of public github actions.
So I trued to push the workflow related modification with direct git commands, like fit add, git commit -a -m 'bal bal' etc...
I also created token what has workflow permission, and I also checked and set for action the workflow permission as well...

The last try, how I wanted to push the dynamic change is:

- name: Push changes
  run: git push origin HEAD:main
  env:    
    GITHUB_TOKEN: ${{ secrets.WORKFLOW2 }}           

...but in all cases I got the same error:

`! [remote rejected] HEAD -> main (refusing to allow a GitHub App to create or update workflow `.github/workflows/deployment.yaml` without `workflows` permission)`

Do you have any idea what am I doing wrong, or missed to set?

Thanks!