Skip to content

Instantly share code, notes, and snippets.

Show Gist options
  • Save brianjbayer/53ef17e0a15f7d80468d3f3077992ef8 to your computer and use it in GitHub Desktop.
Save brianjbayer/53ef17e0a15f7d80468d3f3077992ef8 to your computer and use it in GitHub Desktop.
A How-To Guide for using Environment Variables and GitHub Secrets in GitHub Actions for Secrets Management in Continuous Integration

Adding Basic Secret Handling with Environment Variables and GitHub Secrets

Kitten in a Box - Wendy Bayer


As I mentioned on my failed attempt using Docker Secrets, I have been looking for a relatively simple and portable method of handling secrets (i.e. confidential data like usernames and passwords) in my personal projects. Something that demonstrated the principles and practices of (good) secret management that was still useable locally, in Continuous Integration/Continuous Deployment (CI/CD), and in production.

From the application perspective, I want a general mechanism for secret handling and retrieval that can be used regardless of the external secret management system. Something that can be used when running natively, in Docker Compose, in my GitHub Actions CI and in a potential Kubernetes production deployment. A pattern that can be used in any application in any language.

The approach that I present here in this simple implementation does not give me all of this, but it is definitely more secure (at least in my CI of GitHub Actions) and is a basic starting point for later improvement. It may also help you or get you thinking.

The approach...

  1. Change the application to use environment variables to specify the secrets (making them required)
  2. Use GitHub Secrets as the secrets management system in GitHub Actions CI

About This Approach

About Using Environment Variable for Secret Values

The main benefit of this approach is the change to use environment variables to set the sensitive credentials instead of inline values ("hardcoded") or configuration files. This means that the secret values are set at run time (last minute), ideally by a secrets management system.

Using environment variables is also more flexible, like supporting different credentials and/or different secrets management systems in different environments. This is why it is a 12-Factor App practice.

It also makes password changes (i.e. rotation) easier with usually just a restart and no rebuilds.

Caveat: But, using environment variables for any critical secrets means that they are now required to run your application. If you use environment variables for any credentials that are required for your application to start or run properly like for dependent systems such as databases, then you must now supply these environment variables with valid values. You can't set any default values in code or configuration files without exposing these secrets

🚢 Captain Obvious says that using environment variables to set the values is probably the most common method of passing secrets to applications.

But, of course, using environment variables for secrets does not offer them any real security. It is is simply more transient than having them in persisted files and offers at-run-time flexibility.

You still need a secrets management system for securing your secrets.

For CI, if you are already using GitHub Actions like I am, then using Github Secrets with environment variables is a good and easy solution.

About GitHub Secrets

GitHub Secrets are GitHub's secret management solution for it's GitHub Actions CI/CD system. GitHub Secrets uses a libsodium sealed box approach so that secrets are encrypted before reaching GitHub. Github Secrets are encrypted both in transit and at rest (up until they are assigned to the environment variable).

GitHub Secrets are only available in GitHub Actions and can be assigned to either environment variables or inputs.

There are 3 levels of GitHub Secrets...

  1. Secrets at the repository level
  2. Secrets at the (repository) environment level
  3. Secrets at the organization level

There is also a REST API for the Operations, Administration, and Management (OA&M) of your GitHub Secrets which would support using them with another (i.e. "source of truth") secrets management system. You can create/update secrets with this API, you can NOT retrieve their values.

GitHub Actions does a decent job masking the values of environment variables associated with GitHub Secrets, for example in logs. However you must take care to prevent "leaking", such as assigning these environment variables to other variables and then displaying these other variables. GitHub Actions would not mask these other variables which now contain your secrets.

Github secrets are not shared if the repository is forked.

🚢 Captain Obvious says that you would be vulnerable to Pull Requests with code that intentionally exposes/captures your secrets


Change Application Code to Use Environment Variables for Credentials

Whether or not you use GitHub Secrets, changing your application to use environment variables for your secrets is a great first step. Some frameworks such as .NET/C# even offer the ability to override settings files with environment variables in their configuration management.

Your specific language and implementation will differ, but here is a before and after example in Ruby of a browser acceptance test method that logs in a valid user.

Before...

Originally, this Ruby method has inline values for the credentials...

def login_with_valid_credentials
  username_input[0].set 'tomsmith'
  password_input[0].set 'SuperSecretPassword!'
  submit_button[0].click

After...

And here it is changed to use environment variables...

  def login_with_valid_credentials
    username_input[0].set ENV['LOGIN_USERNAME']
    password_input[0].set ENV['LOGIN_PASSWORD']
    submit_button[0].click

Running the Application Now Requires Secret Environment Variables

Now when the application is run, it requires the new secret environment variables to be set.

On command line...

LOGIN_USERNAME=tomsmith LOGIN_PASSWORD=SuperSecretPassword! 

Or as a sourced and/or default Docker Compose .env file...

LOGIN_USERNAME=tomsmith
LOGIN_PASSWORD=SuperSecretPassword!

🚢 Captain Obvious says that you should never commit any .env files to your source code repository and/or publish them in documentation.


Add Secret Environment Variables to Docker Compose

If you have a Docker Compose framework for your application like me, you will need to add your new (and required) secrets environment variables to your application's service definition usually in the base docker-compose.yml. You will probably want to use local environment variables with the same names to set those in your application's service (i.e. container).

For the Ruby example shown here, you would add...

environment:
  - LOGIN_USERNAME
  - LOGIN_PASSWORD

For example in this docker-compose.yml file...

version: '3.4'
services:
  browsertests:
    image: browsertestsimage
    environment:
      - LOGIN_USERNAME
      - LOGIN_PASSWORD

Testing

This is a good point to test your new secrets environment variables.

Test that your application...

  1. Fails when the new secrets environment variables are not set...
    • Locally
    • Docker Compose
  2. Runs when new secret environment variables are properly set
    • Locally
    • Docker Compose
  3. Fails when pushed to CI which does not yet set the secret environment variables

Add Setting Secret Environment Variables to GitHub Actions

Testing first and proving it fails before your changes are made not only brings certainty that it is your changes that is making it work, but it can also save you time by not having to do or undo work for any negative testing later.

If you start by adding the referencing of your not-yet-created GitHub Secrets in GitHub Actions, you can prove that your added environment variables are being used and that your secrets are not found when the Action fails.

If you have multiple actions, you can start by just changing one and when that fails as you expect, you can then "copy pasta" your tested code into the other actions.

Name Your GitHub Secrets

To add your GitHub Secrets, you will have to name them now. Although, you will be creating them later.

🔬 For more information on naming your GitHub Secrets, see the GitHub documentation on naming GitHub Secrets

To reference your GitHub Secret in GitHub Actions, use the following syntax...

${{ secrets.YOUR_GITHUB_SECRET_NAME }}

For this example, something like this for the GitHub Secrets Names and their reference...

GITHUB SECRET NAME SECRET ACCESS in GITHUB ACTIONS
LOGIN_USERNAME ${{ secrets.LOGIN_USERNAME }}
LOGIN_PASSWORD ${{ secrets.LOGIN_PASSWORD }}

🔬 For more information on accessing your GitHub Secrets, see the GitHub documentation on setting environment variables with secrets

Add Referencing Your GitHub Secrets in your GitHub Actions

Now that you know what you are naming your GitHubs Secrets and how to reference them, you can add them to your GitHub Actions by assigning them to the environment variables used by your application (and Docker Compose)...

    env:
      LOGIN_USERNAME: ${{ secrets.LOGIN_USERNAME }}
      LOGIN_PASSWORD: ${{ secrets.LOGIN_PASSWORD }}

Like this in this GitHub Actions file .github/workflows/check_deployable.yml...

  run-tests-default-chrome:
    needs: build-deploy
    runs-on: ubuntu-latest
    env:
      LOGIN_USERNAME: ${{ secrets.LOGIN_USERNAME }}
      LOGIN_PASSWORD: ${{ secrets.LOGIN_PASSWORD }}

Testing

Now run the GitHub Action to test that it is properly accessing your not-yet-created GitHub Secrets and that it fails.

Create GitHub Secrets

Once you have added your GitHub Secrets to at least one of your Actions, you can create the actual GitHub Secret with its value following the documentation on creating a GitHub Secret.

For this example, something like this for creating the GitHub Secrets and their values...

GITHUB SECRET NAME GITHUB SECRET VALUE
LOGIN_USERNAME tomsmith
LOGIN_PASSWORD SuperSecretPassword!

Testing

Now if you run your GitHub Action using your GitHub Secrets, it should run properly.

You can now change the rest of your actions to use your GitHub Secrets.

Optional: Add Your .env File to Your Ignore Files

Many operating and orchestration environments like Docker Compose recognize .env files by default for setting environment variables.

If you plan on using a .env file (like for local development) to store your secrets environment variables' values, you could add ignoring it to your Ignore files such as .gitignore and .dockerignore. This will prevent it from appearing in your code repository or images.

🚢 Captain Obvious says that you should not store any confidential secrets on your local computer in plain text files

To exclude all files starting with .env (e.g. .env.dev), you could add the following to both your .gitignoreand .dockerignore files...

# Exclude any environment variable dirs/files in root
.env*

That's it for this basic secrets management approach of using environment variables with GitHub Secrets in GitHub Actions. I hope that you find this useful. Even if you are not using GitHub Actions, you may find that your GitOps or CI/CD offers a similar ability to GitHub Secrets.


Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment