Grab projects and contexts that have environment variables

README.txt

To run the script locally:

1. Set an environment variable named `GH_TOKEN` populated with your GitHub personal access token (https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token)
    The scope of this GitHub personal access token needs to be:
    - `repo`, if you have both private and public repos
    or
    - `repo:status`, if you only have public repos

2. Set an environment variable named `CIRCLECI_API_TOKEN` populated with your CircleCI personal API token (https://circleci.com/docs/managing-api-tokens/#creating-a-personal-api-token)

3. Set an environment variable named `ORG_NAME` populated with your organization name

CAVEATS & LIMITATIONS
The current version of this script only works for GitHub; it does not support Bitbucket or GitLab.

secrets-grabber.sh

# #!/bin/bash

if [ $(curl -s https://api.github.com/orgs/$ORG_NAME -H "Authorization: bearer $GH_TOKEN" -I -w "%{http_code}" -o /dev/null) -eq 200 ]; then
  FETCH_REPO_GH_ENDPT="https://api.github.com/orgs/$ORG_NAME/repos?per_page=100"
elif [ $(curl -s "https://api.github.com/user" -H "Authorization: bearer $GH_TOKEN"|jq -r '.login') = "$ORG_NAME" ] && [ $(curl -s "https://api.github.com/users/$ORG_NAME" -H "Authorization: bearer $GH_TOKEN" -I -w "%{http_code}" -o /dev/null) -eq 200 ]; then
  FETCH_REPO_GH_ENDPT="https://api.github.com/user/repos?type=owner&per_page=100"
else
  printf "Couldn't find or access the GitHub organization/account %s% \n" $ORG_NAME
  printf "Make sure that: \n"
  printf " - You have set the correct name in the \`ORG_NAME\` environement variable  \n"
  printf " - The GitHub personal access token has the appropriate scope  \n"
  printf " - If you are running this script for a CircleCI organization associated with a personal GitHub account, make sure the GitHub personal access token you set in the \`GH_TOKEN\` environement variable was generated under that same GitHub user account  \n"
  exit 1
fi

#### Get a list of the organization's project
REPOS_PAGE=1

curl -s -G "$FETCH_REPO_GH_ENDPT" -H "Authorization: bearer $GH_TOKEN" | jq -r '.[].full_name' > repos-$REPOS_PAGE.txt


if [ $(wc -l < repos-$REPOS_PAGE.txt) -eq 100 ]; then
  while [ $(wc -l < repos-$REPOS_PAGE.txt) -eq 100 ]
    do
      ((REPOS_PAGE++))
      #curl -s -G "https://api.github.com/orgs/$ORG_NAME/repos?per_page=100&page=$REPOS_PAGE" -H "Authorization: bearer $GH_TOKEN" | jq -r '.[].full_name' > repos-$REPOS_PAGE.txt
      curl -s -G "$$FETCH_REPO_GH_ENDPT&page=$REPOS_PAGE" -H "Authorization: bearer $GH_TOKEN" | jq -r '.[].full_name' > repos-$REPOS_PAGE.txt
  done
fi

  cat repos-*.txt > repos.txt


#### For each of these project, determine if there are environment variables
for PROJECT in $(cat repos.txt)
  do
    curl -s -G "https://circleci.com/api/v2/project/gh/$PROJECT/envvar" -H "circle-token: $CIRCLECI_API_TOKEN" > env-vars.json
      if [ $(jq '.items|length' env-vars.json) -gt 0 ]; then
        echo "Project $PROJECT has $(jq '.items|length' env-vars.json) environment variables --> https://app.circleci.com/settings/project/gh/$PROJECT/environment-variables"
        printf "\n"
      fi
done


#### For each of these project, determine if there are project API tokens
for PROJECT in $(cat repos.txt)
  do curl -s -G "https://circleci.com/api/v1.1/project/gh/$PROJECT/token" -H "circle-token: $CIRCLECI_API_TOKEN" > project-tokens.json
      if [ $(jq '.|length' project-tokens.json) -gt 0 ]; then
        echo "Project $PROJECT has $(jq '.|length' project-tokens.json) project tokens --> https://app.circleci.com/settings/project/gh/$PROJECT/api"
        printf "\n"
      fi
done

rm -f repos*.txt
rm -f env-vars.json
rm -f project-tokens.json

#### Get a list of the organization's contexts
curl -s -G "https://circleci.com/api/v2/context?owner-slug=gh/$ORG_NAME" -H "circle-token: $CIRCLECI_API_TOKEN" > contexts-list-1.json
PAGE_TOKEN=$(jq -r '.next_page_token' contexts-list-1.json)
CONTEXTS_PAGE=1
if [[ "$PAGE_TOKEN" != "null" ]]; then
  while [[ "$PAGE_TOKEN" != "null" ]]
   do
     ((CONTEXTS_PAGE++))
     curl -s -G "https://circleci.com/api/v2/context?owner-slug=gh/$ORG_NAME&page-token=$PAGE_TOKEN" -H "circle-token: $CIRCLECI_API_TOKEN" > contexts-list-$CONTEXTS_PAGE.json
     PAGE_TOKEN=$(jq -r '.next_page_token' contexts-list-$CONTEXTS_PAGE.json)
  done
fi
cat contexts-list-*.json > contexts-list.json
CONTEXT_IDS=$(jq -r '.items[].id' contexts-list.json)

#### For each of these contexts, determine if there are environment variables
for CONTEXT in $CONTEXT_IDS
  do
    curl -s -G "https://circleci.com/api/v2/context/$CONTEXT/environment-variable" -H "circle-token: $CIRCLECI_API_TOKEN" > context-$CONTEXT-env-vars.json
    if [[ $(jq '.items|length' context-$CONTEXT-env-vars.json) -gt 0 ]]; then
      echo "Context with ID \`$CONTEXT\` has $(jq '.items|length' context-$CONTEXT-env-vars.json) environment variables --> https://app.circleci.com/settings/organization/gh/$ORG_NAME/contexts/$CONTEXT"
      printf "\n"
    fi
done

rm -f contexts-list*.json
rm -f context-*-env-vars.json

Regarding #1 in the readme.

Small gotcha:
If Resource is protected by organization SAML enforcement. You must grant your Personal Access token access to those organizations.
https://docs.github.com/articles/authenticating-to-a-github-organization-with-saml-single-sign-on/

Also if your company has more than one org you have to run this script more than once with different org names.

Hi, which permissions should I set on my GH_TOKEN?

It's only listing one project when running the script

@u-santos

Hi, which permissions should I set on my GH_TOKEN?

The repo:status scope is sufficient if you only have public repos. Otherwise, you'll need the repo scope. (https://docs.github.com/en/developers/apps/building-oauth-apps/scopes-for-oauth-apps#available-scopes)

It's only listing one project when running the script

This either means that:

• the CircleCI project in question is the only one that at least one environment variable declared in the project settings. (https://circleci.com/docs/set-environment-variable/#set-an-environment-variable-in-a-project)
• we were not able to retrieve the name other repositories for which a CircleCI project exists and might have environment variables declared in its settings; this would happen if you have more than 100 repositories. (Note: I just modified the call to the GitHub API so it returns the maximum number of repositories, which is 100 > https://docs.github.com/en/rest/repos/repos?apiVersion=2022-11-28#list-organization-repositories)

The script has now been updated, and handles the GitHub API pagination.

All your repositories will now be retrieved, as opposed to the maximum of 100 previously.

Improvements

The latest version of the script:

• Supports GitHub personal accounts
• Identifies projects that have project API token(s)