Grab projects and contexts that have environment variables

README.txt

To run the script locally:

1. Set an environment variable named `GH_TOKEN` populated with your GitHub personal access token (https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token)
    The scope of this GitHub personal access token needs to be:
    - `repo`, if you have both private and public repos
    or
    - `repo:status`, if you only have public repos

2. Set an environment variable named `CIRCLECI_API_TOKEN` populated with your CircleCI personal API token (https://circleci.com/docs/managing-api-tokens/#creating-a-personal-api-token)

3. Set an environment variable named `ORG_NAME` populated with your organization name

CAVEATS & LIMITATIONS
The current version of this script only works for GitHub; it does not support Bitbucket or GitLab.

secrets-grabber.sh

# #!/bin/bash

if [ $(curl -s https://api.github.com/orgs/$ORG_NAME -H "Authorization: bearer $GH_TOKEN" -I -w "%{http_code}" -o /dev/null) -eq 200 ]; then
  FETCH_REPO_GH_ENDPT="https://api.github.com/orgs/$ORG_NAME/repos?per_page=100"
elif [ $(curl -s "https://api.github.com/user" -H "Authorization: bearer $GH_TOKEN"|jq -r '.login') = "$ORG_NAME" ] && [ $(curl -s "https://api.github.com/users/$ORG_NAME" -H "Authorization: bearer $GH_TOKEN" -I -w "%{http_code}" -o /dev/null) -eq 200 ]; then
  FETCH_REPO_GH_ENDPT="https://api.github.com/user/repos?type=owner&per_page=100"
else
  printf "Couldn't find or access the GitHub organization/account %s% \n" $ORG_NAME
  printf "Make sure that: \n"
  printf " - You have set the correct name in the \`ORG_NAME\` environement variable  \n"
  printf " - The GitHub personal access token has the appropriate scope  \n"
  printf " - If you are running this script for a CircleCI organization associated with a personal GitHub account, make sure the GitHub personal access token you set in the \`GH_TOKEN\` environement variable was generated under that same GitHub user account  \n"
  exit 1
fi

#### Get a list of the organization's project
REPOS_PAGE=1

curl -s -G "$FETCH_REPO_GH_ENDPT" -H "Authorization: bearer $GH_TOKEN" | jq -r '.[].full_name' > repos-$REPOS_PAGE.txt


if [ $(wc -l < repos-$REPOS_PAGE.txt) -eq 100 ]; then
  while [ $(wc -l < repos-$REPOS_PAGE.txt) -eq 100 ]
    do
      ((REPOS_PAGE++))
      #curl -s -G "https://api.github.com/orgs/$ORG_NAME/repos?per_page=100&page=$REPOS_PAGE" -H "Authorization: bearer $GH_TOKEN" | jq -r '.[].full_name' > repos-$REPOS_PAGE.txt
      curl -s -G "$$FETCH_REPO_GH_ENDPT&page=$REPOS_PAGE" -H "Authorization: bearer $GH_TOKEN" | jq -r '.[].full_name' > repos-$REPOS_PAGE.txt
  done
fi

  cat repos-*.txt > repos.txt


#### For each of these project, determine if there are environment variables
for PROJECT in $(cat repos.txt)
  do
    curl -s -G "https://circleci.com/api/v2/project/gh/$PROJECT/envvar" -H "circle-token: $CIRCLECI_API_TOKEN" > env-vars.json
      if [ $(jq '.items|length' env-vars.json) -gt 0 ]; then
        echo "Project $PROJECT has $(jq '.items|length' env-vars.json) environment variables --> https://app.circleci.com/settings/project/gh/$PROJECT/environment-variables"
        printf "\n"
      fi
done


#### For each of these project, determine if there are project API tokens
for PROJECT in $(cat repos.txt)
  do curl -s -G "https://circleci.com/api/v1.1/project/gh/$PROJECT/token" -H "circle-token: $CIRCLECI_API_TOKEN" > project-tokens.json
      if [ $(jq '.|length' project-tokens.json) -gt 0 ]; then
        echo "Project $PROJECT has $(jq '.|length' project-tokens.json) project tokens --> https://app.circleci.com/settings/project/gh/$PROJECT/api"
        printf "\n"
      fi
done

rm -f repos*.txt
rm -f env-vars.json
rm -f project-tokens.json

#### Get a list of the organization's contexts
curl -s -G "https://circleci.com/api/v2/context?owner-slug=gh/$ORG_NAME" -H "circle-token: $CIRCLECI_API_TOKEN" > contexts-list-1.json
PAGE_TOKEN=$(jq -r '.next_page_token' contexts-list-1.json)
CONTEXTS_PAGE=1
if [[ "$PAGE_TOKEN" != "null" ]]; then
  while [[ "$PAGE_TOKEN" != "null" ]]
   do
     ((CONTEXTS_PAGE++))
     curl -s -G "https://circleci.com/api/v2/context?owner-slug=gh/$ORG_NAME&page-token=$PAGE_TOKEN" -H "circle-token: $CIRCLECI_API_TOKEN" > contexts-list-$CONTEXTS_PAGE.json
     PAGE_TOKEN=$(jq -r '.next_page_token' contexts-list-$CONTEXTS_PAGE.json)
  done
fi
cat contexts-list-*.json > contexts-list.json
CONTEXT_IDS=$(jq -r '.items[].id' contexts-list.json)

#### For each of these contexts, determine if there are environment variables
for CONTEXT in $CONTEXT_IDS
  do
    curl -s -G "https://circleci.com/api/v2/context/$CONTEXT/environment-variable" -H "circle-token: $CIRCLECI_API_TOKEN" > context-$CONTEXT-env-vars.json
    if [[ $(jq '.items|length' context-$CONTEXT-env-vars.json) -gt 0 ]]; then
      echo "Context with ID \`$CONTEXT\` has $(jq '.items|length' context-$CONTEXT-env-vars.json) environment variables --> https://app.circleci.com/settings/organization/gh/$ORG_NAME/contexts/$CONTEXT"
      printf "\n"
    fi
done

rm -f contexts-list*.json
rm -f context-*-env-vars.json

The script has now been updated, and handles the GitHub API pagination.

All your repositories will now be retrieved, as opposed to the maximum of 100 previously.

Improvements

The latest version of the script:

• Supports GitHub personal accounts
• Identifies projects that have project API token(s)