branch-protection.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "a720834f-01ef-44b4-a678-bc4f7b7277bd",
   "metadata": {},
   "source": [
    "# Bulk-add branch protection rules\n",
    "\n",
    "Going through all jupyterhub repos and adding the most basic branch protection rule for `main`"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a48d2345-f2f4-4577-85c6-7f03d0c94462",
   "metadata": {},
   "source": [
    "I use a token stored in my keychain.\n",
    "I have two of these, one read-only and one with full access.\n",
    "\n",
    "You can create similar tokens with:\n",
    "\n",
    "- visit https://github.com/settings/personal-access-tokens/new\n",
    "- store with:\n",
    "\n",
    "  ```\n",
    "  python -m keyring set api.github.com read-only\n",
    "  ```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "6d398667-8325-4b92-a3b8-e650c2a680e4",
   "metadata": {},
   "outputs": [],
   "source": [
    "import requests\n",
    "import keyring\n",
    "s = requests.Session()\n",
    "s.headers['Authorization'] = f\"bearer {keyring.get_password('api.github.com', 'full-access')}\"\n",
    "s.headers['X-Github-Next-Global-ID'] = '1'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "8fec2ce5-7c9a-48a3-9a40-659c1c64e326",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'minrk'"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s.get(\"https://api.github.com/user\").json()['login']"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "86ab70d4-ed3f-4b77-84bb-545193fc8b85",
   "metadata": {},
   "source": [
    "## Step 1: collect repos\n",
    "\n",
    "Use graphQL endpoint to collect repositories\n",
    "\n",
    "1. only non-forks\n",
    "2. only need:\n",
    "   - default branch name\n",
    "   - if there is an existing branch protection rule\n",
    "   - repo's global id for use in mutation later"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "aa17caa3-f116-4fb8-a0e2-9557400285d6",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "ca48d4528e8f459b848e392019255d4e",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "fetching: 0repos [00:00, ?repos/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from jinja2 import Template\n",
    "import json\n",
    "from tqdm.notebook import tqdm\n",
    "\n",
    "query_template = Template(\"\"\"\n",
    "{\n",
    "  organization(login:\"jupyterhub\") {\n",
    "#  user(login: \"minrk\") {\n",
    "    repositories(isFork: false, ownerAffiliations: OWNER, first: 100{%- if after %}, after: \"{{ after }}\"{%- endif %}) {\n",
    "      totalCount\n",
    "      pageInfo {\n",
    "        endCursor\n",
    "        hasNextPage\n",
    "      }\n",
    "      nodes {\n",
    "        id\n",
    "        name\n",
    "        defaultBranchRef {\n",
    "          name\n",
    "        }\n",
    "        branchProtectionRules(first: 5) {\n",
    "          nodes {\n",
    "            pattern\n",
    "            matchingRefs(first: 5) {\n",
    "              nodes {\n",
    "                name\n",
    "              }\n",
    "            }\n",
    "          }\n",
    "        }\n",
    "      }\n",
    "    }\n",
    "  }\n",
    "}\n",
    "\"\"\")\n",
    "\n",
    "github_graphql = \"https://api.github.com/graphql\"\n",
    "\n",
    "def fetch_repos():\n",
    "    \n",
    "    after = None\n",
    "    repos = []\n",
    "    has_next_page = True\n",
    "    progress = tqdm(desc=\"fetching\", unit=\"repos\")\n",
    "    while has_next_page:\n",
    "        r = s.post(github_graphql, data=json.dumps(dict(query=query_template.render(after=after))))\n",
    "        r.raise_for_status()\n",
    "        response = r.json()[\"data\"][\"organization\"][\"repositories\"]\n",
    "        progress.total = response[\"totalCount\"]\n",
    "        progress.update(len(response['nodes']))\n",
    "        repos.extend(response['nodes'])\n",
    "        after = response['pageInfo']['endCursor']\n",
    "        has_next_page = response['pageInfo']['hasNextPage']\n",
    "    progress.close()\n",
    "    return repos\n",
    "repos = fetch_repos()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "24017f32-95ca-48ab-afec-3b42cef16c2c",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'id': 'R_kgDOAT0rQg',\n",
       " 'name': 'jupyterhub',\n",
       " 'defaultBranchRef': {'name': 'main'},\n",
       " 'branchProtectionRules': {'nodes': []}}"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "repos[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2661f52f-1056-41f2-b5a8-ae00716724f8",
   "metadata": {},
   "source": [
    "## Step 2: select which repos to protect\n",
    "\n",
    "Iterate through repos, finding:\n",
    "\n",
    "1. if default branch is not main, note that it still needs a rename\n",
    "2. if the default branch is not protected, add repo to list of repos to protect"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "9194ded3-7409-4136-806f-bd66c981e0a8",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "jupyterhub\n",
      " Has rule: main\n",
      "configurable-http-proxy\n",
      " Has rule: main\n",
      "oauthenticator\n",
      " Has rule: main\n",
      "dockerspawner\n",
      " Has rule: main\n",
      "sudospawner\n",
      "  non-main default branch: master\n",
      "  Needs rule\n",
      "batchspawner\n",
      "  non-main default branch: master\n",
      "  Needs rule\n",
      "kubespawner\n",
      " Has rule: main\n",
      "ldapauthenticator\n",
      "  non-main default branch: master\n",
      "  Needs rule\n",
      "dummyauthenticator\n",
      "  non-main default branch: master\n",
      "  Needs rule\n",
      "simplespawner\n",
      "  non-main default branch: master\n",
      "  Needs rule\n",
      "jupyterhub-deploy-docker\n",
      "  non-main default branch: master\n",
      "  Needs rule\n",
      "jupyterhub-deploy-teaching\n",
      "  non-main default branch: master\n",
      "  Needs rule\n",
      "jupyterhub-tutorial\n",
      "  non-main default branch: master\n",
      "  Needs rule\n",
      "jupyterhub-deploy-hpc\n",
      "  non-main default branch: master\n",
      "  Needs rule\n",
      "systemdspawner\n",
      "  non-main default branch: master\n",
      "  Needs rule\n",
      "wrapspawner\n",
      "  non-main default branch: master\n",
      "  Needs rule\n",
      "jupyterlab-hub\n",
      "  non-main default branch: master\n",
      "  Needs rule\n",
      "jupyter-server-proxy\n",
      " Has rule: main\n",
      "firstuseauthenticator\n",
      " Has rule: main\n",
      "jupyterhub-example-kerberos\n",
      "  non-main default branch: master\n",
      "  Needs rule\n",
      "hubshare\n",
      "  non-main default branch: master\n",
      "  Needs rule\n",
      "jupyter-rsession-proxy\n",
      "  non-main default branch: master\n",
      "  Needs rule\n",
      "tmpauthenticator\n",
      "  non-main default branch: master\n",
      "  Needs rule\n",
      "zero-to-jupyterhub-k8s\n",
      " Has rule: main\n",
      "helm-chart\n",
      "  non-main default branch: master\n",
      "  Needs rule\n",
      "binderhub\n",
      " Has rule: main\n",
      "repo2docker\n",
      " Has rule: main\n",
      "mybinder.org-deploy\n",
      "  non-main default branch: master\n",
      " Has rule: master\n",
      "nbgitpuller\n",
      " Has rule: main\n",
      "mybinder.org-user-guide\n",
      "  non-main default branch: master\n",
      "  Needs rule\n",
      "nullauthenticator\n",
      "  non-main default branch: master\n",
      "  Needs rule\n",
      "team-compass\n",
      " Has rule: main\n",
      "ltiauthenticator\n",
      " Has rule: main\n",
      "binder-data\n",
      "  non-main default branch: master\n",
      "  Needs rule\n",
      "binder-billing\n",
      "  non-main default branch: master\n",
      "  Needs rule\n",
      "jhub-proposals\n",
      "  non-main default branch: master\n",
      "  Needs rule\n",
      "chartpress\n",
      " Has rule: main\n",
      "mybinder-tools\n",
      "  non-main default branch: master\n",
      "  Needs rule\n",
      "the-littlest-jupyterhub\n",
      " Has rule: main\n",
      "design\n",
      "  non-main default branch: master\n",
      "  Needs rule\n",
      "research-facilities\n",
      "  non-main default branch: master\n",
      "  Needs rule\n",
      "outreachy\n",
      " Has rule: main\n",
      "alabaster-jupyterhub\n",
      "  non-main default branch: master\n",
      "  Needs rule\n",
      "traefik-proxy\n",
      "  non-main default branch: master\n",
      "  Needs rule\n",
      "nativeauthenticator\n",
      " Has rule: main\n",
      "yarnspawner\n",
      "  non-main default branch: master\n",
      "  Needs rule\n",
      "simpervisor\n",
      " Has rule: main\n",
      "jupyterhub-on-hadoop\n",
      "  non-main default branch: master\n",
      "  Needs rule\n",
      "kerberosauthenticator\n",
      "  non-main default branch: master\n",
      "  Needs rule\n",
      "jupyterhub-the-hard-way\n",
      "  non-main default branch: master\n",
      "  Needs rule\n",
      "autodoc-traits\n",
      " Has rule: main\n",
      "jupyter-remote-desktop-proxy\n",
      " Has rule: main\n",
      "repo2docker-action\n",
      "  non-main default branch: master\n",
      "  Needs rule\n",
      ".github\n",
      "  non-main default branch: master\n",
      "  Needs rule\n",
      "jupyterhub-idle-culler\n",
      " Has rule: main\n",
      "pebble-helm-chart\n",
      " Has rule: main\n",
      "action-k3s-helm\n",
      " Has rule: main\n",
      "action-major-minor-tag-calculator\n",
      " Has rule: main\n",
      "grafana-dashboards\n",
      " Has rule: main\n",
      "action-k8s-namespace-report\n",
      " Has rule: main\n",
      "action-k8s-await-workloads\n",
      " Has rule: main\n",
      "katacoda-scenarios\n",
      " Has rule: main\n",
      "docker-image-cleaner\n",
      " Has rule: main\n",
      "nbgitpuller-downloader-googledrive\n",
      " Has rule: main\n",
      "nbgitpuller-downloader-dropbox\n",
      " Has rule: main\n",
      "nbgitpuller-downloader-generic-web\n",
      " Has rule: main\n",
      "nbgitpuller-downloader-plugins\n",
      " Has rule: main\n",
      "gh-scoped-creds\n",
      " Has rule: main\n",
      "68 total repos\n",
      "34 repos missing main branch\n",
      "35 repos with protected main\n",
      "33 repos will get new protection rules\n"
     ]
    }
   ],
   "source": [
    "to_protect = []\n",
    "no_main = []\n",
    "is_protected = []\n",
    "\n",
    "for repo in repos:\n",
    "    print(repo['name'])\n",
    "    if repo['defaultBranchRef'] is None:\n",
    "        print(\"  Empty repo!\")\n",
    "        continue\n",
    "    default_branch = repo['defaultBranchRef']['name']\n",
    "    if default_branch != 'main':\n",
    "        print(f\"  non-main default branch: {default_branch}\")\n",
    "        no_main.append(repo)\n",
    "\n",
    "    rules = repo['branchProtectionRules']['nodes']\n",
    "    matched = False\n",
    "    for rule in rules:\n",
    "        if default_branch in [ref['name'] for ref in rule['matchingRefs']['nodes']]:\n",
    "            print(f\" Has rule: {rule['pattern']}\")\n",
    "            matched = True\n",
    "            is_protected.append(repo)\n",
    "            break\n",
    "\n",
    "    if not matched:\n",
    "        print(f\"  Needs rule\")\n",
    "        to_protect.append(repo)\n",
    "\n",
    "print(f\"{len(repos)} total repos\")\n",
    "print(f\"{len(no_main)} repos missing main branch\")\n",
    "print(f\"{len(is_protected)} repos with protected main\")\n",
    "print(f\"{len(to_protect)} repos will get new protection rules\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "ab540f2f-7ccc-431c-8dc9-3e3682bf70b3",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "mutation {\n",
      "  \n",
      "  sudospawner: createBranchProtectionRule(input: {repositoryId: \"R_kgDOAZ2C_g\", pattern: \"master\"}) {\n",
      "    clientMutationId\n",
      "  }\n",
      "  \n",
      "  batchspawner: createBranchProtectionRule(input: {repositoryId: \"R_kgDOAq7ijQ\", pattern: \"master\"}) {\n",
      "    clientMutationId\n",
      "  }\n",
      "  \n",
      "  ldapauthenticator: createBranchProtectionRule(input: {repositoryId: \"R_kgDOAvfy_A\", pattern: \"master\"}) {\n",
      "    clientMutationId\n",
      "  }\n",
      "  \n",
      "  dummyauthenticator: createBranchProtectionRule(input: {repositoryId: \"R_kgDOAxUgLA\", pattern: \"master\"}) {\n",
      "    clientMutationId\n",
      "  }\n",
      "  \n",
      "  simplespawner: createBranchProtectionRule(input: {repositoryId: \"R_kgDOAxVTvw\", pattern: \"master\"}) {\n",
      "    clientMutationId\n",
      "  }\n",
      "  \n",
      "  jupyterhubdeploydocker: createBranchProtectionRule(input: {repositoryId: \"R_kgDOA1otfA\", pattern: \"master\"}) {\n",
      "    clientMutationId\n",
      "  }\n",
      "  \n",
      "  jupyterhubdeployteaching: createBranchProtectionRule(input: {repositoryId: \"R_kgDOA1oucQ\", pattern: \"master\"}) {\n",
      "    clientMutationId\n",
      "  }\n",
      "  \n",
      "  jupyterhubtutorial: createBranchProtectionRule(input: {re\n"
     ]
    }
   ],
   "source": [
    "mutation_template = Template(\"\"\"\n",
    "mutation {\n",
    "  {% for repo in repos %}\n",
    "  {{ repo['name'] | replace(\"-\", \"\") | replace(\".\",\"\") }}: createBranchProtectionRule(input: {repositoryId: \"{{ repo['id'] }}\", pattern: \"{{ repo['defaultBranchRef']['name'] }}\"}) {\n",
    "    clientMutationId\n",
    "  }\n",
    "  {% endfor %}\n",
    "}\n",
    "\"\"\") \n",
    "print(mutation_template.render(repos=to_protect)[:1024])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "d9e4396c-3526-42bd-a62d-3bd335b43d3e",
   "metadata": {},
   "outputs": [],
   "source": [
    "r = s.post(github_graphql, data=json.dumps(dict(query=mutation_template.render(repos=to_protect))))\n",
    "r.raise_for_status()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "71a6886b-af11-4185-8b99-a4ab7ddcc1ff",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'data': {'sudospawner': {'clientMutationId': None},\n",
       "  'batchspawner': {'clientMutationId': None},\n",
       "  'ldapauthenticator': {'clientMutationId': None},\n",
       "  'dummyauthenticator': {'clientMutationId': None},\n",
       "  'simplespawner': {'clientMutationId': None},\n",
       "  'jupyterhubdeploydocker': {'clientMutationId': None},\n",
       "  'jupyterhubdeployteaching': {'clientMutationId': None},\n",
       "  'jupyterhubtutorial': {'clientMutationId': None},\n",
       "  'jupyterhubdeployhpc': {'clientMutationId': None},\n",
       "  'systemdspawner': {'clientMutationId': None},\n",
       "  'wrapspawner': {'clientMutationId': None},\n",
       "  'jupyterlabhub': {'clientMutationId': None},\n",
       "  'jupyterhubexamplekerberos': {'clientMutationId': None},\n",
       "  'hubshare': {'clientMutationId': None},\n",
       "  'jupyterrsessionproxy': {'clientMutationId': None},\n",
       "  'tmpauthenticator': {'clientMutationId': None},\n",
       "  'helmchart': {'clientMutationId': None},\n",
       "  'mybinderorguserguide': {'clientMutationId': None},\n",
       "  'nullauthenticator': {'clientMutationId': None},\n",
       "  'binderdata': {'clientMutationId': None},\n",
       "  'binderbilling': {'clientMutationId': None},\n",
       "  'jhubproposals': {'clientMutationId': None},\n",
       "  'mybindertools': {'clientMutationId': None},\n",
       "  'design': {'clientMutationId': None},\n",
       "  'researchfacilities': {'clientMutationId': None},\n",
       "  'alabasterjupyterhub': {'clientMutationId': None},\n",
       "  'traefikproxy': {'clientMutationId': None},\n",
       "  'yarnspawner': {'clientMutationId': None},\n",
       "  'jupyterhubonhadoop': {'clientMutationId': None},\n",
       "  'kerberosauthenticator': {'clientMutationId': None},\n",
       "  'jupyterhubthehardway': {'clientMutationId': None},\n",
       "  'repo2dockeraction': {'clientMutationId': None},\n",
       "  'github': {'clientMutationId': None}}}"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "r.json()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}