deploy_metrics.md

Measuring development practice

Aims

• Measure and monitor interval between production deploys
• Measure and monitor how long code takes between first local commit to being live on production
• Measure and monitor some metrics about pull requests
  • Total changes
  • Changes per commit

Dependencies

• Projects we want to measure need to expose an endpoint with the current SHA on production

Some of the repos

https://github.com/dxw/deployment-frequency

https://github.com/dxw/detect-a-deployment-hackday

https://github.com/dxw/production-lead-time

https://github.com/dxw/detect-a-deployment-hackday

https://github.com/dxw/measure-pr-stats

The Slack channel

https://dxw.slack.com/archives/C02FF56AFFV

Tooling we can use

• Oktokit Ruby or JS (it has a Gundam Octocat ❤️)
  • https://github.com/octokit/octokit.rb
  • https://github.com/octokit/octokit.js

Questions

• if we build a GitHUb action are we limiting where we can use this, CAEW for example does not use GA?
• if we use a GA javascript runs natively so is fast, but a large number of devs gravitate to Ruby which would have to run in a container - there is a decision here about the langauge
• can we stil use GA as Github knows when we pushed to the branch that triggers a deploy and then when that bulid completes?
• How far did we get?
• Based on the existing spiky work, are there ways we want to proceed and maybe ways we don’t want to choose?
• One repo or several?
• Which path do we choose for gathering information about projects and deployments?
• Poll a healthcheck to find out information about the current live version
• Write a GitHub Action that deployments are supposed to call to send info about the current live version
• Which of the aims do we take on as a project first?

Thoughts

It feels like a good shape for this would be a custom action that we can just use, perhaps before and after actions that run around the deploy? It might look a little like this pseudo code in use:

on:
  push:
    branches: [ main ]
jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - name: PR stats
        uses: dxw-actions/pr-stats@v1
        
      - name: Before deploy
        uses: dxw-actions/before-deploy@v1
        
      - name: Checkout code
        uses: actions/checkout@v2

      - name: Deploy
        id: deploy
        run: |
          echo "faking a deploy"
          sleep 5
          echo "fake deploy finished"
          
      - name: After deploy
        uses: dxw-actions/after-deploy@v1 

Our three goals here are then loosly contained in acitons:

pr stats

• collects stats on the PR, see https://github.com/dxw/measure-pr-stats/blob/main/get-pr-commits.rb

before deploy

• collects the start time and passes it on for later, see https://github.com/dxw/deployment-frequency

after deploy

• collects the stop time and writes everything to Influx, see https://github.com/dxw/deployment-frequency