State runner on salt-api POST from GitHub push hook

0-STATE-RUN-SALT-API-GITHUB-HOOK-USE-CASE.md

Validate and run a state when a valid GitHub hook call has been received from salt-api

Idea is that we can setup salt-api to receive hook call from GitHub, and configured run stat.sls only if the request HMAC signature matche is successful.

Unfortunately most documentation says to deactivate salt-api hooks authentication (i.e. webhook_disable_auth: True) which is not a good idea.

This Gist is about finding a way to declare which state to run based on data GitHub sends on push hook. But ONLY if the request is valid.

Skeleton defines desired logic, see reactor_github_push.py below.

Problem is that I have next to no experience in Python and it goes beyond my skill set.

Help would be much appreciated.

Reproduce a POST

curl -XPOST -ski https://128.52.178.67:8080/hook/github/push \
            -H "Content-Type: application/json"\
            -H "User-Agent: GitHub-Hookshot/renoirb"\
            -H "X-Github-Delivery: bf9d3700-ec39-11e4-87ec-abf8ab3d9134"\
            -H "X-Github-Event: push"\
            -H "X-Hub-Signature: sha1=756c2d03cdb736432072b61280194e51f11bd696"\
            --data @github_push_payload.json

Setup

Setup a reactor similar to:

# /etc/salt/master.d/reactor.conf
reactor:
  - 'salt/netapi/hook/github/push':
    - /srv/salt/reactor/github/push.sls'
    
rest_cherrypy:
  port: 8080
  ssl_crt: /etc/pki/tls/certs/128.52.178.67.crt
  ssl_key: /etc/pki/tls/certs/128.52.178.67.key
  webhook_disable_auth: True

github_push_payload.json

{
  "after": "edc70f85adb425c066f183b46879c5215147bb53",
  "base_ref": null,
  "before": "68b1264dc533b487c0618abffd543c86506e7d78",
  "commits": [
      {
          "added": [],
          "author": {
              "email": "renoir@w3.org",
              "name": "Renoir Boulanger",
              "username": "renoirb"
          },
          "committer": {
              "email": "renoir@w3.org",
              "name": "Renoir Boulanger",
              "username": "renoirb"
          },
          "distinct": true,
          "id": "edc70f85adb425c066f183b46879c5215147bb53",
          "message": "Testing hook on IRC 6",
          "modified": [
              "README.md"
          ],
          "removed": [],
          "timestamp": "2015-04-26T13:29:05-04:00",
          "url": "https://github.com/renoirb/renoirboulanger-styleguide/commit/edc70f85adb425c066f183b46879c5215147bb53"
      }
  ],
  "compare": "https://github.com/renoirb/renoirboulanger-styleguide/compare/68b1264dc533...edc70f85adb4",
  "created": false,
  "deleted": false,
  "forced": false,
  "head_commit": {
      "added": [],
      "author": {
          "email": "renoir@w3.org",
          "name": "Renoir Boulanger",
          "username": "renoirb"
      },
      "committer": {
          "email": "renoir@w3.org",
          "name": "Renoir Boulanger",
          "username": "renoirb"
      },
      "distinct": true,
      "id": "edc70f85adb425c066f183b46879c5215147bb53",
      "message": "Testing hook on IRC 6",
      "modified": [
          "README.md"
      ],
      "removed": [],
      "timestamp": "2015-04-26T13:29:05-04:00",
      "url": "https://github.com/renoirb/renoirboulanger-styleguide/commit/edc70f85adb425c066f183b46879c5215147bb53"
  },
  "pusher": {
      "email": "hello@renoirboulanger.com",
      "name": "renoirb"
  },
  "ref": "refs/heads/master",
  "repository": {
      "archive_url": "https://api.github.com/repos/renoirb/renoirboulanger-styleguide/{archive_format}{/ref}",
      "assignees_url": "https://api.github.com/repos/renoirb/renoirboulanger-styleguide/assignees{/user}",
      "blobs_url": "https://api.github.com/repos/renoirb/renoirboulanger-styleguide/git/blobs{/sha}",
      "branches_url": "https://api.github.com/repos/renoirb/renoirboulanger-styleguide/branches{/branch}",
      "clone_url": "https://github.com/renoirb/renoirboulanger-styleguide.git",
      "collaborators_url": "https://api.github.com/repos/renoirb/renoirboulanger-styleguide/collaborators{/collaborator}",
      "comments_url": "https://api.github.com/repos/renoirb/renoirboulanger-styleguide/comments{/number}",
      "commits_url": "https://api.github.com/repos/renoirb/renoirboulanger-styleguide/commits{/sha}",
      "compare_url": "https://api.github.com/repos/renoirb/renoirboulanger-styleguide/compare/{base}...{head}",
      "contents_url": "https://api.github.com/repos/renoirb/renoirboulanger-styleguide/contents/{+path}",
      "contributors_url": "https://api.github.com/repos/renoirb/renoirboulanger-styleguide/contributors",
      "created_at": 1372165967,
      "default_branch": "master",
      "description": "renoirboulanger.com static workspace version using Grunt, RoughDraft.js and automated deployment and minification",
      "downloads_url": "https://api.github.com/repos/renoirb/renoirboulanger-styleguide/downloads",
      "events_url": "https://api.github.com/repos/renoirb/renoirboulanger-styleguide/events",
      "fork": false,
      "forks": 0,
      "forks_count": 0,
      "forks_url": "https://api.github.com/repos/renoirb/renoirboulanger-styleguide/forks",
      "full_name": "renoirb/renoirboulanger-styleguide",
      "git_commits_url": "https://api.github.com/repos/renoirb/renoirboulanger-styleguide/git/commits{/sha}",
      "git_refs_url": "https://api.github.com/repos/renoirb/renoirboulanger-styleguide/git/refs{/sha}",
      "git_tags_url": "https://api.github.com/repos/renoirb/renoirboulanger-styleguide/git/tags{/sha}",
      "git_url": "git://github.com/renoirb/renoirboulanger-styleguide.git",
      "has_downloads": true,
      "has_issues": true,
      "has_pages": false,
      "has_wiki": true,
      "homepage": "https://renoirboulanger.com/styleguide2/",
      "hooks_url": "https://api.github.com/repos/renoirb/renoirboulanger-styleguide/hooks",
      "html_url": "https://github.com/renoirb/renoirboulanger-styleguide",
      "id": 10938692,
      "issue_comment_url": "https://api.github.com/repos/renoirb/renoirboulanger-styleguide/issues/comments{/number}",
      "issue_events_url": "https://api.github.com/repos/renoirb/renoirboulanger-styleguide/issues/events{/number}",
      "issues_url": "https://api.github.com/repos/renoirb/renoirboulanger-styleguide/issues{/number}",
      "keys_url": "https://api.github.com/repos/renoirb/renoirboulanger-styleguide/keys{/key_id}",
      "labels_url": "https://api.github.com/repos/renoirb/renoirboulanger-styleguide/labels{/name}",
      "language": "JavaScript",
      "languages_url": "https://api.github.com/repos/renoirb/renoirboulanger-styleguide/languages",
      "master_branch": "master",
      "merges_url": "https://api.github.com/repos/renoirb/renoirboulanger-styleguide/merges",
      "milestones_url": "https://api.github.com/repos/renoirb/renoirboulanger-styleguide/milestones{/number}",
      "mirror_url": null,
      "name": "renoirboulanger-styleguide",
      "notifications_url": "https://api.github.com/repos/renoirb/renoirboulanger-styleguide/notifications{?since,all,participating}",
      "open_issues": 0,
      "open_issues_count": 0,
      "owner": {
          "email": "hello@renoirboulanger.com",
          "name": "renoirb"
      },
      "private": false,
      "pulls_url": "https://api.github.com/repos/renoirb/renoirboulanger-styleguide/pulls{/number}",
      "pushed_at": 1430069350,
      "releases_url": "https://api.github.com/repos/renoirb/renoirboulanger-styleguide/releases{/id}",
      "size": 864,
      "ssh_url": "git@github.com:renoirb/renoirboulanger-styleguide.git",
      "stargazers": 1,
      "stargazers_count": 1,
      "stargazers_url": "https://api.github.com/repos/renoirb/renoirboulanger-styleguide/stargazers",
      "statuses_url": "https://api.github.com/repos/renoirb/renoirboulanger-styleguide/statuses/{sha}",
      "subscribers_url": "https://api.github.com/repos/renoirb/renoirboulanger-styleguide/subscribers",
      "subscription_url": "https://api.github.com/repos/renoirb/renoirboulanger-styleguide/subscription",
      "svn_url": "https://github.com/renoirb/renoirboulanger-styleguide",
      "tags_url": "https://api.github.com/repos/renoirb/renoirboulanger-styleguide/tags",
      "teams_url": "https://api.github.com/repos/renoirb/renoirboulanger-styleguide/teams",
      "trees_url": "https://api.github.com/repos/renoirb/renoirboulanger-styleguide/git/trees{/sha}",
      "updated_at": "2015-04-26T07:49:50Z",
      "url": "https://github.com/renoirb/renoirboulanger-styleguide",
      "watchers": 1,
      "watchers_count": 1
  },
  "sender": {
      "avatar_url": "https://avatars.githubusercontent.com/u/296940?v=3",
      "events_url": "https://api.github.com/users/renoirb/events{/privacy}",
      "followers_url": "https://api.github.com/users/renoirb/followers",
      "following_url": "https://api.github.com/users/renoirb/following{/other_user}",
      "gists_url": "https://api.github.com/users/renoirb/gists{/gist_id}",
      "gravatar_id": "",
      "html_url": "https://github.com/renoirb",
      "id": 296940,
      "login": "renoirb",
      "organizations_url": "https://api.github.com/users/renoirb/orgs",
      "received_events_url": "https://api.github.com/users/renoirb/received_events",
      "repos_url": "https://api.github.com/users/renoirb/repos",
      "site_admin": false,
      "starred_url": "https://api.github.com/users/renoirb/starred{/owner}{/repo}",
      "subscriptions_url": "https://api.github.com/users/renoirb/subscriptions",
      "type": "User",
      "url": "https://api.github.com/users/renoirb"
  }
}

reactor_github_push.py

#!py

# Figure out how to get this from pillar or grains #TODO
shared_secretkey = 'foobarbazz'


import json
import hmac


def run():
  """
  Read a payload from GitHub push hook and validate it

  See also:
  - http://bencane.com/2014/07/17/integrating-saltstack-with-other-services-via-salt-api/
  - http://bencane.com/2014/12/30/building-self-healing-applications-with-salt-api/
  - https://developer.github.com/webhooks/securing/
  - https://github.com/saltstack-formulas/ec2-autoscale-reactor/blob/master/reactor/ec2-autoscale.sls
  - http://salt-api.readthedocs.org/en/latest/ref/netapis/all/saltapi.netapi.rest_cherrypy.html
  - http://docs.saltstack.com/en/latest/ref/netapi/all/salt.netapi.rest_cherrypy.html
  """
  payload = data.get('post', {})
  payload_json = json.load(payload)
  # GitHub also sends:
  #   - X-Github-Signature: sha1=...
  #   - X-Github-Event: push
  payload_sig = data.get('headers', {}).get('X-Github-Delivery') 
  digest_handler = hmac.new(shared_secretkey)
  digest = digest_handler.hexdigest()

  # Check if payload_sig === digest, otherwise do nothing

  # Request is coming from something that knew what to send and how to sign it, launch state.sls runner

  # Get payload.repository.name (e.g. "mygithubproject")
  # Get payload.owner.name (e.g. "renoirb")
  # Call state.sls method based on a Mapping.

See also the thread in this Pull request