blog.md

Lint checker comments on GitHub PR using Travis

I really like PEP8. It's a standard that's used pretty much everywhere in the python "world". It's job is to make sure that your code adheres to certain rules, for example no line of code should exceed 80 characters.

Recently I opened up a pull request to add a flake8 check to the snippets-service project at mozilla. Flake8 is a command line tool that "lints" your code against certain standards, most notibly PEP8. It's pretty straightforward adding flake8 checks to your build.

- before_install
  - pip install flake8
  - flake8 <project_name>

The problem with this configuration is that it will fail the build if there are any style errors or warnings and as the snippets-service project owner Micheal Kelly said:

I am not a fan of making a test suite fail due to style guidelines. There are plenty of good reasons to break PEP8, and it's a bit obnoxious to have to mark each instance just to get your tests to pass. I'm a fan of automatically running the style check against PRs though, just not having the test suite fail because of it.

(For what it's worth I personally like to fail builds on style builds but I live to please :P)

Dealing with theses requrements I checked out out flake8 and turns out that it has a flag that makes sure that it's exit code is always 0, and thus not failing the build, aptly named --exit-zero. That in combination of moving the flake8 run to the last of the travis config makes it so the tests doesn't report any pep8 into the github pull request but still runs in and outputs into the build.

Giorgos Logiotatidis, another snippets-service collobarator, chimed in:

Having a jenkins (or other service) to comment on PRs with pep8 warnings would be ideal, because otherwise you'll have to take action to see if there are any warnings. I used to work on imhotep and flake8 plugin. If that sounds interesting we can revive it.

I put my shower cap on, since I have my best thoughts in the shower, and came back to the PR with the following idea:

I was in the shower this morning and thinking to myself that I could probably write some small app in python/bash that I could execute on travis that would comment on the PR with the flake8 results. I'm looking into that possibility now and I might use this PR to test it so it might get noisy in here.

I went on to the GitHub API to get info on how to comment on pull requests and turns out it's a breeze. Just generate a token and make a POST request like so:

curl -XPOST -H 'Authorization: token {token}' -d '{"body": "Test comment"}' 'https://api.github.com/repos/mozilla/snippets-service/issues/{number}/comments'

Then I went on to the travis documentation for avalible enviromental variables to get what PR the build is for and was able to build a little script that reports messages to the PR.

Only thing now is to somehow encrypt the github token and use that in the script. So I dove into the reading material and soon returned to the PR with this message.

I actually went on and wrote a little script that does this with a GitHub API token key and a new dummy user @mozilla-travis, but it seems not possible without exposing the token to the world since travis won't let PRs get access to secure variables. Would that be such a bad idea? (I'm just thinking out loud here). Since @mozilla-travis is just a dummy account with no access to anything would it be so bad if someone had it's token key?

So that's where I'm now. I'm thinking that putting the token key out there in plain text might not be the optimal situation but at the other hand I'm having trouble thinking of situations where this would become a serious security issue.

ps. Here's the script that I wrote.

import os
import sys
import json
import requests

def comment_on_pull_request(pr_number, slug, token, comment):
    url = 'https://api.github.com/repos/{slug}/issues/{number}/comments'.format(
        slug=slug, number=pr_number)
    response = requests.post(url, data=json.dumps({'body': comment}),
        headers={'Authorization': 'token ' + token})
    return response.json()

if __name__ == '__main__':
    PR_NUMBER = os.environ.get('TRAVIS_PULL_REQUEST')
    REPO_SLUG = os.environ.get('TRAVIS_REPO_SLUG')
    TOKEN = os.environ.get('GITHUB_TOKEN')
    results = sys.stdin.read()
    comment = "```{flake_results}```".format(flake_results=results)
    
    if all([PR_NUMBER, REPO_SLUG, TOKEN, results.strip()]):
        comment_on_pull_request(PR_NUMBER, REPO_SLUG, TOKEN, comment)