escapewindow

Intent to implement: l10n bumper on mozilla-beta

Summary

This would be a script that grabs signed-off l10n changesets from elmo (l10n.mozilla.org) and lands them in-tree for Fx 53+. We would start using in-tree l10n changesets for Fennec 53.0b1, at the same time we start running our Fennec release automation and builds in Taskcluster.

We can schedule this script to run automatically at whichever rate we want. We can monitor elmo for changes aggressively, and bump the l10n changesets on change, or every N minutes if there are changes. We can also run this nightly (or never), and releaseduty could bump the l10n changesets manually when relman is ready to give go-to-build.

Use cases / Motivation

escapewindow

Nightly Chain of Trust Verification

(done) scriptworker hang - fixed oct 28, reviewed/released/deployed nov1

Polling dies suddenly, but the git pull process keeps going. We have a potential fix in place; this needs monitoring and fixing or landing, depending on how it goes. Not blocked by anything else, but definitely blocks tier1.

This is either new, or hidden by bug 1310120 and exposed now that it's fixed.

(done) scriptworker next, code complete - nov 7

escapewindow

Tl;dr: we're aiming for the task.extra.chainOfTrust.inputs model below. This is a writeup of the goals, choices, and decision.

chain of trust dependency traversal

For Chain of Trust verification, we need to be able to trace the graph back to the tree or trusted task (e.g., the signing task). To do that, it makes sense to explicitly add information to the graph at decision task time, rather than making scriptworker query the entire upstream graph and guess which tasks need validation.

The tasks that need validation are the tasks that can modify inputs. For a build signing task, the build, the docker-image task, and the decision task(s) can modify the inputs, and mere graph traversal can work. For the balrog submitter or google play submitter, which live at the end of the graph, we [initially] need to be able to ignore tests and other tasks that tell us about the inputs, but don't modify them. At some point we may decide to inspect and validate the entire graph before scriptworker tasks can proceed,

escapewindow

#TaskCluster worker GPG key management

Tl;dr: taskcluster workers will have a gpg keypair as a second security check. We need to decide how to manage the keys. For anyone who manages worker AMIs or hardware worker imaging, this will likely mean there will be an additional required step or two.

##Goals

• Security of the private key. The gpg private keys are ideally untouchable by humans or unvetted code.
• Separation of roles. RelEng and RelOps users should not be able to perform sensitive TaskCluster tasks, or vice versa.
• Low maintenance overhead. As little as possible, given the above goals

##Assumptions

escapewindow

#!/usr/bin/env python
# using https://github.com/escapewindow/python-generic-worker/blob/1046f6d177554dc3ddf2c14110849426913df00b/test.py
#  linked to lib.py
import asyncio
import atexit
import logging
import time

#log = logging.getLogger(__name__)
from lib import update_logging_config, log_errors, read_stdout, get_log_filenames, get_log_fhs, run_task, Context, close_asyncio_loop, log

escapewindow

• get assume:project:taskcluster:worker-test-scopes scope. That will give the following scopes:
• queue:define-task:test-dummy-provisioner/dummy-worker-*
• queue:task-group-id:test-dummy-scheduler/*
• queue:schedule-task:test-dummy-scheduler/*
• queue:poll-task-urls:test-dummy-provisioner/dummy-worker-*
• queue:claim-task:test-dummy-provisioner/dummy-worker-*
• queue:worker-id:test-dummy-workers/dummy-worker-*
• go to https://tools.taskcluster.net/auth/clients/ , create a mozilla-ldap/asasaki@mozilla.com/worker-test clientid with the assume:project:taskcluster:worker-test-scopes scope. This will grant access to the needed scopes for the worker
• scopes needed:
• To create task: (not legacy)

escapewindow

#!/usr/bin/env python
from __future__ import absolute_import, division, print_function, \
    unicode_literals

import asyncio
import aiohttp
from contextlib import contextmanager

import taskcluster.utils as utils
import taskcluster.baseclient as baseclient

escapewindow

Keybase proof

I hereby claim:

• I am escapewindow on github.
• I am escapewindow (https://keybase.io/escapewindow) on keybase.
• I have a public key ASBf4dMddNnZj9LwfEQvm8rb-IRA3aVgzcqNqCUi2vZkjwo

To claim this, I am signing this object:

escapewindow

#!/usr/bin/env python
from __future__ import absolute_import, division, print_function, \
    unicode_literals

import asyncio

def sleep(duration):
    yield from asyncio.sleep(duration)

escapewindow

#!/usr/bin/env python
from __future__ import print_function

repo_config = {'repo': 'm-c'}
source_dest = '.'
hg = ['hg', '--aki']
args = [hg + ['clone', '--noupdate', repo_config['repo'],
        repo_config.get('hg_clone_option'), source_dest], ]
print(args)
args = [x for x in args if x]