Skip to content

Instantly share code, notes, and snippets.

@djmitche

djmitche/@@todo.md

Last active May 31, 2018
Star
Embed
What would you like to do?
Learn more about clone URLs
Download ZIP
Raw
@@todo.md

TODO

  • documentation
    • in-tree
      • generic actions
      • actionPerm
      • naming conventions & roles
      • config options: who has hooks:trigger-hook; hook's triggerSchema; and scopes in the hook-id role
    • actions spec
      • type=hook
      • preferences between type=hook and type=task (use of hooks as "sudo", etc.)
  • tc-admin changes
    • scm level roles
    • hook roles
    • hooks
  • implement actions in-tree
  • implement type=hook in treeherder
  • implement type=hook in tools
  • one-click-loaner as an action in treeherder, tested

Implementation Plan

  1. Add everything needed (hooks, roles, etc.)
  • hooks & hook-id roles are added; rest is implemented in tc-admin
  1. Test by manually triggering hooks
  • in progress
    • upgrade hooks to latest tc-lib-api for decent error messages
  1. Implement actions in tools/treeherder and test on try pushes
  2. Land in-tree
  3. Announce
  4. Wait
  5. Remove old scopes after next ESR (but can start removing repos from scm level groups as they get the changes)

Issues

  • CoT verification of action tasks
    • Might be easier if hooks are generated in-trees?
  • ownTaskId - can hooks provide this?
  • lots of inputs, many of which must be threaded through schema as default values
Raw
config-changes.md

Config Changes

Changes to the TC runtime config

mozilla-group:active_scm_level_${level} Roles

  • add hooks:trigger-hook:.. scopes
  • remove assume:repo:... scopes (later)

other mozilla-group groups

Some way of saying "this hooks:trigger-hook:.. should only be in these mozilla-group roles" and adding/removing it as necessary, without specifying the entire content of those roles.

project-${trustDomain}/in-tree-action-${level}-${actionPerm} Hooks

Create new hooks with appropriate triggerSchema. It would be really nice if this was in-tree, but let's not, yet.

hook-id:project-${trustDomain}/in-tree-action-${level}-${actionPerm} Roles

assume:repo:hg.mozilla.org/${repo}:action:${actionPerm} for each repo at that level

repo:hg.mozilla.org/${repo}:action:${actionPerm} Roles

Only needed for non-generic actions; use parameterized roles

project:${trustDomain}:in-tree-action:level-${level}:${actionPerm}:* Roles

Parameterized roles for non-generic actions.

Raw
test-payload.yml
# https://tools.taskcluster.net/groups/RAFda3f0RFSAsV0vGm8Y9g/tasks/RAFda3f0RFSAsV0vGm8Y9g (decision)
# https://tools.taskcluster.net/groups/RAFda3f0RFSAsV0vGm8Y9g/tasks/Pe87AOGuSramJTxUSakVPg
action:
cb_name: mochitest_retrigger_action
description: Retriggers the specified mochitest/reftest job with additional options
name: retrigger-mochitest-reftest-with-options
actionPerm: generic
repo_scope: assume:repo:hg.mozilla.org/try:branch:default
symbol: tr
taskGroupId: RAFda3f0RFSAsV0vGm8Y9g
title: Mochitest/Reftest Retrigger
push:
owner: mozilla-taskcluster-maintenance@mozilla.com
pushlog_id: '266081'
revision: deda6c63ed8dfefc392e95b1d5bed9560c09e543
repository:
level: '1'
project: try
url: https://hg.mozilla.org/try
callback: test_retrigger_action
input:
environment:
MOZ_LOG: ''
logLevel: debug
path: ''
preferences:
mygeckopreferences.pref: myvalue2
repeat: 30
runUntilFail: true
parameters:
app_version: 61.0a1
base_repository: https://hg.mozilla.org/mozilla-unified
build_date: 1524169028
build_number: 1
do_not_optimize: []
existing_tasks: {}
filters:
- check_servo
- target_tasks_method
head_ref: deda6c63ed8dfefc392e95b1d5bed9560c09e543
head_repository: https://hg.mozilla.org/try
head_rev: deda6c63ed8dfefc392e95b1d5bed9560c09e543
include_nightly: true
level: '1'
message: ' '
moz_build_date: '20180419201708'
next_version: null
optimize_target_tasks: false
owner: ahalberstadt@mozilla.com
project: try
pushdate: 1524169028
pushlog_id: '266081'
release_eta: ''
release_history: {}
release_type: ''
target_tasks_method: try_tasks
try_mode: try_task_config
try_options: null
try_task_config:
tasks:
- test-android-4.2-x86/opt-mochitest-chrome-1
- test-android-4.3-arm7-api-16/debug-mochitest-1
- test-android-4.3-arm7-api-16/debug-mochitest-chrome-1
- test-android-4.3-arm7-api-16/debug-mochitest-media-1
- test-android-4.3-arm7-api-16/opt-mochitest-1
- test-android-4.3-arm7-api-16/opt-mochitest-chrome-1
- test-android-4.3-arm7-api-16/opt-mochitest-media-1
- test-linux32/debug-mochitest-1
- test-linux32/debug-mochitest-browser-chrome-e10s-1
- test-linux32/debug-mochitest-chrome-1
- test-linux32/debug-mochitest-e10s-1
- test-linux32/debug-mochitest-media-e10s-1
- test-linux32/debug-mochitest-webgl-1
- test-linux32/debug-mochitest-webgl-e10s-1
- test-linux32/opt-mochitest-browser-chrome-e10s-1
- test-linux32/opt-mochitest-chrome-1
- test-linux32/opt-mochitest-devtools-chrome-e10s-1
- test-linux32/opt-mochitest-e10s-1
- test-linux32/opt-mochitest-media-e10s-1
- test-linux32/opt-mochitest-webgl-e10s-1
- test-linux64-asan/opt-mochitest-browser-chrome-e10s-1
- test-linux64-asan/opt-mochitest-chrome-1
- test-linux64-asan/opt-mochitest-devtools-chrome-e10s-1
- test-linux64-asan/opt-mochitest-e10s-1
- test-linux64-asan/opt-mochitest-media-e10s-1
- test-linux64-asan/opt-mochitest-webgl-e10s-1
- test-linux64-ccov/opt-mochitest-browser-chrome-e10s-1
- test-linux64-ccov/opt-mochitest-chrome-1
- test-linux64-ccov/opt-mochitest-devtools-chrome-e10s-1
- test-linux64-ccov/opt-mochitest-e10s-1
- test-linux64-ccov/opt-mochitest-media-e10s-1
- test-linux64-ccov/opt-mochitest-webgl-e10s-1
- test-linux64-jsdcov/opt-mochitest-browser-chrome-e10s-1
- test-linux64-jsdcov/opt-mochitest-e10s-1
- test-linux64-pgo/opt-mochitest-browser-chrome-e10s-1
- test-linux64-pgo/opt-mochitest-chrome-1
- test-linux64-pgo/opt-mochitest-devtools-chrome-e10s-1
- test-linux64-pgo/opt-mochitest-e10s-1
- test-linux64-pgo/opt-mochitest-media-e10s-1
- test-linux64-pgo/opt-mochitest-webgl-e10s-1
- test-linux64-qr/debug-mochitest-e10s-1
- test-linux64-qr/debug-mochitest-media-e10s-1
- test-linux64-qr/debug-mochitest-webgl-e10s-1
- test-linux64-qr/opt-mochitest-e10s-1
- test-linux64-qr/opt-mochitest-media-e10s-1
- test-linux64-qr/opt-mochitest-webgl-e10s-1
- test-linux64/debug-mochitest-browser-chrome-e10s-1
- test-linux64/debug-mochitest-chrome-1
- test-linux64/debug-mochitest-devtools-chrome-e10s-1
- test-linux64/debug-mochitest-e10s-1
- test-linux64/debug-mochitest-media-e10s-1
- test-linux64/debug-mochitest-plain-headless-e10s-1
- test-linux64/debug-mochitest-webgl-e10s-1
- test-linux64/opt-mochitest-browser-chrome-e10s-1
- test-linux64/opt-mochitest-chrome-1
- test-linux64/opt-mochitest-devtools-chrome-e10s-1
- test-linux64/opt-mochitest-e10s-1
- test-linux64/opt-mochitest-media-e10s-1
- test-linux64/opt-mochitest-plain-headless-e10s-1
- test-linux64/opt-mochitest-webgl-e10s-1
- test-macosx64/debug-mochitest-browser-chrome-e10s-1
- test-macosx64/debug-mochitest-chrome-1
- test-macosx64/debug-mochitest-devtools-chrome-e10s-1
- test-macosx64/debug-mochitest-e10s-1
- test-macosx64/debug-mochitest-webgl-e10s-1
- test-macosx64/opt-mochitest-browser-chrome-e10s-1
- test-macosx64/opt-mochitest-chrome-1
- test-macosx64/opt-mochitest-devtools-chrome-e10s-1
- test-macosx64/opt-mochitest-e10s-1
- test-macosx64/opt-mochitest-webgl-e10s-1
- test-windows10-64-ccov/debug-mochitest-browser-chrome-e10s-1
- test-windows10-64-ccov/debug-mochitest-chrome-1
- test-windows10-64-ccov/debug-mochitest-devtools-chrome-e10s-1
- test-windows10-64-ccov/debug-mochitest-e10s-1
- test-windows10-64-pgo/opt-mochitest-browser-chrome-e10s-1
- test-windows10-64-pgo/opt-mochitest-chrome-1
- test-windows10-64-pgo/opt-mochitest-devtools-chrome-e10s-1
- test-windows10-64-pgo/opt-mochitest-e10s-1
- test-windows10-64-pgo/opt-mochitest-webgl-e10s-1
- test-windows10-64-qr/debug-mochitest-webgl-e10s-1
- test-windows10-64-qr/opt-mochitest-webgl-e10s-1
- test-windows10-64/debug-mochitest-browser-chrome-e10s-1
- test-windows10-64/debug-mochitest-chrome-1
- test-windows10-64/debug-mochitest-devtools-chrome-e10s-1
- test-windows10-64/debug-mochitest-e10s-1
- test-windows10-64/debug-mochitest-plain-headless-e10s-1
- test-windows10-64/debug-mochitest-webgl-e10s-1
- test-windows10-64/opt-mochitest-browser-chrome-e10s-1
- test-windows10-64/opt-mochitest-chrome-1
- test-windows10-64/opt-mochitest-devtools-chrome-e10s-1
- test-windows10-64/opt-mochitest-e10s-1
- test-windows10-64/opt-mochitest-plain-headless-e10s-1
- test-windows10-64/opt-mochitest-webgl-e10s-1
- test-windows7-32-pgo/opt-mochitest-browser-chrome-e10s-1
- test-windows7-32-pgo/opt-mochitest-chrome-1
- test-windows7-32-pgo/opt-mochitest-devtools-chrome-e10s-1
- test-windows7-32-pgo/opt-mochitest-e10s-1
- test-windows7-32-pgo/opt-mochitest-media-e10s-1
- test-windows7-32-pgo/opt-mochitest-webgl-e10s-1
- test-windows7-32/debug-mochitest-browser-chrome-e10s-1
- test-windows7-32/debug-mochitest-chrome-1
- test-windows7-32/debug-mochitest-devtools-chrome-e10s-1
- test-windows7-32/debug-mochitest-e10s-1
- test-windows7-32/debug-mochitest-media-e10s-1
- test-windows7-32/debug-mochitest-webgl-e10s-1
- test-windows7-32/opt-mochitest-browser-chrome-e10s-1
- test-windows7-32/opt-mochitest-chrome-1
- test-windows7-32/opt-mochitest-devtools-chrome-e10s-1
- test-windows7-32/opt-mochitest-e10s-1
- test-windows7-32/opt-mochitest-media-e10s-1
- test-windows7-32/opt-mochitest-webgl-e10s-1
templates:
artifact:
enabled: '1'
version: 61.0a1
task: {
"provisionerId": "aws-provisioner-v1",
"workerType": "gecko-t-linux-large",
"schedulerId": "gecko-level-1",
"taskGroupId": "RAFda3f0RFSAsV0vGm8Y9g",
"dependencies": [
"exyt3MqBR22vEUJ8APZw_g",
"fghJ4AW5QsGZyxP48KlOJQ"
],
"requires": "all-completed",
"routes": [
"tc-treeherder.v2.try.deda6c63ed8dfefc392e95b1d5bed9560c09e543.266081"
],
"priority": "very-low",
"retries": 5,
"created": "2018-04-19T20:18:49.459Z",
"deadline": "2018-04-20T20:18:49.459Z",
"expires": "2018-05-03T20:18:49.459Z",
"scopes": [
"secrets:get:project/taskcluster/gecko/hgfingerprint",
"docker-worker:feature:allowPtrace",
"docker-worker:capability:device:loopbackVideo",
"docker-worker:cache:level-1-try-test-workspace-v3-697eee6c304d222fdb39",
"docker-worker:cache:level-1-checkouts-v3-697eee6c304d222fdb39"
],
"payload": {
"onExitStatus": {
"retry": [
4
]
},
"maxRunTime": 3600,
"image": {
"path": "public/image.tar.zst",
"type": "task-image",
"taskId": "exyt3MqBR22vEUJ8APZw_g"
},
"cache": {
"level-1-try-test-workspace-v3-697eee6c304d222fdb39": "/builds/worker/workspace",
"level-1-checkouts-v3-697eee6c304d222fdb39": "/builds/worker/checkouts"
},
"capabilities": {
"devices": {
"loopbackVideo": true
}
},
"artifacts": {
"public/logs/": {
"path": "/builds/worker/workspace/build/upload/logs/",
"expires": "2018-05-03T20:18:49.459Z",
"type": "directory"
},
"public/test": {
"path": "/builds/worker/artifacts/",
"expires": "2018-05-03T20:18:49.459Z",
"type": "directory"
},
"public/test_info/": {
"path": "/builds/worker/workspace/build/blobber_upload_dir/",
"expires": "2018-05-03T20:18:49.459Z",
"type": "directory"
}
},
"command": [
"/builds/worker/bin/run-task",
"--",
"/builds/worker/bin/test-linux.sh",
"--no-read-buildbot-config",
"--installer-url=https://queue.taskcluster.net/v1/task/fghJ4AW5QsGZyxP48KlOJQ/artifacts/public/build/target.tar.bz2",
"--test-packages-url=https://queue.taskcluster.net/v1/task/fghJ4AW5QsGZyxP48KlOJQ/artifacts/public/build/target.test_packages.json",
"--mochitest-suite=chrome",
"--allow-software-gl-layers",
"--total-chunk=3",
"--this-chunk=1"
],
"env": {
"SCCACHE_DISABLE": "1",
"MOZ_NODE_PATH": "/usr/local/bin/node",
"TRY_COMMIT_MSG": " ",
"TASKCLUSTER_UNTRUSTED_CACHES": "1",
"HG_STORE_PATH": "/builds/worker/checkouts/hg-store",
"MOZILLA_BUILD_URL": "https://queue.taskcluster.net/v1/task/fghJ4AW5QsGZyxP48KlOJQ/artifacts/public/build/target.tar.bz2",
"GECKO_HEAD_REPOSITORY": "https://hg.mozilla.org/try",
"GECKO_HEAD_REV": "deda6c63ed8dfefc392e95b1d5bed9560c09e543",
"MOCHITEST_FLAVOR": "chrome",
"TASKCLUSTER_VOLUMES": "/builds/worker/.cache;/builds/worker/checkouts;/builds/worker/tooltool-cache;/builds/worker/workspace",
"MOZHARNESS_URL": "https://queue.taskcluster.net/v1/task/fghJ4AW5QsGZyxP48KlOJQ/artifacts/public/build/mozharness.zip",
"NEED_PULSEAUDIO": "true",
"MOZ_AUTOMATION": "1",
"NEED_WINDOW_MANAGER": "true",
"MOZHARNESS_CONFIG": "unittests/linux_unittest.py remove_executables.py",
"ENABLE_E10S": "false",
"MOZHARNESS_SCRIPT": "desktop_unittest.py",
"GECKO_BASE_REPOSITORY": "https://hg.mozilla.org/mozilla-unified",
"TASKCLUSTER_CACHES": "/builds/worker/checkouts;/builds/worker/workspace"
},
"features": {
"taskclusterProxy": true,
"allowPtrace": true
}
},
"metadata": {
"owner": "ahalberstadt@mozilla.com",
"source": "https://hg.mozilla.org/try/file/deda6c63ed8dfefc392e95b1d5bed9560c09e543/taskcluster/ci/test",
"description": "Mochitest chrome run ([Treeherder push](https://treeherder.mozilla.org/#/jobs?repo=try&revision=deda6c63ed8dfefc392e95b1d5bed9560c09e543))",
"name": "test-linux64-asan/opt-mochitest-chrome-1"
},
"tags": {
"kind": "test",
"worker-implementation": "docker-worker",
"createdForUser": "ahalberstadt@mozilla.com",
"label": "test-linux64-asan/opt-mochitest-chrome-1",
"test-type": "mochitest",
"os": "linux"
},
"extra": {
"chunks": {
"current": 1,
"total": 3
},
"suite": {
"flavor": "chrome",
"name": "mochitest"
},
"treeherder": {
"jobKind": "test",
"groupSymbol": "M",
"collection": {
"asan": true
},
"machine": {
"platform": "linux64"
},
"groupName": "Mochitests",
"tier": 1,
"symbol": "c1"
},
"parent": "RAFda3f0RFSAsV0vGm8Y9g",
"index": {
"rank": 1524169028
}
}
}
taskId: Pe87AOGuSramJTxUSakVPg
taskGroupId: RAFda3f0RFSAsV0vGm8Y9g
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment