Skip to content

Instantly share code, notes, and snippets.

@djmitche
Last active Jul 25, 2016
Embed
What would you like to do?
(1118394) Notes on running Android builds in TaskCluster

Goal

"Drive Android builds out of taskcluster" .. meaning to run builds of Firefox for Android (we don't build Android) within taskcluster.

The idea is to get something non-b2g from releng running in taskcluster, both as a learning opportunity and as a proof of concept.

Task List

  • build a docker image for building Android using the tools in alder
  • build Android by hand / shell script in that docker image
  • do a basic Android build in a simple, docker-compatible way
    • handle tooltool cache
    • handle ccache for m-c and tools
    • encode the build process in a mozharness script
      • put that script in-tree
    • put all the make commands under 'mach build' (see below)
  • coordinate with mshal / @mrrrgn
    • post my docker patches for fb from @mrrrgn
    • refactor my docker patches to match https://github.com/mrrrgn/ff64-build/blob/master/Dockerfile
    • try running an fx desktop build in that container
    • Factor in some stuff from my first attempt
      • use tc-vcs (check out in the location mozharness will try to check it out later)
      • tc-vcs, tooltool caches
      • move some build.sh env's to docker
    • update tools repo (already installed in image) with tc-vcs in build.sh
    • Start Xvfb
    • try running an Android build in that container using https://bugzilla.mozilla.org/show_bug.cgi?id=1155349
    • parameterize other mozharness options
    • install java
      • newbug: do so via tooltool
  • rebase onto the new tooltool
    • allow specification of relengapi token via env variable
  • re-evaluate subsequent steps (maybe no in-tree MH script, maybe a script based on fx_desktop, etc.)
  • upload/export artifacts somewhere
  • use TC's usual mechanism for caching objdirs
  • generalize into a "mozharness runner" that takes a MH script and config as args
    • compare to build-emulator.sh
  • upload docker images somewhere they can be used by taskcluster
  • run a build in a hand-written, hand-submitted task
  • handle uploads / artifacts
  • figure out how to send a relengapi token without mozharness happily logging it everywhere -- just unset in build.sh?
  • hook up to try

Improvements after hooking up to try (all made into bugs)

  • move build-setup.sh from ubuntu-build to desktop-build; rename the latter to firefox-build
  • encrypt the relengapi token :( :( :(
  • handle ccache properly
  • handle symbol uploads
  • test that the APK actually works
  • figure out why compiler / binutils hacks are required and fix
  • zconf.h symlink
  • a.out.h symlink
  • LIBRARY_PATH setting
  • retry starting Xvfb in a loop
  • align input parameter names with b2g jobs (MOZHARNESS_REF, etc.)
  • figure out if we have to set MOZ_BUILD_DATE, and if so to what (comments in bug 1125973)
{
"provisionerId": "aws-provisioner",
"workerType": "b2g-desktop-opt",
"created": "2015-05-06T16:23:12.808Z",
"deadline": "2015-05-06T17:23:12.808Z",
"payload": {
"image": "quay.io/djmitche/desktop-build:0.0.1",
"env": {
"MOZHARNESS_SCRIPT": "mozharness/scripts/fx_desktop_build.py",
"MOZHARNESS_CONFIG": "builds/releng_base_android_64_builds.py",
"MOZHARNESS_HEAD_REPOSITORY": "https://bitbucket.org/djmitche/mozharness",
"MOZHARNESS_REV": "bug1125973",
"NEED_XVFB": "false",
"MH_CUSTOM_BUILD_VARIANT_CFG": "api-11",
"RELENGAPI_TOKEN": "..yeah, should encrypt this.."
},
"command": [
"/bin/bash",
"bin/build.sh"
],
"maxRunTime": 36000
},
"metadata": {
"name": "Test Task",
"description": "Testing execution of a thing in my own docker image",
"owner": "dustin@mozilla.com",
"source": "http://tools.taskcluster.net/task-creator/"
}
}
11:27 * lightsofapollo dustin: use the artifacts sectiion of the payload
http://docs.taskcluster.net/docker-worker/
11:27 <~lightsofapollo> dustin: mshal has been using the apis to do the same
in buildbot itself... he is an example
11:27 <~lightsofapollo>
https://hg.mozilla.org/projects/alder/file/5b37e35e89fc/testing/taskcluster/tasks/build.yml#l33
11:28 < mshal> dustin: here's the mozharness bug I'm working on to use TC to
upload files: https://bugzilla.mozilla.org/show_bug.cgi?id=1112252
11:28 < mshal> it hasn't landed yet, but the patches there show how I'm
planning to use it

(from wcosta and garndt)

caches

Specify a scope, e.g.,

scopes:
    - 'docker-worker:cache:sources-gecko'

Then use the cache in the payload:

payload:
  cache:
    sources-gecko: '/home/worker/gecko'

which causes the docker-worker to mount the thing identified by sources-gecko at that path.

checkouts

The tc-vcs tool does reliable checkouts, using caches and falling back to bundles on S3. It can be run from shell scripts. The env vars there come from the task as generated by the decision task.

Wander added support for using tc-vcs from Mozharness.

In the future, will be converting shell scripts to mozharness scripts.

11:31 < dustin> should I try to condense that to a mozharness script, do you
think?  And then the taskcluster task just invokes that script?
11:32 < dustin> if I do that, I have to put the mozharness script in the
mozharness repo, right?  In which case, the build is no longer reproducible if
that repo changes..
11:32 < dustin> I feel like that's a fatal, showstopper bug with mozharness :(
11:33 < mshal> dustin: I don't think the mozharness scripts have to be in the
mozharness repo, though most of them are there now. IIRC there's a bug to move
them in tree somewhere...
11:33 < dustin> ok
11:34 < dustin> anyone in particular I should talk to for guidance there?  I
guess ideally I'd have moharness installed in the docker image, and the
mozharness script in-tree
11:34 < garndt> dustin, if they were in the mozharness repo, we have an
example in our phone build tasks that can pin a mozharness repo/revision
11:34 < mshal> dustin: as for uploading, my understanding is that whatever's
running inside docker doesn't actually do file uploads, but just puts the
build outputs into the artifacts directory
11:34 < dustin> mshal: right, I think that's the lead I'll follow
11:34 <~lightsofapollo> dustin: armenzg_mtg  has been doing some awesome work
to move us in that directly
11:34 <~lightsofapollo> direction**
11:34 < dustin> garndt: got a pointer?
11:34 < dustin> lightsofapollo: cool, I'll check with him
11:34 < mshal> dustin: jlund would be good to chat with about overall
mozharness direction too, though he might be out on PTO?
11:34 <~lightsofapollo> and wcosta just landed some code so we can set the
mozharness version / repo from graph generation
11:34 < garndt> dustin: here is what was done for the phone builds
http://hg.mozilla.org/projects/alder/file/5b37e35e89fc/testing/taskcluster/tasks/phone_build.yml#l45
11:35 < dustin> lightsofapollo: is that what you're talking about ^^?
11:35 <~lightsofapollo> dustin: yup
11:35 <~lightsofapollo> set in the mach taskcluster-graph command
11:35 < garndt> dustin:
http://hg.mozilla.org/projects/alder/file/5b37e35e89fc/testing/taskcluster/mach_commands.py#l149
11:35 < garndt> ^^ mach command for building a graph with mozharness revision
11:35 < dustin> thanks!
11:36 < dustin> I'll talk to Armen and decide which way is more futuristic
11:36 <~lightsofapollo> (gaia + tc does not submit to gaia-try but
gaia/gaia-master)
11:36 <~lightsofapollo> dustin: IMO I think the future is in tree mozharness
scripts + locked down version 

In b2g

  • task calls build-emulator.sh which is baked into the image, specifying mozharness repo and revision as env vars
  • that script checks out mozharness and B2G
  • invokes ./mozharness/scripts/whatever.py with arguments -- so, in the mozharness repo

My plan

  • task calls build.sh which is baked into the docker image, with gecko, mozharness info as env vars (with defaults)
  • script checks out mozharness, gecko, tools
  • script invokes something like ./build/mozharness/android-emulator.py with config -- so, in the gecko tree
15:18:24 <lightsofapollo> dustin: mrrrgn|brb : the docs are not amazing here yet (one of my Q2 goals) but it's pretty easy to add new jobs/tasks to TC for try
15:18:46 <lightsofapollo> you add a new file here https://dxr.mozilla.org/mozilla-central/source/testing/taskcluster/tasks/builds
15:19:05 <lightsofapollo> add your flag name https://dxr.mozilla.org/mozilla-central/source/testing/taskcluster/tasks/branches/base_job_flags.yml
15:19:18 <lightsofapollo> schedule it https://dxr.mozilla.org/mozilla-central/source/testing/taskcluster/tasks/branches/try/job_flags.yml
15:20:03 <dustin> "schedule it"?
15:20:21 <dustin> not sure what that means
15:20:57 → jrmuizel joined • DerekH → DerekH|Lunch
15:21:40 <dustin> how are base_job_flags.yml and job_flags.yml related?
15:22:16 <•bhearsum> job_flags.yml inehrits from base IIRC?
15:22:28 <•bhearsum> yeah
15:22:30 <•bhearsum> https://dxr.mozilla.org/mozilla-central/source/testing/taskcluster/tasks/branches/mozilla-central/job_flags.yml#6
15:25:18 <dustin> yep, so I'm not sure why I'd need to change both
15:25:29 <•bhearsum> yeah...i don't fully understand what goes where either
15:25:35 <•bhearsum> you might want to ask in #taskcluster about that part
15:25:45 <dustin> ok
15:25:47 <•bhearsum> jonas and garndt are my go tos for that :)
15:26:00 <dustin> I'm hoping lightsofapollo replies :)
15:26:07 <dustin> but I can futz with it too
15:26:10 <lightsofapollo> dustin: ah sorry
15:27:02 <lightsofapollo> dustin: so if you look here https://dxr.mozilla.org/mozilla-central/source/testing/taskcluster/tasks/branches/try/job_flags.yml#9
15:27:20 <lightsofapollo> you can see that linux_gecko is a try flag
15:27:28 <lightsofapollo> then we map the opt/debug types to specific tsks
15:27:51 <lightsofapollo> dustin: you can omit adding the flag to base but then your jobs won't show up when you do try: -p all
15:28:13 <lightsofapollo> (which is an unintentional feature we added which is actually a real use case =p)
@djmitche
Copy link
Author

djmitche commented Jan 23, 2015

tc-vcs clone isn't actually what we want; we want tc-vcs checkout-revision. Cloning was getting a version of m-c from November, somehow (maybe the latest bundle?), which had

[worker@2f7346934ee7 build]$ grep -r  gcc-4.7.2 
./mobile/android/config/mozconfigs/common:HOST_CC="/tools/gcc-4.7.2-0moz1/bin/gcc"
./mobile/android/config/mozconfigs/common:HOST_CXX="/tools/gcc-4.7.2-0moz1/bin/g++"

The most recent version has

HOST_CC="$topsrcdir/gcc/bin/gcc"
HOST_CXX="$topsrcdir/gcc/bin/g++"

which make a little more sense. However:

configure: error: installation or configuration problem: host compiler /home/worker/build/gcc/bin/gcc cannot create executables.

and in fact that directory doesn't exist at all. I assume some of the tooltool stuff should have installed this?

@djmitche
Copy link
Author

djmitche commented Jan 23, 2015

Ah, yes, I've been using a mishmash of different dates for things. Latest mozilla-central has https://hg.mozilla.org/mozilla-central/rev/338a9ad8a020 which has a new tooltool manifest

@djmitche
Copy link
Author

djmitche commented Jan 28, 2015

Items from https://bugzilla.mozilla.org/attachment.cgi?id=8554772&action=edit

  • check with Jordan / Mike about android + mach
  • remove references to NDKs / SDKs from image script

@djmitche
Copy link
Author

djmitche commented Jan 28, 2015

Regarding android + mach:

17:10 < mshal> dustin: the available automation flags are here: http://mxr.mozilla.org/mozilla-central/source/build/mozconfig.automation (the default values are the 1 or 0 after the dash). You can override it in a mozconfig by doing something like this: 
               http://mxr.mozilla.org/mozilla-central/source/browser/config/mozconfigs/macosx64/debug-static-analysis#1
17:11 < mshal> if there are new steps for android that aren't covered by those, we would need a slightly more complicated change

@djmitche
Copy link
Author

djmitche commented Feb 9, 2015

So the basic idea there is that 'mach build' will do all of the 'make symbols', etc. if MOZ_AUTOMATION_SYMBOLS, etc. are set in the mozconfig.

@djmitche
Copy link
Author

djmitche commented Feb 10, 2015

17:13:27    ERROR - Can't run command ['make', 'buildsymbols'] in non-existent directory 'build/obj-firefox'!
17:13:27    FATAL - Build failed!
17:13:27    FATAL - Running post_fatal callback...
17:13:27    FATAL - Exiting -1

@djmitche
Copy link
Author

djmitche commented Apr 29, 2015

I've rebased everything into hg with a nice set of commits, and I'm currently re-running the docker build process.

@djmitche
Copy link
Author

djmitche commented Apr 29, 2015

woo,

19:49:19     INFO -  59:54.97 We know it took a while, but your build finally finished successfully!
19:49:19     INFO -  For more information on what to do now, see https://developer.mozilla.org/docs/Developer_Guide/So_You_Just_Built_Firefox
19:49:19     INFO - Return code: 0
19:49:19     INFO - Copying logs to upload dir...
19:49:19     INFO - mkdir: /home/worker/build/upload/logs
19:49:19    DEBUG - Copying /home/worker/logs/localconfig.json to /home/worker/build/upload/logs/localconfig.json
19:49:19    DEBUG - mkdir_p: /home/worker/build/upload/logs Already exists.
19:49:19    DEBUG - Copying /home/worker/logs/log_info.log to /home/worker/build/upload/logs/log_info.log
19:49:19    DEBUG - mkdir_p: /home/worker/build/upload/logs Already exists.
19:49:19    DEBUG - Copying /home/worker/logs/log_raw.log to /home/worker/build/upload/logs/log_raw.log
19:49:19    DEBUG - mkdir_p: /home/worker/build/upload/logs Already exists.
19:49:19    DEBUG - Copying /home/worker/logs/log_warning.log to /home/worker/build/upload/logs/log_warning.log
19:49:19    DEBUG - mkdir_p: /home/worker/build/upload/logs Already exists.
19:49:19    DEBUG - Copying /home/worker/logs/log_critical.log to /home/worker/build/upload/logs/log_critical.log
19:49:19    DEBUG - mkdir_p: /home/worker/build/upload/logs Already exists.
19:49:19    DEBUG - Copying /home/worker/logs/log_error.log to /home/worker/build/upload/logs/log_error.log
19:49:19    DEBUG - mkdir_p: /home/worker/build/upload/logs Already exists.
19:49:19    DEBUG - Copying /home/worker/logs/log_debug.log to /home/worker/build/upload/logs/log_debug.log
19:49:19    DEBUG - mkdir_p: /home/worker/build/upload/logs Already exists.
19:49:19    DEBUG - Copying /home/worker/logs/log_fatal.log to /home/worker/build/upload/logs/log_fatal.log

@djmitche
Copy link
Author

djmitche commented Apr 29, 2015

Talked to @mrrrgn and @mshal about all of the ongoing projects. It's time to abandon the in-tree MH project - that's ongoing separately anyway. And Morgan's got a good thing going with building images using mozbootstrap, rather than manually specifying a ton of packages. And @mshal's got good work on using the fx_desktop.py script to build Android. So let's use those.

@djmitche
Copy link
Author

djmitche commented May 2, 2015

Desktop builds are now running, but are ending with test failures:

23:00:14     INFO - Ran 48 tests in 7.225s
23:00:14     INFO - FAILED (failures=4, skipped=1)

I suspect this has to do with Xvfb not running, but it is running, so ..

@djmitche
Copy link
Author

djmitche commented May 4, 2015

Deciding to leave the desktop builds aside, I moved to running the android builds landed in bug 1155349.

That first ended in

20:47:09     INFO - abs_src_mozconfig: /home/worker/workspace/build/src/mobile/android/config/mozconfigs/android/nightly
20:47:09    FATAL - "abs_src_mozconfig" path could not be determined. Please make sure it is a valid path off of "abs_src_dir"

but on mshal's advice I added --custom-build-variant-cfg api-9 which got past that point. Now:

14:27:51     INFO -  configure: error: The program java was not found.  Set $JAVA_HOME to your Java SDK directory or use --with-java-bin-path={java-bin-dir}

@djmitche
Copy link
Author

djmitche commented May 4, 2015

Plan is to install java in the docker image (and, wow, that's a lot of deps). Eventually that should be installed with tooltool

@djmitche
Copy link
Author

djmitche commented May 4, 2015

Boo:

18:00:04     INFO -  configure: error: The program javac was not found.  Set $JAVA_HOME to your Java SDK directory or use --with-java-bin-path={java-bin-dir}

@djmitche
Copy link
Author

djmitche commented May 4, 2015

Looks like installing openjdk-7-jdk (instead of -jre) will do the trick

@djmitche
Copy link
Author

djmitche commented May 4, 2015

20:03:56     INFO -  In file included from /home/worker/workspace/build/src/mozglue/linker/Zip.h:11:0,
20:03:56     INFO -                   from /home/worker/workspace/build/src/mozglue/linker/SeekableZStream.h:8,
20:03:56     INFO -                   from /home/worker/workspace/build/src/mozglue/linker/SeekableZStream.cpp:6:
20:03:56     INFO -  /usr/include/zlib.h:34:19: fatal error: zconf.h: No such file or directory

That's on disk, just apparently not in the include path:

root@taskcluster-worker:~# find / -name zconf.h
/usr/include/x86_64-linux-gnu/zconf.h

dunno what the right solution is, but a symlink will probably do the trick.

@djmitche
Copy link
Author

djmitche commented May 4, 2015

21:01:04     INFO -  PeerConnectionImpl.o
21:01:06     INFO -  In file included from ../../../dist/include/mozilla/dom/BindingUtils.h:17:0,
21:01:06     INFO -                   from ../../../dist/include/mozilla/dom/IDBCursorBinding.h:10,
21:01:06     INFO -                   from ../../../dist/include/mozilla/dom/indexedDB/IDBCursor.h:13,
21:01:06     INFO -                   from ../../../dist/include/mozilla/dom/indexedDB/SerializationHelpers.h:14,
21:01:06     INFO -                   from /home/worker/workspace/build/src/obj-firefox/ipc/ipdl/_ipdlheaders/mozilla/dom/PContentChild.h:86,
21:01:06     INFO -                   from ../../../dist/include/mozilla/dom/ContentChild.h:14,
21:01:06     INFO -                   from /home/worker/workspace/build/src/dom/media/gmp/GMPServiceChild.cpp:7,
21:01:06     INFO -                   from /home/worker/workspace/build/src/obj-firefox/dom/media/gmp/Unified_cpp_dom_media_gmp0.cpp:128:
21:01:06     INFO -  ../../../dist/include/mozilla/CycleCollectedJSRuntime.h:140:74: internal compiler error: Segmentation fault
21:01:06     INFO -  Please submit a full bug report,
21:01:06     INFO -  with preprocessed source if appropriate.
21:01:06     INFO -  See <http://source.android.com/source/report-bugs.html> for instructions.
21:01:06     INFO -  make[5]: *** [Unified_cpp_dom_media_gmp0.o] Error 1
21:01:06     INFO -  make[5]: Leaving directory `/home/worker/workspace/build/src/obj-firefox/dom/media/gmp'

@djmitche
Copy link
Author

djmitche commented May 4, 2015

Apparently that was a fluke, because re-running without change gave a different error:

22:29:01     INFO -  /home/worker/workspace/build/src/android-sdk-linux/build-tools/21.1.1/aapt package -f -m -M AndroidManifest.xml -I /home/worker/workspace/build/src/android-sdk-linux/platforms/android-21/android.jar --max-res-version 10 --auto-add-overlay -S /home/worker/workspace/build/src/mobile/android/base/crashreporter/res -S /home/worker/workspace/build/src/mobile/android/branding/nightly/res -S /home/worker/workspace/build/src/mobile/android/base/resources -S /home/worker/workspace/build/src/obj-firefox/mobile/android/base/res  --extra-packages org.mozilla.mozstumbler --custom-package org.mozilla.gecko --non-constant-id -F gecko.ap_ -J generated/ --output-text-symbols ./ -c mdpi,hdpi --ignore-assets "!.svn:!.git:.*:<dir>_*:!CVS:!thumbs.db:!picasa.ini:!*.scc:*~:#*:*.rej:*.orig"
22:29:01     INFO -  /home/worker/workspace/build/src/android-sdk-linux/build-tools/21.1.1/aapt: error while loading shared libraries: libz.so.1: cannot open shared object file: No such file or directory

@djmitche
Copy link
Author

djmitche commented May 4, 2015

Yet,

root@taskcluster-worker:~# find / -name 'libz.so.1'
/lib/x86_64-linux-gnu/libz.so.1

so I'm not sure what's going on here.

@djmitche
Copy link
Author

djmitche commented May 4, 2015

After unpacking by hand:

worker@taskcluster-worker:~$ ldd android-sdk-linux/build-tools/21.1.1/aapt
        linux-gate.so.1 =>  (0xf77b4000)
        librt.so.1 => /lib32/librt.so.1 (0xf7698000)
        libdl.so.2 => /lib32/libdl.so.2 (0xf7693000)
        libpthread.so.0 => /lib32/libpthread.so.0 (0xf7677000)
        libz.so.1 => not found
        libstdc++.so.6 => /usr/lib32/libstdc++.so.6 (0xf758e000)
        libm.so.6 => /lib32/libm.so.6 (0xf7547000)
        libgcc_s.so.1 => /usr/lib32/libgcc_s.so.1 (0xf752a000)
        libc.so.6 => /lib32/libc.so.6 (0xf737f000)
        /lib/ld-linux.so.2 (0xf77b5000)

I'm guessing this is a multilib thing?

@djmitche
Copy link
Author

djmitche commented May 4, 2015

Indeed -- and I bet that means I can get rid of the symlink. Now signing's broken, but I think that should just be disabled for now:

23:40:35     INFO -  cp  ../../build/mobile/robocop/robocop-debug-unsigned-unaligned.apk  ../../_tests/testing/mochitest/robocop.apk-unaligned.apk && python /home/worker/workspace/build/tools/release/signing/signtool.py --cachedir /home/worker/workspace/build/signing_cache -t /home/worker/token -n /home/worker/nonce -c /home/worker/workspace/build/tools/release/signing/host.cert -H gpg:osslsigncode:signcode:mar:jar:b2gmar:emevoucher:signing4.srv.releng.scl3.mozilla.com:9110 -H gpg:osslsigncode:signcode:mar:jar:b2gmar:emevoucher:signing5.srv.releng.scl3.mozilla.com:9110 -H gpg:osslsigncode:signcode:mar:jar:b2gmar:emevoucher:signing6.srv.releng.scl3.mozilla.com:9110 -H dmg:mac-signing2.srv.releng.scl3.mozilla.com:9110 -H dmg:mac-signing3.srv.releng.scl3.mozilla.com:9110 -H dmgv2:mac-v2-signing1.srv.releng.scl3.mozilla.com:9110 -H dmgv2:mac-v2-signing2.srv.releng.scl3.mozilla.com:9110 -H dmgv2:mac-v2-signing3.srv.releng.scl3.mozilla.com:9110 -H dmgv2:mac-v2-signing4.srv.releng.scl3.mozilla.com:9110 -f jar  ../../_tests/testing/mochitest/robocop.apk-unaligned.apk && /home/worker/workspace/build/src/android-sdk-linux/build-tools/21.1.1/zipalign -f -v 4  ../../_tests/testing/mochitest/robocop.apk-unaligned.apk  ../../_tests/testing/mochitest/robocop.apk && rm -f  ../../_tests/testing/mochitest/robocop.apk-unaligned.apk
23:40:36     INFO -  Traceback (most recent call last):
23:40:36     INFO -    File "/home/worker/workspace/build/tools/release/signing/signtool.py", line 221, in <module>
23:40:36     INFO -      main()
23:40:36     INFO -    File "/home/worker/workspace/build/tools/release/signing/signtool.py", line 174, in main
23:40:36     INFO -      token = open(options.tokenfile, 'rb').read()
23:40:36     INFO -  IOError: [Errno 2] No such file or directory: '/home/worker/token'

@djmitche
Copy link
Author

djmitche commented May 5, 2015

That didn't mean I could get rid of the zconf.h symlink.

Disabling signing allowed the build to complete. That's progress!

@djmitche
Copy link
Author

djmitche commented May 6, 2015

I pushed the image to quay.io/djmitche/desktop-build:

docker tag quay.io/{mozilla,djmitche}/desktop-build:0.0.1 && docker push quay.io/djmitche/desktop-build:0.0.1

now to build a task

@djmitche
Copy link
Author

djmitche commented May 6, 2015

Tried again with

{
    "provisionerId": "aws-provisioner",
    "workerType": "b2gtest",
    "created": "2015-05-06T16:23:12.808Z",
    "deadline": "2015-05-07T17:23:12.808Z",
    "scopes": [
        "docker-worker:cache:tc-vcs"
    ],
    "payload": {
        "image": "quay.io/djmitche/desktop-build:0.0.1",
        "env": {
            "MOZHARNESS_SCRIPT": "mozharness/scripts/fx_desktop_build.py",
            "MOZHARNESS_CONFIG": "builds/releng_base_android_64_builds.py",
            "MOZHARNESS_HEAD_REPOSITORY": "https://bitbucket.org/djmitche/mozharness",
            "MOZHARNESS_REV": "bug1125973",
            "NEED_XVFB": "false",
            "MH_CUSTOM_BUILD_VARIANT_CFG": "api-11",
        "RELENGAPI_TOKEN": ".."
        },
        "cache": {
        "tc-vcs": "/home/worker/.tc-vcs"
      },
        "command": [
            "/bin/bash",
            "bin/build.sh"
        ],
        "maxRunTime": 36000
    },
    "metadata": {
        "name": "Test Task",
        "description": "Testing execution of a thing in my own docker image",
        "owner": "dustin@mozilla.com",
        "source": "http://tools.taskcluster.net/task-creator/"
    }
}

but the task is rejected:

{
  "message": "Authorization Failed",
  "error": {
    "info": "None of the scope-sets was satisfied",
    "scopesets": [
      [
        "docker-worker:cache:tc-vcs"
      ]
    ]
  }
}

@djmitche
Copy link
Author

djmitche commented May 6, 2015

It looks like my own credentials don't include any caches, so I'll need to get some credentials via some other technique.

@djmitche
Copy link
Author

djmitche commented May 11, 2015

I really screwed up the file paths by not linking /builds to worksspace/build (which is an entirely different directory!)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment