Skip to content

Instantly share code, notes, and snippets.

@gabriel-v

gabriel-v/issues-mach-try-coverage.md

Last active September 5, 2018 16:30
Show Gist options
  • Save gabriel-v/66e23d30f6a470a1b714f3e77dfa3bfe to your computer and use it in GitHub Desktop.
Save gabriel-v/66e23d30f6a470a1b714f3e77dfa3bfe to your computer and use it in GitHub Desktop.
Download ZIP
List of issues with |mach try coverage|, see https://bugzilla.mozilla.org/show_bug.cgi?id=1429463
Raw
issues-mach-try-coverage.md

Bugs

[1]: GTest fails when run by mach try coverage

Bug no: TODO

Error message: shutil error: Destination path 'Z:\task_1535734500\build\application\firefox\gmp-clearkey' already exists

Further investigation: try running it with mach try fuzzy, and then mach try fuzzy with a path prefix.

[2]: Marionette and marionette-headless fail with per-test

Bug no: 1480142

[3]: mach try coverage test run fails with "no tests to run"

Bug no: TODO

When running single tests from suites on multiple platforms, some chunks turn up empty (because the requested test is skipped or disabled on that respective platform). Running them will fail with no tests to run or similar messages. They are harmless, but checking each one of them each time is a manual process.

Should we turn all of these no tests to run outcomes green? Could we avoid scheduling them? Could we make them green only for mach try coverage runs, via some MOZ_TRY_COVERAGE envvar?

[4]: mach try coverage data is not up to date

Bug no: TODO

Explain how moving files around the tree will cause UNKNOWN TEST failures when scheduling tasks, and the issue will persist until there's a new chunk mapping file.

[5]: Fix tests failing when run by mach try coverage

Bug no: TODO

This bug should be a list of all the tests that fail when run by mach try coverage but don't fail when run normally, with their whole suite. For each one, test it again with mach try fuzzy with and without a path selection.

key test reason
test-windows10-64/opt-firefox-ui-functional-remote-e10s testing\firefox-ui\tests\functional\safebrowsing\test_initial_download.py TestSafeBrowsingInitialDownload.test_safe_browsing_initial_download assertion error
test-android-em-4.3-arm7-api-16/opt-mochitest-1 toolkit/content/tests/widgets/test_videocontrols_iframe_fullscreen.html test timed out
test-macosx64/opt-mochitest-browser-chrome-e10s-1 browser/components/preferences/in-content/tests/browser_change_app_handler.js uncaught exception
test-linux64/opt-mochitest-browser-chrome-e10s-1 browser/base/content/test/forms/browser_selectpopup_colors.js test failure

[6]: test-verify tasks fail with mach try coverage

Bug no: TODO

This happens when mach try coverage adds a path for a test that is not registered with TV or TVw. This happens, for example, with paths from the python suite. This causes instant task failure.

Should we avoid scheduling TV and TVw tasks when running mach try coverage?

[7]: Crashtests don't support running tests in isolation

Bug no: TODO

Suites left to test

  • cppunittest

Suites tested

Files tested by multiple suites

xpcom/io/nsWildCard.cpp

https://treeherder.mozilla.org/#/jobs?repo=try&revision=3e4497508bd9398ec14c35c419e6d0bfd95c288d

Tests: Run looks OK. Passing: opt-mochitest-browser-chrome, xpcshell and firefox-ui-functional.

Test failures:

  • gtest on both linux and windows. Error message: shutil error: Destination path 'Z:\task_1535734500\build\application\firefox\gmp-clearkey' already exists [1]
  • marionette. Error message: untests.py: error: Test file(s) not found: js/src/tests/test262/language/expressions/new.target/value-via-fpcall.js testing/marionette/harness/marionette_harness/tests/unit/test_typing.py. This is right, that file is a jsreftest, not a marionette test. Some of these unknwon tests are actually marionette tests, but they get listed here regardless. [2]
  • marionette-headless. Error message: same as above. Same test harness, same problem. [2]
  • xpcshell on Android: failed with no tests to run. This is normal, the tests have been skipped. [3]

Missing tests: a lot of UNKNOWN TEST for jsreftests. This may be caused by old chunk mapping coverage data. It worked in the past, so I'll investigate further. [4]

Artifact builds enabled, pass --no-artifact to disable
Found 2 modified source files with test coverage:
         tools/tryselect/test/test_tasks.py
         xpcom/io/nsWildCard.cpp
Found 6 modified source files with no coverage:
         tools/tryselect/mach_commands.py
         tools/tryselect/selectors/coverage.py
         tools/tryselect/selectors/fuzzy.py
         tools/tryselect/tasks.py
         tools/tryselect/test/python.ini
         tools/tryselect/test/test_fuzzy.py
Running 126 individual test files.
Running 9 test chunks.
         linux firefox-ui-functional-local
         linux firefox-ui-functional-remote
         linux gtest
         linux marionette
         linux marionette-headless
         windows firefox-ui-functional-local
         windows gtest
         windows marionette
         windows marionette-headless
Task configuration changed, generating target task set
UNKNOWN TEST: js/src/tests/test262/language/expressions/new.target/value-via-fpcall.js
UNKNOWN TEST: js/src/tests/test262/built-ins/Reflect/defineProperty/return-boolean.js
UNKNOWN TEST: js/src/tests/test262/built-ins/Proxy/getPrototypeOf/call-parameters.js
UNKNOWN TEST: js/src/tests/test262/intl402/RelativeTimeFormat/prototype/format/en-us-style-short.js
UNKNOWN TEST: js/src/tests/test262/built-ins/Reflect/deleteProperty/return-boolean.js
UNKNOWN TEST: js/src/tests/test262/built-ins/Reflect/set/length.js
UNKNOWN TEST: js/src/tests/test262/built-ins/RegExp/CharacterClassEscapes/character-class-non-whitespace-class-escape.js
UNKNOWN TEST: js/src/tests/test262/built-ins/Proxy/get/trap-is-undefined.js
UNKNOWN TEST: js/src/tests/test262/built-ins/RegExp/S15.10.2.12_A1_T1.js
UNKNOWN TEST: js/src/tests/test262/built-ins/Reflect/getPrototypeOf/return-prototype.js
UNKNOWN TEST: js/src/tests/test262/built-ins/Proxy/create-target-not-object-throw-symbol.js
UNKNOWN TEST: js/src/tests/test262/built-ins/Proxy/get/trap-is-not-callable-realm.js
UNKNOWN TEST: js/src/tests/test262/built-ins/Proxy/get/accessor-get-is-undefined-throws.js
UNKNOWN TEST: js/src/tests/test262/built-ins/Proxy/defineProperty/call-parameters.js
UNKNOWN TEST: js/src/tests/test262/built-ins/RegExp/CharacterClassEscapes/character-class-non-digit-class-escape.js
UNKNOWN TEST: js/src/tests/test262/built-ins/Reflect/setPrototypeOf/target-is-not-object-throws.js
UNKNOWN TEST: js/src/tests/test262/built-ins/Object/fromEntries/evaluation-order.js
UNKNOWN TEST: js/src/tests/test262/intl402/RelativeTimeFormat/constructor/supportedLocalesOf/options-null.js
UNKNOWN TEST: js/src/tests/test262/built-ins/RegExp/S15.10.2.12_A1_T2.js
UNKNOWN TEST: js/src/tests/test262/built-ins/Object/fromEntries/iterator-closed-for-throwing-entry-key-accessor.js
UNKNOWN TEST: js/src/tests/test262/built-ins/Proxy/getOwnPropertyDescriptor/trap-is-not-callable.js
UNKNOWN TEST: js/src/tests/test262/built-ins/Reflect/construct/arguments-list-is-not-array-like.js
UNKNOWN TEST: js/src/tests/test262/built-ins/RegExp/CharacterClassEscapes/character-class-digit-class-escape-plus-quantifier.js
UNKNOWN TEST: js/src/tests/test262/built-ins/Proxy/enumerate/removed-does-not-trigger.js
UNKNOWN TEST: js/src/tests/test262/built-ins/Atomics/wake/name.js
UNKNOWN TEST: js/src/tests/test262/built-ins/Atomics/wait/bigint/no-spurious-wakeup-on-xor.js
UNKNOWN TEST: js/src/tests/non262/Date/time-zones-pedantic.js
UNKNOWN TEST: js/src/tests/test262/built-ins/RegExp/CharacterClassEscapes/character-class-digit-class-escape-flags-u.js
UNKNOWN TEST: js/src/tests/test262/language/directive-prologue/10.1.1-19-s.js
UNKNOWN TEST: js/src/tests/test262/built-ins/Proxy/revocable/revocation-function-nonconstructor.js
UNKNOWN TEST: js/src/tests/test262/language/directive-prologue/func-expr-final-runtime.js
UNKNOWN TEST: js/src/tests/test262/intl402/RelativeTimeFormat/constructor/supportedLocalesOf/locales-invalid.js
UNKNOWN TEST: toolkit/components/microformats/test/marionette/test_modules.py
UNKNOWN TEST: js/src/tests/test262/built-ins/Proxy/isExtensible/trap-is-undefined.js
UNKNOWN TEST: js/src/tests/test262/built-ins/Proxy/apply/trap-is-not-callable.js
UNKNOWN TEST: js/src/tests/test262/built-ins/RegExp/S15.10.2.12_A2_T3.js
UNKNOWN TEST: js/src/tests/test262/built-ins/Proxy/deleteProperty/trap-is-undefined-not-strict.js
UNKNOWN TEST: js/src/tests/test262/built-ins/Reflect/preventExtensions/return-abrupt-from-result.js
UNKNOWN TEST: js/src/tests/test262/built-ins/Atomics/wait/undefined-index-defaults-to-zero.js
UNKNOWN TEST: js/src/tests/test262/built-ins/Proxy/function-prototype.js
UNKNOWN TEST: layout/base/tests/marionette/test_accessiblecaret_selection_mode.py
UNKNOWN TEST: js/src/tests/test262/built-ins/Reflect/set/return-abrupt-from-property-key.js
UNKNOWN TEST: js/src/tests/test262/built-ins/RegExp/CharacterClassEscapes/character-class-whitespace-class-escape.js
UNKNOWN TEST: js/src/tests/test262/built-ins/Proxy/set/trap-is-undefined.js
UNKNOWN TEST: js/src/tests/test262/built-ins/Proxy/construct/return-is-abrupt.js
UNKNOWN TEST: js/src/tests/test262/built-ins/Atomics/notify/notify-zero.js
UNKNOWN TEST: js/src/tests/test262/built-ins/Reflect/getOwnPropertyDescriptor/undefined-own-property.js
UNKNOWN TEST: js/src/tests/test262/built-ins/Object/fromEntries/iterator-closed-for-null-entry.js
UNKNOWN TEST: js/src/tests/test262/built-ins/Reflect/set/creates-a-data-descriptor.js
UNKNOWN TEST: toolkit/components/microformats/test/marionette/test_interface.py
UNKNOWN TEST: js/src/tests/test262/intl402/RelativeTimeFormat/prototype/format/en-us-numeric-auto.js
UNKNOWN TEST: js/src/tests/test262/built-ins/Proxy/getPrototypeOf/extensible-target-return-handlerproto.js
UNKNOWN TEST: js/src/tests/test262/built-ins/Object/fromEntries/iterator-not-closed-for-throwing-done-accessor.js
UNKNOWN TEST: js/src/tests/non262/Date/time-zone-2038-pst.js
UNKNOWN TEST: js/src/tests/test262/language/directive-prologue/func-expr-no-semi-runtime.js
UNKNOWN TEST: js/src/tests/test262/built-ins/Proxy/set/boolean-trap-result-is-false-undefined-return-false.js
UNKNOWN TEST: layout/base/tests/marionette/test_accessiblecaret_cursor_mode.py
UNKNOWN TEST: js/src/tests/test262/built-ins/Atomics/wake/bigint/wake-all-on-loc.js
UNKNOWN TEST: js/src/tests/test262/built-ins/Reflect/defineProperty/defineProperty.js
UNKNOWN TEST: js/src/tests/test262/built-ins/Reflect/getOwnPropertyDescriptor/symbol-property.js
UNKNOWN TEST: js/src/tests/test262/built-ins/Proxy/get/return-trap-result.js
UNKNOWN TEST: js/src/tests/test262/intl402/RelativeTimeFormat/prototype/formatToParts/pl-pl-style-narrow.js
UNKNOWN TEST: js/src/tests/test262/built-ins/Object/fromEntries/empty-iterable.js
UNKNOWN TEST: js/src/tests/non262/Intl/DateTimeFormat/format_timeZone-non-meta.js
UNKNOWN TEST: js/src/tests/test262/built-ins/Atomics/wake/count-tointeger-throws-then-wake-throws.js
UNKNOWN TEST: js/src/tests/test262/language/eval-code/direct/new.target-fn.js
UNKNOWN TEST: js/src/tests/test262/built-ins/RegExp/CharacterClassEscapes/character-class-whitespace-class-escape-plus-quantifier-flags-u.js
UNKNOWN TEST: js/src/tests/test262/built-ins/Reflect/ownKeys/return-on-corresponding-order-large-index.js
UNKNOWN TEST: js/src/tests/test262/built-ins/Atomics/wake/negative-count.js
UNKNOWN TEST: js/src/tests/test262/built-ins/Atomics/wait/bigint/no-spurious-wakeup-on-or.js
UNKNOWN TEST: js/src/tests/non262/Proxy/regress-bug950407.js
UNKNOWN TEST: js/src/tests/test262/built-ins/Reflect/getPrototypeOf/null-prototype.js
UNKNOWN TEST: js/src/tests/test262/intl402/RelativeTimeFormat/prototype/format/pl-pl-style-short.js
UNKNOWN TEST: js/src/tests/test262/built-ins/RegExp/CharacterClassEscapes/character-class-non-digit-class-escape-plus-quantifier-flags-u.js
UNKNOWN TEST: js/src/tests/non262/Date/time-zones.js
UNKNOWN TEST: js/src/tests/test262/built-ins/Proxy/ownKeys/return-type-throws-array.js
UNKNOWN TEST: js/src/tests/test262/language/literals/string/7.8.4-12-s-strict.js
UNKNOWN TEST: js/src/tests/test262/built-ins/Object/fromEntries/iterator-closed-for-throwing-entry-value-accessor.js
UNKNOWN TEST: js/src/tests/test262/built-ins/Proxy/set/return-true-target-property-accessor-is-configurable-set-is-undefined.js
UNKNOWN TEST: js/src/tests/test262/built-ins/Object/fromEntries/iterator-closed-for-string-entry.js
UNKNOWN TEST: js/src/tests/non262/Date/reset-time-zone-cache-same-offset.js
UNKNOWN TEST: js/src/tests/test262/built-ins/Proxy/get/not-same-value-configurable-false-writable-false-throws.js
UNKNOWN TEST: js/src/tests/test262/built-ins/RegExp/CharacterClassEscapes/character-class-digit-class-escape.js
UNKNOWN TEST: toolkit/components/microformats/test/marionette/test_standards.py
UNKNOWN TEST: js/src/tests/test262/intl402/RelativeTimeFormat/prototype/format/value-symbol.js
UNKNOWN TEST: js/src/tests/test262/built-ins/Atomics/wait/symbol-for-timeout-throws-agent.js
UNKNOWN TEST: js/src/tests/test262/built-ins/RegExp/CharacterClassEscapes/character-class-non-word-class-escape-plus-quantifier.js
UNKNOWN TEST: js/src/tests/non262/Date/time-zones-posix.js
UNKNOWN TEST: js/src/tests/test262/built-ins/Proxy/has/return-true-target-prop-exists.js
UNKNOWN TEST: js/src/tests/test262/language/directive-prologue/set-accsr-inside-func-expr-runtime.js
UNKNOWN TEST: js/src/tests/non262/Date/15.9.5.7.js
UNKNOWN TEST: js/src/tests/test262/built-ins/Atomics/wake/bigint/non-bigint64-typedarray-throws.js
UNKNOWN TEST: js/src/tests/test262/intl402/RelativeTimeFormat/constructor/constructor/options-toobject.js
UNKNOWN TEST: js/src/tests/test262/intl402/RelativeTimeFormat/prototype/formatToParts/value-symbol.js
UNKNOWN TEST: js/src/tests/test262/intl402/RelativeTimeFormat/constructor/supportedLocalesOf/options-undefined.js
UNKNOWN TEST: js/src/tests/test262/built-ins/Reflect/preventExtensions/return-boolean-from-proxy-object.js
UNKNOWN TEST: js/src/tests/test262/built-ins/Proxy/getPrototypeOf/not-extensible-not-same-proto-throws.js
UNKNOWN TEST: js/src/tests/test262/built-ins/Proxy/setPrototypeOf/return-abrupt-from-target-getprototypeof.js
UNKNOWN TEST: js/src/tests/test262/intl402/RelativeTimeFormat/prototype/formatToParts/result-type.js
UNKNOWN TEST: js/src/tests/test262/built-ins/Reflect/defineProperty/return-abrupt-from-result.js
UNKNOWN TEST: js/src/tests/test262/built-ins/Reflect/get/get.js
UNKNOWN TEST: js/src/tests/test262/built-ins/Reflect/isExtensible/return-abrupt-from-result.js
UNKNOWN TEST: js/src/tests/test262/built-ins/Object/fromEntries/iterator-not-closed-for-uncallable-next.js
UNKNOWN TEST: js/src/tests/test262/built-ins/Proxy/set/boolean-trap-result-is-false-string-return-false.js
UNKNOWN TEST: js/src/tests/test262/built-ins/Proxy/defineProperty/null-handler.js
UNKNOWN TEST: js/src/tests/non262/Date/time-zones-historic.js
UNKNOWN TEST: js/src/tests/test262/built-ins/Proxy/getOwnPropertyDescriptor/resultdesc-is-invalid-descriptor.js
UNKNOWN TEST: js/src/tests/test262/built-ins/Proxy/has/trap-is-not-callable-using-with.js
UNKNOWN TEST: js/src/tests/test262/intl402/RelativeTimeFormat/constructor/supportedLocalesOf/options-localeMatcher-invalid.js
UNKNOWN TEST: js/src/tests/non262/regress/regress-665355.js
UNKNOWN TEST: js/src/tests/non262/Date/time-zone-pst.js
warning: no tasks could be resolved from flavor 'python' and subsuite 'try'
warning: no tasks could be resolved from flavor 'marionette'
warning: no tasks could be resolved from flavor 'crashtest'
Found 214 test files that cover these changes (21 tasks to be scheduled)

image/Image.cpp

https://treeherder.mozilla.org/#/jobs?repo=try&revision=fd3040eb033d86e3305d3eaff04cc8843f997ec1

Tests: Run looks OK, with more failures than the previous one. Passing: most of mochitest, xpcshell and firefox-ui-functional. Also passing: web-platform-tests (but not web-platform-tests-reftests or web-platform-tests-wdspec.

Test failures:

  • gtest same as above [1]
  • marionette and marionette-headless: same as above [2]
  • xpcshell on Android: same as above [3]
  • mochitest-clipboard: No tests were found for flavor 'chrome' and the following manifest filters:. It fails like this for linux, windows and OSX, but it succeeds on Android. Since it only found 1 test to run on Android, I think all the tests have been skipped, and so the harness should work with per-test. [4]
  • mochitest-browser-chrome: Test failures on linux and OSX. Works on windows.
  • mochitest on Android: test failures. [5]
  • firefox-ui-functional-remote: Test failures on windows. All other functional tests work. [5]
  • xpcshell on Android: failed with no tests to run using specified combination of filters. All tests are skipped. [3]

Missing tests:: a lot of UNKNOWN TEST for jsreftests. I suspect this is caused by old coverage data, or files being moved.

Artifact builds enabled, pass --no-artifact to disable
Found 2 modified source files with test coverage:
         image/Image.cpp
         tools/tryselect/test/test_tasks.py
Found 6 modified source files with no coverage:
         tools/tryselect/mach_commands.py
         tools/tryselect/selectors/coverage.py
         tools/tryselect/selectors/fuzzy.py
         tools/tryselect/tasks.py
         tools/tryselect/test/python.ini
         tools/tryselect/test/test_fuzzy.py
Running 117 individual test files.
Running 10 test chunks.
         linux firefox-ui-functional-local
         linux firefox-ui-functional-remote
         linux gtest
         linux marionette
         linux marionette-headless
         windows firefox-ui-functional-local
         windows firefox-ui-functional-remote
         windows gtest
         windows marionette
         windows marionette-headless
UNKNOWN TEST: js/src/tests/test262/language/expressions/new.target/value-via-fpcall.js
UNKNOWN TEST: js/src/tests/test262/intl402/RelativeTimeFormat/prototype/format/en-us-style-short.js
UNKNOWN TEST: js/src/tests/test262/built-ins/Proxy/setPrototypeOf/trap-is-undefined-or-null.js
UNKNOWN TEST: js/src/tests/test262/built-ins/Proxy/defineProperty/call-parameters.js
UNKNOWN TEST: js/src/tests/test262/built-ins/RegExp/CharacterClassEscapes/character-class-non-digit-class-escape.js
UNKNOWN TEST: testing/web-platform/tests/css/css-backgrounds/box-shadow-syntax-001.html
UNKNOWN TEST: layout/base/tests/marionette/test_accessiblecaret_selection_mode.py
UNKNOWN TEST: js/src/tests/test262/built-ins/Reflect/getOwnPropertyDescriptor/undefined-own-property.js
UNKNOWN TEST: layout/base/tests/marionette/test_accessiblecaret_cursor_mode.py
UNKNOWN TEST: js/src/tests/test262/built-ins/Object/fromEntries/iterator-not-closed-for-uncallable-next.js
UNKNOWN TEST: js/src/tests/test262/intl402/RelativeTimeFormat/prototype/format/pl-pl-style-short.js
UNKNOWN TEST: toolkit/components/microformats/test/marionette/test_modules.py
UNKNOWN TEST: toolkit/components/microformats/test/marionette/test_interface.py
UNKNOWN TEST: toolkit/components/microformats/test/marionette/test_standards.py
UNKNOWN TEST: js/src/tests/test262/built-ins/Reflect/get/get.js
UNKNOWN TEST: js/src/tests/test262/built-ins/RegExp/CharacterClassEscapes/character-class-word-class-escape-flags-u.js
UNKNOWN TEST: js/src/tests/test262/intl402/RelativeTimeFormat/constructor/supportedLocalesOf/options-localeMatcher-invalid.js
warning: no tasks could be resolved from flavor 'marionette'
warning: no tasks could be resolved from flavor 'python' and subsuite 'try'
Found 205 test files that cover these changes (59 tasks to be scheduled)

mochitest accessibility

accessible/tests/mochitest/jsat/test_content_integration.html

https://treeherder.mozilla.org/#/jobs?repo=try&revision=3429ad812ce4267273bfe7934b4c1737be947cc6

Tests: Tests a11y ran ok.

Other errors: Test verification task TV failed on all platforms with error Per-test run could not find requested test 'tools/tryselect/test/test_tasks.py'. It's right, that file is not listed with test-verify it's part of the python/try suite. This means the TV task will fail when the targeted tests are part of different suites. [6]

Artifact builds enabled, pass --no-artifact to disable
Found 2 modified source files with test coverage:
         accessible/tests/mochitest/jsat/test_content_integration.html
         tools/tryselect/test/test_tasks.py
Found 6 modified source files with no coverage:
         tools/tryselect/mach_commands.py
         tools/tryselect/selectors/coverage.py
         tools/tryselect/selectors/fuzzy.py
         tools/tryselect/tasks.py
         tools/tryselect/test/python.ini
         tools/tryselect/test/test_fuzzy.py
Running 2 individual test files.
Could not find any test chunks to run.
warning: no tasks could be resolved from flavor 'python' and subsuite 'try'
Found 2 test files that cover these changes (9 tasks to be scheduled)

crashtest

accessible/tests/crashtests/471493.xul

NO TRY PUSH

Tests: Nothing was run. This looks like we collect no coverage data for the crashtests. Also, crashtests don't seem to be runnable on a per-test basis. [7]

Artifact builds enabled, pass --no-artifact to disable
Found 2 modified source files with test coverage:
         accessible/tests/crashtests/471493.xul
         tools/tryselect/test/test_tasks.py
Found 6 modified source files with no coverage:
         tools/tryselect/mach_commands.py
         tools/tryselect/selectors/coverage.py
         tools/tryselect/selectors/fuzzy.py
         tools/tryselect/tasks.py
         tools/tryselect/test/python.ini
         tools/tryselect/test/test_fuzzy.py
Running 2 individual test files.
Could not find any test chunks to run.
warning: no tasks could be resolved from flavor 'python' and subsuite 'try'
warning: no tasks could be resolved from flavor 'crashtest'
ERROR Did not find any matching tasks after filtering.

web platform tests

testing/web-platform/tests/webaudio/the-audio-api/the-constantsourcenode-interface/constant-source-output.html

https://treeherder.mozilla.org/#/jobs?repo=try&revision=fc84d941e063ed9adff40610f3ff38bc666fdb56

Tests: wpt1 succeeds on all platforms in running selected test.

Failures: Wd and Wr1 test tasks fail with no tests ran. [3]

Other errors: Test verification task TVw fails on all platforms with error message Per-test run could not find requested web-platform test 'tools/tryselect/test/test_tasks.py'. It's right, that file is not a web platform test, it's part of the python/try suite. This means the TVw task will fail when the tests are part of different suites. [6]

Artifact builds enabled, pass --no-artifact to disable
Found 2 modified source files with test coverage:
         testing/web-platform/tests/webaudio/the-audio-api/the-constantsourcenode-interface/constant-source-output.html
         tools/tryselect/test/test_tasks.py
Found 6 modified source files with no coverage:
         tools/tryselect/mach_commands.py
         tools/tryselect/selectors/coverage.py
         tools/tryselect/selectors/fuzzy.py
         tools/tryselect/tasks.py
         tools/tryselect/test/python.ini
         tools/tryselect/test/test_fuzzy.py
Running 2 individual test files.
Could not find any test chunks to run.
warning: no tasks could be resolved from flavor 'python' and subsuite 'try'
Found 2 test files that cover these changes (13 tasks to be scheduled)

jsreftest

js/src/tests/non262/TypedObject/method_from.js

NO PUSH

Tests: Nothing was run. This may be caused by old chunk mapping coverage data. It worked in the past, so I'll investigate further. [4]

2 files updated, 0 files merged, 0 files removed, 0 files unresolved
Artifact builds enabled, pass --no-artifact to disable
Found 1 modified source files with test coverage:
         tools/tryselect/test/test_tasks.py
Found 7 modified source files with no coverage:
         js/src/tests/non262/TypedObject/method_from.js
         tools/tryselect/mach_commands.py
         tools/tryselect/selectors/coverage.py
         tools/tryselect/selectors/fuzzy.py
         tools/tryselect/tasks.py
         tools/tryselect/test/python.ini
         tools/tryselect/test/test_fuzzy.py
Running 1 individual test files.
Could not find any test chunks to run.
warning: no tasks could be resolved from flavor 'python' and subsuite 'try'
ERROR Did not find any matching tasks after filtering.

mochitest-chrome

browser/components/shell/test/headless.html

https://treeherder.mozilla.org/#/jobs?repo=try&revision=5c1d25b7e725baa5ec9525bee31310448271dcff

Tests: mochitest-chrome works fine.

Test failures:

  • TV failed because of [6]
  • test-android-em-4.3-arm7-api-16/opt-mochitest-chrome-1 failed because of [3]
Artifact builds enabled, pass --no-artifact to disable
Found 2 modified source files with test coverage:
         browser/components/shell/test/headless.html
         tools/tryselect/test/test_tasks.py
Found 6 modified source files with no coverage:
         tools/tryselect/mach_commands.py
         tools/tryselect/selectors/coverage.py
         tools/tryselect/selectors/fuzzy.py
         tools/tryselect/tasks.py
         tools/tryselect/test/python.ini
         tools/tryselect/test/test_fuzzy.py
Running 2 individual test files.
Could not find any test chunks to run.
warning: no tasks could be resolved from flavor 'python' and subsuite 'try'
Found 2 test files that cover these changes (10 tasks to be scheduled)

cppunittest

js/src/jit/Linker.cpp

https://treeherder.mozilla.org/#/jobs?repo=try&revision=0624444898ab3a93c75f86fe550b74ff2575ec1d

Tests: Sill waiting for the try push results.

Artifact builds enabled, pass --no-artifact to disable
Found 2 modified source files with test coverage:
         js/src/jit/Linker.cpp
         tools/tryselect/test/test_tasks.py
Found 6 modified source files with no coverage:
         tools/tryselect/mach_commands.py
         tools/tryselect/selectors/coverage.py
         tools/tryselect/selectors/fuzzy.py
         tools/tryselect/tasks.py
         tools/tryselect/test/python.ini
         tools/tryselect/test/test_fuzzy.py
Running 12 individual test files.
Running 18 test chunks.
         linux cppunit
         linux firefox-ui-functional-local
         linux firefox-ui-functional-remote
         linux gtest
         linux jittest-1
         linux jittest-2
         linux jittest-3
         linux jittest-4
         linux jittest-5
         linux jittest-6
         linux marionette
         linux marionette-headless
         windows cppunit
         windows firefox-ui-functional-local
         windows firefox-ui-functional-remote
         windows gtest
         windows marionette
         windows marionette-headless
UNKNOWN TEST: toolkit/components/microformats/test/marionette/test_modules.py
UNKNOWN TEST: layout/base/tests/marionette/test_accessiblecaret_selection_mode.py
UNKNOWN TEST: toolkit/components/microformats/test/marionette/test_interface.py
UNKNOWN TEST: layout/base/tests/marionette/test_accessiblecaret_cursor_mode.py
UNKNOWN TEST: toolkit/components/microformats/test/marionette/test_standards.py
warning: no tasks could be resolved from flavor 'marionette'
warning: no tasks could be resolved from flavor 'python' and subsuite 'try'
Found 100 test files that cover these changes (47 tasks to be scheduled)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment