|
e1bc8de3
|
2021-09-08T07:59:37
|
|
Change links from 'master' to 'main' branch.
Bug: chromium:1226949
Change-Id: Ie7b28b2fa094cf0c0a407968d681e580519cbb88
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3148210
Reviewed-by: Geoff Lang <geofflang@chromium.org>
Commit-Queue: Jamie Madill <jmadill@chromium.org>
|
|
09bd5578
|
2021-06-21T23:47:15
|
|
Test runner: Capture test stderr
With this change, stderr is also output when a test fails, including
logs such as those originating from UNIMPLEMENTED().
Bug: angleproject:6077
Change-Id: Ice1e1f9aff26acfb630aef99865503643f0a9658
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2976184
Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org>
Reviewed-by: Jamie Madill <jmadill@chromium.org>
|
|
|
4194d400
|
2021-06-21T13:08:56
|
|
Test Runner: Add maximum failure count.
This will make the test runner early exit after a certain count
of failures. This will prevent the bots from running tens of
thousands of tests when the whole test run is corrupt.
Bug: angleproject:6075
Change-Id: I8f5c19ea694a5ad3a5d5df1429d2ba4b1fe868e3
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2976179
Commit-Queue: Jamie Madill <jmadill@chromium.org>
Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org>
Reviewed-by: Jonah Ryan-Davis <jonahr@google.com>
|
|
|
85efb9d5
|
2021-01-26T14:39:19
|
|
Log dEQP QPA files as test artifacts.
This adds artifact output to the test runner. We add a fake test
at the start of a test run that owns the artifacts.
Bug: angleproject:5236
Change-Id: Ice8001bf1f2aafbd8123fee76e0e7fcc3e5a8a0c
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2657535
Commit-Queue: Jamie Madill <jmadill@chromium.org>
Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org>
Reviewed-by: Mohan Maiya <m.maiya@samsung.com>
Reviewed-by: Yuly Novikov <ynovikov@chromium.org>
|
|
|
c06a424f
|
2020-11-02T21:13:39
|
|
Test Runner: Robustly handle unexpected crashes.
This adds the ability for the test runner to parse unexpected crashes
from the stdout. It also processes the stdout to determine which tests
failed. Tests that run after the crash are re-tried in a follow-up
child process.
Will allow for the test runner to handle very crashy test suites and
also processes crashes from win-asan and other configs where it is
harder to intercept crashes.
Bug: angleproject:5251
Change-Id: Iee03130622571580cb7910f4fb097fe3659d75ec
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2513288
Commit-Queue: Jamie Madill <jmadill@chromium.org>
Reviewed-by: Yuly Novikov <ynovikov@chromium.org>
Reviewed-by: Tim Van Patten <timvp@google.com>
|
|
|
59fbb989
|
2020-11-02T13:27:52
|
|
Test Runner: Add ability to retry flaky tests.
Bug: angleproject:5273
Change-Id: Ie89559bb0897a04213981aa8fe4e2f2bfe78959a
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2513287
Commit-Queue: Jamie Madill <jmadill@chromium.org>
Reviewed-by: Yuly Novikov <ynovikov@chromium.org>
Reviewed-by: Tim Van Patten <timvp@google.com>
|
|
|
23624c53
|
2020-10-05T21:07:02
|
|
Test Runner: More docs.
Bug: angleproject:3162
Change-Id: I5b282d7815973cef9f7e029bc3c65cf42729e009
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2451596
Commit-Queue: Jamie Madill <jmadill@chromium.org>
Reviewed-by: Yuly Novikov <ynovikov@chromium.org>
|
|
831056bf
|
2020-08-19T12:04:59
|
|
Fix typo in test harness README
Larget->larger
Bug: angleproject:3162
Change-Id: I7ee84b714af179f97db1aa61b58ed946b28373bc
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2364191
Reviewed-by: Jamie Madill <jmadill@chromium.org>
Commit-Queue: Jamie Madill <jmadill@chromium.org>
|
|
|
debe4683
|
2020-07-15T14:36:33
|
|
Test Runner: Accept Chromium args.
This change lets the test runner accept the same isolated script
parameter syntax as Chromium. It keeps the old parameter names so
the tests accept both. This will let us more easily integrate with
the Chromium Android test_runner.py script.
Also update the README which was missing a few flags.
Bug: angleproject:3162
Change-Id: Ic22b289a095418ffdaa19a04caa1b45c6c657872
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2300530
Reviewed-by: Jonah Ryan-Davis <jonahr@google.com>
Reviewed-by: Yuly Novikov <ynovikov@chromium.org>
Commit-Queue: Jamie Madill <jmadill@chromium.org>
|
|
|
238adb7a
|
2020-01-13T17:21:53
|
|
Fix test harness running disabled tests.
Adds special handling for the DISABLED_ GTest test name. Fixes the test
for the test suite harness itself when run with the new "--bot-mode"
flag.
Bug: angleproject:3162
Change-Id: Idf34098d2d6bcb78263773be7b01c55df9ec624b
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/1998660
Commit-Queue: Jamie Madill <jmadill@chromium.org>
Reviewed-by: Yuly Novikov <ynovikov@chromium.org>
|
|
|
5407aaa0
|
2019-12-16T15:50:12
|
|
Re-land "Add new test runner harness." (#2)
Re-land #2 changes:
* export labels are fixed for the CFI build
* crash test disabled because of flakiness and issues with asan
Re-land changes:
* Unit test is suppressed in ASAN
* --deqp-case is fixed
* Debug layer errors should correctly work with failure expectations
Original message:
The ANGLE test harness is a harness around GoogleTest that provides
functionality similar to the Chromium test harness. It supports:
* splitting a test set into shards
* catching and reporting crashes and timeouts
* outputting to the Chromium JSON test results format
* multi-process execution
Unit tests are added in test_utils_unittest.cpp.
Bug: angleproject:3162
Bug: chromium:1030192
Change-Id: I71d66a407ea0e53d73cbe75b5b4bfb9e73791534
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/1965091
Commit-Queue: Jamie Madill <jmadill@chromium.org>
Reviewed-by: Jonah Ryan-Davis <jonahr@google.com>
|
|
|
b92ec244
|
2019-12-06T15:08:54
|
|
Revert "Re-land "Add new test runner harness.""
This reverts commit e20560faf1de86c01198143ef7733a12a098a90b.
Reason for revert: Now fails on Linux CFI and also is a bit flaky.
Original change's description:
> Re-land "Add new test runner harness."
>
> Re-land changes:
>
> * Unit test is suppressed in ASAN
> * --deqp-case is fixed
> * Debug layer errors should correctly work with failure expectations
>
> Original message:
>
> The ANGLE test harness is a harness around GoogleTest that provides
> functionality similar to the Chromium test harness. It supports:
>
> * splitting a test set into shards
> * catching and reporting crashes and timeouts
> * outputting to the Chromium JSON test results format
> * multi-process execution
>
> Unit tests are added in test_utils_unittest.cpp.
>
> Bug: angleproject:3162
> Change-Id: I841f2b5dfe51f7f44dac68324bdf6afd418b8bfb
> Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/1948240
> Reviewed-by: Jonah Ryan-Davis <jonahr@google.com>
> Reviewed-by: Yuly Novikov <ynovikov@chromium.org>
> Commit-Queue: Jamie Madill <jmadill@chromium.org>
TBR=ynovikov@chromium.org,ianelliott@google.com,jonahr@google.com,jmadill@chromium.org
Change-Id: Ibfd65b8b18ead3a232abb6cb75fd6489b0ff5f38
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: angleproject:3162
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/1954570
Reviewed-by: Jamie Madill <jmadill@chromium.org>
Commit-Queue: Jamie Madill <jmadill@chromium.org>
|
|
|
e20560fa
|
2019-12-04T13:18:36
|
|
Re-land "Add new test runner harness."
Re-land changes:
* Unit test is suppressed in ASAN
* --deqp-case is fixed
* Debug layer errors should correctly work with failure expectations
Original message:
The ANGLE test harness is a harness around GoogleTest that provides
functionality similar to the Chromium test harness. It supports:
* splitting a test set into shards
* catching and reporting crashes and timeouts
* outputting to the Chromium JSON test results format
* multi-process execution
Unit tests are added in test_utils_unittest.cpp.
Bug: angleproject:3162
Change-Id: I841f2b5dfe51f7f44dac68324bdf6afd418b8bfb
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/1948240
Reviewed-by: Jonah Ryan-Davis <jonahr@google.com>
Reviewed-by: Yuly Novikov <ynovikov@chromium.org>
Commit-Queue: Jamie Madill <jmadill@chromium.org>
|
|
|
8f31872c
|
2019-12-03T18:34:52
|
|
Revert "Add new test runner harness."
This reverts commit fb40d231c3e2ee7c38f8445ef5defc0ab0f5f15d.
Reason for revert: Has a bug with the ASan build and also has a few
bugs with ANGLE standalone test expectations an filter.
Bug: chromium:1030192
Bug: angleproject:4193
Original change's description:
> Add new test runner harness.
>
> The ANGLE test harness is a harness around GoogleTest that provides
> functionality similar to the Chromium test harness. It supports:
>
> * splitting a test set into shards
> * catching and reporting crashes and timeouts
> * outputting to the Chromium JSON test results format
> * multi-process execution
>
> Unit tests are added in test_utils_unittest.cpp.
>
> Bug: angleproject:3162
> Change-Id: Idb15f113de8eb32db12bc93542de93b08d7c1447
> Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/1478016
> Commit-Queue: Jamie Madill <jmadill@chromium.org>
> Reviewed-by: Yuly Novikov <ynovikov@chromium.org>
TBR=ynovikov@chromium.org,jonahr@google.com,jmadill@chromium.org
Change-Id: I647e747571784b1ca7c1d0687193c70a63eb08d1
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: angleproject:3162
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/1947456
Reviewed-by: Jamie Madill <jmadill@chromium.org>
Commit-Queue: Jamie Madill <jmadill@chromium.org>
|
|
|
fb40d231
|
2019-12-02T16:39:18
|
|
Add new test runner harness.
The ANGLE test harness is a harness around GoogleTest that provides
functionality similar to the Chromium test harness. It supports:
* splitting a test set into shards
* catching and reporting crashes and timeouts
* outputting to the Chromium JSON test results format
* multi-process execution
Unit tests are added in test_utils_unittest.cpp.
Bug: angleproject:3162
Change-Id: Idb15f113de8eb32db12bc93542de93b08d7c1447
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/1478016
Commit-Queue: Jamie Madill <jmadill@chromium.org>
Reviewed-by: Yuly Novikov <ynovikov@chromium.org>
|