|
f30cfd18
|
2022-08-16T16:50:33
|
|
Perf and gold tests detect test SKIPs from json results.
Make angle_test_util.RunTestSuite() gets and returns json results
(using --isolated-script-test-output).
perf tests has an additional special case
(filed https://anglebug.com/7578):
Previously when metrics were missing it was assumed that this was due to
tests being skipped - which seems to currently be the case except for
the special case of GLMark2Benchmark* tests. Those produce .fps/.score
metrics instead of the metrics expected by this script. This CL keeps
those tests running (once) but then marks them as SKIP in test results
when no metrics are found (which is the behavior before this CL; it
might actually make more sense to count this as PASS because everything
matches our expectations).
We're still running them once, so if they FAIL we'll get notified.
gold tests:
Use json results file instead of checking for '[ SKIPPED ]' in stdout.
Bug: angleproject:7299
Bug: angleproject:7578
Change-Id: Ia751784ad1aa94dc855c8b58ebfe5ba3e06e462f
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3826167
Reviewed-by: Amirali Abdolrashidi <abdolrashidi@google.com>
Commit-Queue: Roman Lavrov <romanl@google.com>
|
|
|
c5efd2b5
|
2022-08-15T16:20:25
|
|
Improve Ctrl-c handling.
1. Restore original handlers after subprocess terminates. Otherwise the
installed handlers persist and will swallow sigint outside of
subprocess invocations.
2. Raise KeyboardInterrupt when subprocesses terminate to interrupt the
script itself. run_perf_tests.py allows for some subprocess failures
(--max-errors, with non-zero default). This results in ctrl-c
interrupting subprocesses but the script just marks the test as FAIL
and keeps running.
Bug: angleproject:7299
Change-Id: I2917008baa389491c7591596fc14ee139fa6417e
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3831887
Commit-Queue: Roman Lavrov <romanl@google.com>
Reviewed-by: Amirali Abdolrashidi <abdolrashidi@google.com>
Reviewed-by: Yuly Novikov <ynovikov@chromium.org>
|
|
|
64c652d7
|
2022-08-12T18:14:39
|
|
Use stdout instead of (default) stderr in logging.StreamHandler
Currently logging output appears before stdout on bots,
makes logs so confusing as subprocess logs get out of order with logs
from python. Using StreamHandler(stdout) seems to be the common practice
in other python scripts in code search.
Note: subprocess's stderr is redirected to stdout:
https://crsrc.org/c/third_party/angle/src/tests/py_utils/angle_test_util.py;drc=2de8bb42a9ef93dfafe476e7dc81643285fa7ef1;l=92
Bug: angleproject:7299
Change-Id: I825f16d9c479f33a280d8fdbafb8297cda0c18b8
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3827957
Auto-Submit: Roman Lavrov <romanl@google.com>
Reviewed-by: Amirali Abdolrashidi <abdolrashidi@google.com>
Commit-Queue: Amirali Abdolrashidi <abdolrashidi@google.com>
|
|
|
2de8bb42
|
2022-08-03T18:15:52
|
|
Converge run wrappers into angle_test_util.RunTestSuite
Bug: angleproject:7299
Change-Id: Ifa882028a6f49579b4a9c6893053cb3bee8d83fd
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3808302
Reviewed-by: Jamie Madill <jmadill@chromium.org>
Reviewed-by: Amirali Abdolrashidi <abdolrashidi@google.com>
Commit-Queue: Roman Lavrov <romanl@google.com>
|
|
|
19c6600c
|
2022-08-02T12:45:57
|
|
Refactoring: move a few common functions to angle_test_util.
Drop ScriptTest, Jamie says no longer needed.
Bug: angleproject:7299
Change-Id: I32bdd1d70701f63b578b4f79512868daac3ad520
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3805381
Reviewed-by: Jamie Madill <jmadill@chromium.org>
Commit-Queue: Roman Lavrov <romanl@google.com>
|
|
|
ebfb9b9f
|
2022-04-26T10:37:52
|
|
Configure logging in perf and gold tests to include time.
Currently no timestamp is logged:
[I2022-04-25T18:45:14.761163Z 548 0 cmd_stream.go:336] {cmd}
INFO:root:adb --version ...
Before switching to android_helper relative timestamps in seconds
were printed, set up by catapult. Timestamps proposed in this CL are
more consistent with outer logging but keep prefix short. Example:
[I2022-04-25T18:45:14.761163Z 548 0 cmd_stream.go:336] {cmd}
I18:45:14.999999Z adb --version ...
Bug: angleproject:6854
Change-Id: I8ef7c9ae44eb5dba564a41456ce9417e3fe0f06c
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3606913
Reviewed-by: Jamie Madill <jmadill@chromium.org>
Commit-Queue: Roman Lavrov <romanl@google.com>
|