kc3-lang/angle/src/tests/test_utils/runner

src/tests/test_utils/runner

Log

Author Commit Date CI Message
Roman Lavrov 2fcd88e1 2025-04-08T09:30:59 Tests: order bot mode batch configs alphabetically Currently ordered by hashmap key ordering which shuffles the order even with --max-processes=1. Adding a sort to make this deterministic. Bug: angleproject:408276172 Change-Id: Ice96e350ba4247a1f40870e275d04d2f654bc4f5 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/6438024 Reviewed-by: Yuly Novikov <ynovikov@chromium.org> Commit-Queue: Roman Lavrov <romanl@google.com>
Mark Lobodzinski d50cd35e 2025-02-20T08:29:07 Tests: Add expectations file for angle_trace_tests Skipping trace tests was handled in the TracePerfTest source and is now specified in angle_trace_tests_expectations.txt. Bug: b/398048048 Change-Id: Idbf5e95e74699374eb46330dc3880a0c2dfc9bc1 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/6302912 Reviewed-by: Cody Northrop <cnorthrop@google.com> Reviewed-by: Yuly Novikov <ynovikov@chromium.org> Commit-Queue: Yuly Novikov <ynovikov@chromium.org>
Roman Lavrov 729b55ba 2025-03-04T10:45:34 Tests: Fix --gtest_repeat reporting Pass on intermediate Fail When running with --gtest_repeat=N, the same test is executed multiple times and the harness was overwriting a previous Fail status with a subsequent Pass. Also made Android logging more clear. Bug: None Change-Id: Ib1d71ca89e9e9b2af8f6a0df2c9e280b86143265 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/6320657 Reviewed-by: Yuxin Hu <yuxinhu@google.com>
Austin Annestrand 957bafb5 2025-01-30T11:49:41 Infra: Print stdout for TIMEOUT cases Bug: angleproject:392929827 Change-Id: I47625474e4ce9253a4cf275ea027290a6a2fbc33 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/6207671 Commit-Queue: Austin Annestrand <a.annestrand@samsung.com> Reviewed-by: Yuly Novikov <ynovikov@chromium.org> Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org>
Shashank Sharma 0c9ce8c7 2023-09-08T21:34:07 Trace-based testing app Modifying angle_trace_tests app to run traces automatically on invocation and dumps fps information. Also, compares the screenshot with the golden images if provided and dumps the histogram of image diff. Golden image naming format: <trace_name>_golden.png - Run traces - Capture screenshot - Record FPS value - Record image diff histogram Bug: b/270421213 Doc: doc/StandaloneBenchmark.md Change-Id: I42b8d229e1e358d48887f29f2424b9e771093ce9 Commit-Queue: Shashank Sharma <shawshanks@google.com> Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/6072349 Commit-Queue: Cody Northrop <cnorthrop@google.com> Auto-Submit: Shashank Sharma <shawshanks@google.com> Reviewed-by: Roman Lavrov <romanl@google.com> Reviewed-by: Cody Northrop <cnorthrop@google.com>
Roman Lavrov df90bbc5 2024-07-12T18:04:53 Refactoring: move angle::HashMap and HashSet to own header Underlying abseil includes pull in a large set of headers Bug: angleproject:42266508 Change-Id: Icee47143a8a59bb0795a054b67c0aa4ddcfca4d4 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/5704137 Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org> Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org>
Shahbaz Youssefi d193d51b 2024-06-17T22:46:08 Replace issue ids post migration to new issue tracker This change replaces anglebug.com/NNNN links. Bug: None Change-Id: I8ac3aec8d2a8a844b3d7b99fc0a6b2be8da31761 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/5637912 Reviewed-by: Geoff Lang <geofflang@chromium.org> Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org>
angle-autoroll 298abbc1 2024-02-16T08:50:43 Roll Chromium from 29bec8631d2f to 9d4a35b46e1e (1220 revisions) Amend the roll with http://crrev.com/c/5301614 courtesy agrieve@. https://chromium.googlesource.com/chromium/src.git/+log/29bec8631d2f..9d4a35b46e1e If this roll has caused a breakage, revert this CL and stop the roller using the controls here: https://autoroll.skia.org/r/chromium-angle-autoroll Please CC angle-team@google.com,geofflang@google.com on the revert to ensure that a human is aware of the problem. To file a bug in Chromium: https://bugs.chromium.org/p/chromium/issues/entry To file a bug in ANGLE: https://bugs.chromium.org/p/angleproject/issues/entry To report a problem with the AutoRoller itself, please file a bug: https://issues.skia.org/issues/new?component=1389291&template=1850622 Documentation for the AutoRoller is here: https://skia.googlesource.com/buildbot/+doc/main/autoroll/README.md Changed dependencies * build: https://chromium.googlesource.com/chromium/src/build.git/+log/2cece96e17..a11ae4664a * buildtools/linux64: git_revision:a2e2717ea670249a34b0de4b3e54f268d320bdfa..git_revision:8b973aa51d02aa1ab327100007d4070c24b862b0 * buildtools/mac: git_revision:a2e2717ea670249a34b0de4b3e54f268d320bdfa..git_revision:8b973aa51d02aa1ab327100007d4070c24b862b0 * buildtools/win: git_revision:a2e2717ea670249a34b0de4b3e54f268d320bdfa..git_revision:8b973aa51d02aa1ab327100007d4070c24b862b0 * testing: https://chromium.googlesource.com/chromium/src/testing/+log/c37870b5fd..5e8f58ef80 * third_party/abseil-cpp: https://chromium.googlesource.com/chromium/src/third_party/abseil-cpp/+log/d353dfb095..984707e65c * third_party/android_deps: https://chromium.googlesource.com/chromium/src/third_party/android_deps/+log/f39e9d8c1a..ad0b8e7a0e * third_party/catapult: https://chromium.googlesource.com/catapult.git/+log/51589f432d..3d6c15240b * third_party/depot_tools: https://chromium.googlesource.com/chromium/tools/depot_tools.git/+log/a5a09d23fa..b1b1a43f07 * third_party/fuchsia-sdk/sdk: version:18.20240207.3.1..version:18.20240215.1.1 * third_party/r8: tp4vVuXzmyHJxDFlwxDb7RYZLLEufc3EnGTyOTCTNkgC..f7pPpoVTaC3WoBY5dURVrI-N1_4YZpSYipdiDUeWDAEC * third_party/siso: git_revision:3cd0e6e55246b8eca6646e33b124e54e67fc5660..git_revision:4bede4f49669d3a8bdaa8dbd01620c3a07ae0c4e * tools/clang: https://chromium.googlesource.com/chromium/src/tools/clang.git/+log/894ad691c6..47756615f5 No update to Clang. Bug: angleproject:6081, chromium:324916799 Change-Id: Ib097e6dca9374111632838beab4f0224ff6196fc Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/5302166 Commit-Queue: Yuly Novikov <ynovikov@chromium.org> Reviewed-by: Geoff Lang <geofflang@chromium.org>
Roman Lavrov cbdbe3bd 2024-01-26T10:05:41 deqp: Unused test expectations do not cause failure, only log deqp is the only suite that appears to be handling unused expectations this way. This causes an inconvenience where a test cannot be pre-emptively added to the expectations file. For example, a VK-GL-CTS roll might be adding a new test that is failing in some configurations. The expectation cannot be added to the expectations file pre-emptively because of the failure due to the unused expectation, but that is the more convenient way to solve this problem than patching the roll manually. Bug: angleproject:8507 Change-Id: If2ee65d9119043c42e8ca0705a1cc8c40827fae3 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/5238992 Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org> Commit-Queue: Roman Lavrov <romanl@google.com> Auto-Submit: Roman Lavrov <romanl@google.com> Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org>
Daniel Cheng bc7af34c 2023-12-06T00:15:33 Use googletest flag macros to access googletest flags. The implementation details of flags can change; fixing this proactively makes it easier to roll googletest. Bug: chromium:1409870 Change-Id: I5efa2487d9e38e122ca20f201748faf8e8050347 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/5092928 Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org> Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org> Auto-Submit: Daniel Cheng <dcheng@chromium.org>
Igor Nazarov c38f3374 2023-09-15T19:22:44 Fix "win-asan-test" random failures and timeouts While exact reason for the failures is unknown, it seems like disabling Vulkan Loader DLL unloading fixes the problem. More details in the issue 8347. This CL also sets the variable on the Linux platform, because it does not hurt and will probably help with similar issues. Bug: angleproject:8347 Change-Id: I398d55a7c13f1ce6790988983fdf8fee7cc5776c Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4868220 Commit-Queue: Igor Nazarov <i.nazarov@samsung.com> Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org>
Roman Lavrov a88635c4 2023-04-28T18:54:25 Android logs also go to stdout/err, line-buffered if possible Currently logs like WARN() or INFO() only go to logcat, which doesn't make them visible in console. This change makes them go to both logcat and stdout/err. Fix android_helper choking on missing json data output when test crashes and test runner doesn't have a chance to produce that file. We still have the stdout/err in this case, so print it. However, a crash (such as SIGSEGV or __builtin_trap()) doesn't flush output so the non-flushed part of the buffer doesn't get in there. AngleNativeTest redirects stdout/err to a file which enables fully-buffered (stdlib) mode. Change this to line-buffered when stdout path is set to /data/ (/sdcard/ is slow) and use /data/data/com.android.angle.test/tmp/ as the temp path in android_helper. This makes e.g. FATAL() << "message"; show up in the logs in CI or local runs. This currently only applies to trace and perf tests, other tests will continue using fully-buffered /stdout/ as chromium's test_runner uses /sdcard/Download/ for temp files. Bug: b/276742336 Change-Id: Ia622c610f776f501223b0cca11d5ec1976eb9887 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4493463 Reviewed-by: Cody Northrop <cnorthrop@google.com> Commit-Queue: Roman Lavrov <romanl@google.com>
Jamie Madill 968c429e 2023-01-04T14:18:11 Tests: Use ISOLATED_OUTDIR for test artifact directory. This should fix the missing CAS outputs on the bots. Bug: angleproject:7926 Change-Id: Id09e5ccc19a454e2c0bf63076788187f3c7ef24f Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4135158 Commit-Queue: Yuly Novikov <ynovikov@chromium.org> Commit-Queue: Jamie Madill <jmadill@chromium.org> Auto-Submit: Jamie Madill <jmadill@chromium.org> Reviewed-by: Yuly Novikov <ynovikov@chromium.org>
Roman Lavrov 056f8044 2022-10-11T11:58:20 Perf tests: write metrics to file and read from merge script. The idea behind is to save metrics in a structured output with full context so that we don't have to parse stdout and reconstruct that context (e.g. which test was running when the output was printed etc). Note: we already have a "histogram" file (addHistogramSample) written but it is structured using some UUID-based ids making its digestion more complicated (https://anglebug.com/7299#c3). This CL just writes events as one-json-per-line simplifying downstream handling and debugging. Just a sanity check / logging in the merge script for now. Bug: angleproject:7299 Change-Id: I56dada643eceef180ce9bb1aa9ae6a641ea41e4c Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3945112 Reviewed-by: Jamie Madill <jmadill@chromium.org> Commit-Queue: Roman Lavrov <romanl@google.com>
Jamie Madill e495e7fd 2022-10-14T13:52:47 Redo perf and trace test parameters. - Moves common argument parsing code into test_util. - Changes the perf test arg parsing to use the common functions. - Adds new --use-angle and --use-gl parameters to the trace tests. - Also adds new --offscreen and --vsync parameters to the traces. - Removes the now unneeded --enable-all-trace-tests argument. - Both --arg=value and --arg value work in test suites now. Now, instead of using --enable-all-trace-tests you can specify the backend with --use-angle=swiftshader, --offscreen, or combinations of those parameters. The test names are the same as they were before, but only the configured tests will run in a session. We could opt to simplify the test names in later CLs if we want to simplify the test running. Ideally we'd keep the perf reporting the same because then we'd keep the time series the same on the test infra. This also allows us to split up the trace tests into separate targets on the bots, which will better allow us to control the workloads and sampling of the tests. For example: - angle_perftests becomes - angle_perftests (microbenchmarks) - angle_trace_perf_vulkan_tests (traces with vulkan back-end) - angle_trace_perf_native_tests (traces with system GL) Bug: angleproject:7755 Change-Id: I537168f3a6de96425dfda05ed98220eff9b19b76 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3956937 Reviewed-by: Yuly Novikov <ynovikov@chromium.org> Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Cody Northrop <cnorthrop@google.com>
Roman Lavrov 6f037273 2022-09-28T18:46:13 Make last lines of batch stdout more visible in logs. So that it looks like this: https://gist.github.com/romanl-g/dd0d2f7e0eddf16fd802bed97789024c Bug: angleproject:7708 Change-Id: Ic92a9de6cba24637119ae9041e9126364d414948 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3923459 Commit-Queue: Roman Lavrov <romanl@google.com> Reviewed-by: Jamie Madill <jmadill@chromium.org> Auto-Submit: Roman Lavrov <romanl@google.com>
Geoff Lang 87ed2c9d 2022-08-11T11:19:17 TestSuiteTest: Generate more unique result file names. Speculative fix for flaky runs of TestSuiteTest. Flakes appear to read the results of other tests suggesting rand is not unique enough to protect against tests generating the same results file. Bug: chromium:1352134 Change-Id: Icadf64c557e23e6d87fd18aa60aa48373ecec9fd Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3827144 Commit-Queue: Geoff Lang <geofflang@chromium.org> Reviewed-by: Jonah Ryan-Davis <jonahr@google.com>
Roman Lavrov 3d0faa02 2022-08-09T16:48:06 Remove workaround affecting --list-tests --gtest_filter=X. One of the multiple workarounds added in https://crrev.com/c/3062240. It makes `--list-tests --gtest_filter=X` behave as if it was just `--list-tests`. Not sure why this was necessary, bots appear to work without it?.. Could be something that changed since that CL? Bug: angleproject:7508 Change-Id: I9988add40ddf3168e00cea2739e5a9209f633161 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3821374 Reviewed-by: Yuxin Hu <yuxinhu@google.com>
Roman Lavrov 6d0452b7 2022-08-10T13:34:29 Fix ANGLE_HAS_HISTOGRAMS following crrev.com/c/3817602 Bug: angleproject:7299 Change-Id: I1d6f97e61dfdbea04009cc08a17344a59e438e51 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3823618 Reviewed-by: Yuly Novikov <ynovikov@chromium.org> Commit-Queue: Roman Lavrov <romanl@google.com>
Roman Lavrov dc82ff15 2022-08-08T16:06:13 Add config angle_maybe_has_histograms to angle_perftests_common. ANGLE_HAS_HISTOGRAMS wasn't consistently defined in compilation targets. I was working on a prototype and ran into crashes due to an inline function in TestSuite (where this class is used) getting invalid offsets in some cases due to this :-O Shabi: """ angle_source_set makes something the equivalent of a temporary .a, so objects are not recompiled for every target that links to it """ Also change ANGLE_HAS_HISTOGRAMS to =1 or =0 as Yuly suggested so that we can check if it's defined in the header to avoid inconsistency. Bug: angleproject:7299 Change-Id: I907184a6ca2bc673f5bffc473b584d78625a11c8 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3817602 Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org> Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org> Auto-Submit: Roman Lavrov <romanl@google.com> Commit-Queue: Roman Lavrov <romanl@google.com>
Jamie Madill 08df7ce4 2022-07-28T10:59:30 Test Runner: Disable --bot-mode on Fuchsia. This flag is a bit hard to get going because of the differences in the Fuchsia system APIs. Disable it explicitly for now to avoid issues when we switch bot configs. Bug: angleproject:7312 Change-Id: I47cd038b198f265a6b60893faf6f50f02be49fda Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3792163 Reviewed-by: Yuly Novikov <ynovikov@chromium.org> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Jamie Madill b968b6c9 2022-07-26T14:34:41 Test Utils: Get executable path from command line. This simplifies the process fork test and will allow the test to work on Fuchsia. Bug: angleproject:3161 Change-Id: I5bb5424c44957f2faef1970b3cbc5e52b8aff630 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3788297 Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org> Commit-Queue: Yuly Novikov <ynovikov@chromium.org> Auto-Submit: Jamie Madill <jmadill@chromium.org> Reviewed-by: Yuly Novikov <ynovikov@chromium.org>
Roman Lavrov 2b43d630 2022-07-22T14:19:05 Remove unused (and incorrect on Android) mTestSuiteName. On Android it ends up being just: _ Bug: angleproject:7299 Change-Id: I3dabcfcbacc3f43338b6fa91ebd9917e43f783ce Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3780587 Commit-Queue: Roman Lavrov <romanl@google.com> Reviewed-by: Yuxin Hu <yuxinhu@google.com>
Eddie Hatfield 916a35dd 2022-06-28T14:40:53 Merge getTempPath and CreateTemporaryFile Also includes moving these two (and some helpers) to common/system_utils.h Bug: angleproject:5095 Change-Id: I7d2fcbd27a67dc13215265ac5b5e8391b15a980b Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3733519 Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org> Commit-Queue: Eddie Hatfield <eddiehatfield@google.com> Reviewed-by: Cody Northrop <cnorthrop@google.com>
Eddie Hatfield 91976352 2022-06-21T15:41:02 Use C++17 attributes instead of custom macros Bug: angleproject:6747 Change-Id: Iad6c7cd8a18d028e01da49b647c5d01af11e0522 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3718999 Reviewed-by: Cody Northrop <cnorthrop@google.com> Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org> Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org>
Jamie Madill ca96cba9 2022-06-16T11:33:14 Make ReadEntireFileToString return a std::string. Note: this makes the method unavailable for export from angle_util, which is probably why it was designed the way it was in the first place. However, we import the test utils source file as a static lib into each test executable and test shared module, so it works. Bug: angleproject:7404 Change-Id: Ia957268882c2b8529643660d7d4f34d142c0dc43 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3708602 Commit-Queue: Roman Lavrov <romanl@google.com> Reviewed-by: Roman Lavrov <romanl@google.com>
Yuly Novikov 5f1d0742 2022-06-17T13:11:00 Increase batch timeout with sanitizer MemoryBarrierBufferOnlyTest got slow recently Bug: angleproject:7439 Change-Id: I6f24fa9abb9dcc98ebb8c964860a292e89ec1e19 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3711745 Reviewed-by: Jamie Madill <jmadill@chromium.org> Commit-Queue: Jamie Madill <jmadill@chromium.org> Auto-Submit: Yuly Novikov <ynovikov@chromium.org>
Jamie Madill 81585fe2 2022-06-08T14:37:38 Use consistent test names with dEQP. This removes the GTest/dEQP test name distinction. Using the "RegisterTests" method in GTest we can use the full dEQP test names. The one limitation is that "-" is reserved in GoogleTest for exclusion filters, so we need to do some extra work to sanitize our filters before we pass them to GoogleTest. Makes maintaining test expectations easier going forward. Bug: angleproject:7390 Change-Id: I23893c425ddc934c11fe539b9173ec06c58abcc2 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3695518 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Amirali Abdolrashidi <abdolrashidi@google.com>
Roman Lavrov b9bdc53b 2022-06-07T14:50:46 Make TestEventListener and inner class of TestSuite. Avoids having to pass dependencies to the class as it's automatically a friend; external linkage sadly but such is life. Bug: angleproject:7299 Change-Id: I5726dcab5a1a98b93c0713073cd02d6eb3a8e216 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3693826 Commit-Queue: Roman Lavrov <romanl@google.com> Reviewed-by: Jamie Madill <jmadill@chromium.org>
Rohit Rao 4b911686 2022-06-06T22:25:37 Converts from sprintf() to snprintf(). sprintf() is deprecated in Xcode 14, so update to safer equivalents in order to keep the compiler happy on iOS and macOS. Bug: chromium:1331345 Change-Id: Id5348088bf69cbd360d9251e6323596cb710666d Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3690747 Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org> Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Jamie Madill <jmadill@chromium.org>
Peter Kasting 54a6b76f 2022-05-19T07:36:54 Roll third_party/rapidjson/src/ 7484e06c5..781a4e667 (156 commits) Bug: chromium:1284275 Change-Id: I5351f6b38ad56da0a5ccae82a1b340dfe2a3c588 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3646003 Auto-Submit: Peter Kasting <pkasting@chromium.org> Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Jamie Madill <jmadill@chromium.org> Commit-Queue: Peter Kasting <pkasting@chromium.org>
Jamie Madill 580f90c7 2022-05-12T09:53:17 Test Runner: Accept Chromium bot mode flag. This is for compatiblity with Chrome's windowed test launcher. Bug: angleproject:5417 Change-Id: I3a366b2b8aa895ab996fbcab68a777514038a206 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3644854 Commit-Queue: Yuly Novikov <ynovikov@chromium.org> Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Yuly Novikov <ynovikov@chromium.org> Auto-Submit: Jamie Madill <jmadill@chromium.org>
Roman Lavrov 6930184e 2022-04-27T10:34:28 Log runTests so that it shows up in logcat dumps. 04-27 14:41:32.303 11507 11538 I NativeTest: runTests: --list-tests 04-27 14:41:32.321 11507 11538 I NativeTest: runTests finished ... 04-27 14:41:34.742 11585 11615 I NativeTest: runTests: --gtest_filter=TracePerfTest.Run/native_trex_200 --verbose --calibration-time 2 --calibration --warmup-loops 2 --isolated-script-test-output=/sdcard/Download/temp_file-cbce720a15e62e25 04-27 14:41:56.628 11585 11615 I NativeTest: runTests finished Bug: angleproject:7242 Change-Id: I5c03501982d6b268c0dd83d71ce82080fb3c551a Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3611063 Commit-Queue: Roman Lavrov <romanl@google.com> Reviewed-by: Jamie Madill <jmadill@chromium.org> Auto-Submit: Roman Lavrov <romanl@google.com> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Yuly Novikov 7a9856f6 2022-04-05T19:25:26 Report durations of all flaky retries in JSON test output Bug: angleproject:7184 Change-Id: I19295daed7b13919a2b528a43ebe8d9cbac9d0a1 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3572713 Reviewed-by: Jamie Madill <jmadill@chromium.org> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Shahbaz Youssefi 7a85d114 2022-03-25T15:01:17 Use [[nodiscard]] on RAII classes Scoped* classes provide an RAII way of adding cleanup/restore state/etc in a robust way. Unfortunatley, it's very easy to mistakenly leave the variable name, leading to the destructor being called immediately instead of at the end of the scope: { ScopedX(parameters); // instead of ScopedX x(parameters); // Code here is run after destructor } The [[nodiscard]] attribute, if specified on the ScopedX class would lead to a warning (turned to error with -Werror). This change does that for classes named *Scoped* in ANGLE. Bug: chromium:1103817 Change-Id: I65c9922c9b4eba1f9c033e093fe8fe534648ab62 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3552092 Reviewed-by: Lingfeng Yang <lfy@google.com> Reviewed-by: Jamie Madill <jmadill@chromium.org> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Jamie Madill b2c01ac4 2021-09-28T12:57:45 Report process memory in perf tests. Bug: angleproject:6440 Change-Id: Ifdd4389266e3ac55963a3069d437c631bf73f72a Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3191194 Reviewed-by: Charlie Lao <cclao@google.com> Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org> Reviewed-by: Yuxin Hu <yuxinhu@google.com> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Jamie Madill 3739a195 2022-03-09T13:56:36 perf tests: Record perf counter metrics. This adds a new command line argument that will allow the user to specify perf counters to record into the test output. Bug: angleproject:4918 Change-Id: Ia7432ff96eadf13ef681f67d2d503d00fd83e06e Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3516970 Reviewed-by: Lingfeng Yang <lfy@google.com> Reviewed-by: Charlie Lao <cclao@google.com> Reviewed-by: Yuxin Hu <yuxinhu@google.com> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Jamie Madill 9b7a77b0 2022-02-11T11:34:19 Double batch timeout for sanitizer tests. It seems the batch was timing out in ASAN builds. Bug: angleproject:6965 Change-Id: I059e138e7fc43928fb25f905bf85fbb34becd500 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3456662 Reviewed-by: Yuly Novikov <ynovikov@chromium.org> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Jamie Madill ef3fffd6 2022-01-19T16:00:42 Fix running tests with empty test set. This was causing a crash when using an overly strict filter. Bug: angleproject:5417 Change-Id: I38bae1a155a6b1047bae0856348de67f9cda21eb Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3402939 Reviewed-by: Yuly Novikov <ynovikov@chromium.org> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Yuly Novikov f071aaaf 2021-07-22T21:21:35 Use first test name instead of shard index for TestArtifacts Shard index is not available on Android if run_gtest_angle_test.py is not used, since host doesn't pass it to device by default. We'd like to remove run_gtest_angle_test.py usage, since custom scripts are not supported on iOS, so this CL is a pre-requisite to that. We still need to distinguish between dEQP test artifacts when we merge test results of different shards, so we'll be using the name of the first test in a batch instead. However, now, on Android, instead of a single TestArtifactsFakeTest entry per shard, we will have multiple entries, an entry for each GTest batch of 256 tests. On other platforms we will still have one entry per shard, since batching is done by TestSuite and not GTests there, just with a different name: Old: "tests": { "TestArtifactsFakeTest-Shard00": { "actual": "PASS", "artifacts": { "TestResults-Batch001.qpa": [ "TestResults-Batch001.qpa" ], New: "tests": { "TestArtifactsFakeTest-GLES3/info_renderer": { "actual": "PASS", "artifacts": { "TestResults-Batch001.qpa": [ "TestResults-Batch001.qpa" ], Bug: angleproject:5417 Change-Id: I649036c151e078c862e380ccab347d6c59acc929 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3046764 Reviewed-by: Jamie Madill <jmadill@chromium.org> Reviewed-by: Yuly Novikov <ynovikov@chromium.org> Commit-Queue: Yuly Novikov <ynovikov@chromium.org>
Jamie Madill fdadc420 2021-09-17T14:39:36 Fix non-bot mode sharding on Android and disable elsewhere. There is no need to apply sharding internally on Android, since it is done on the host, except when listing tests. And on other platforms sharding in non-bot mode is not useful. Patch authored by Yuly Novikov (ynovikov@chromium.org) Bug: angleproject:5417 Change-Id: Iddc0e1a38fb514617dce527acc99d0dce85ca177 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3399250 Reviewed-by: Yuly Novikov <ynovikov@chromium.org> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Lubosz Sarnecki 3e493db2 2021-11-19T10:47:04 tests/AndroidManifest: Disable allowNativeHeapPointerTagging. In order for the mprotect tests to pass on Android, tagged pointers need to be disabled. Bug: angleproject:5857 Change-Id: I2a9703b98443a96b486581c5c5d57e40c3365555 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3291655 Commit-Queue: Lubosz Sarnecki <lubosz.sarnecki@collabora.com> Reviewed-by: Jamie Madill <jmadill@chromium.org> Reviewed-by: Cody Northrop <cnorthrop@google.com>
Jamie Madill 14f55a7f 2021-11-02T10:16:58 infra: Enable TSAN and UBSAN tests. These tests are not yet enabled on the CQ. Bug: angleproject:5795 Change-Id: I3fc58ab800ebbc0840da8221b03b4ab145d28028 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3258003 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org> Reviewed-by: Yuly Novikov <ynovikov@chromium.org>
Lingfeng Yang e9f9fa17 2021-11-09T18:13:15 Add cpu time measurement to perf tests Bug: angleproject:6667 Change-Id: I16570a7b51d363dd1c0f35789b985520cb76d488 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3270601 Commit-Queue: Lingfeng Yang <lfy@google.com> Reviewed-by: Tim Van Patten <timvp@google.com> Reviewed-by: Jamie Madill <jmadill@chromium.org> Reviewed-by: Charlie Lao <cclao@google.com>
Jamie Madill 3ce1abd8 2021-11-16T14:38:15 Test Runner: Print output snippet in test timeout. This will be helpful in diagnosing test failures. Bug: angleproject:6694 Change-Id: I0ea491fe8ab48dfd017ce40150f606c571f3fa6c Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3285805 Reviewed-by: Yuly Novikov <ynovikov@chromium.org> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Jonah Ryan-Davis f422f21a 2021-11-09T18:42:15 Reland "Metal: Reintroduce GPU power preference selection code." This reverts commit 67a8cf07a740c5ce3aafd2ad7fddd370451b3525. Reason for revert: Landed Chromium-side dependency: https://chromium-review.googlesource.com/c/chromium/src/+/3271170 Original change's description: > Revert "Metal: Reintroduce GPU power preference selection code." > > This reverts commit 017161701b7dbf70a13f3c180a39e1fa45c27d9f. > > Reason for revert: Blocking roller, please re-land with the fix. > > Original change's description: > > Metal: Reintroduce GPU power preference selection code. > > > > This CL re-introduces the GPU power preference code to > > the metal backend. It also reworks EGLDisplay caching > > in the frontend to cache based on the native display > > as well as the power preference attribute. > > A new extension, EGL_ANGLE_display_power_preference is > > added based on EGL_ANGLE_power_preference. This extension > > is a client extension that allows selection of GPU on > > display creation, similar to how GPUs are selected on > > context creation in EGL_ANGLE_power_preference. > > This CL adds EGLDisplayPowerPreferenceTest and enables it on > > the metal backend. > > > > Bug: angleproject:6143 > > Change-Id: I0a081dcd2e3f18ab365fdd3498ddcb6e2ba35212 > > Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3231986 > > Reviewed-by: Kenneth Russell <kbr@chromium.org> > > Reviewed-by: Jamie Madill <jmadill@chromium.org> > > Reviewed-by: Gregg Tavares <gman@chromium.org> > > Commit-Queue: Jonah Ryan-Davis <jonahr@google.com> > > TBR=kbr@chromium.org,gman@chromium.org,jonahr@google.com,jmadill@chromium.org,angle-scoped@luci-project-accounts.iam.gserviceaccount.com > > Change-Id: I4f775bf7139253a87b033a30e0da2100b3c1bb02 > No-Presubmit: true > No-Tree-Checks: true > No-Try: true > Bug: angleproject:6143 > Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3270749 > Reviewed-by: Jamie Madill <jmadill@chromium.org> > Commit-Queue: Jamie Madill <jmadill@chromium.org> # Not skipping CQ checks because this is a reland. Bug: angleproject:6143 Change-Id: Id9b0a5cbb76e4dea9e2f2da2b1c47a0587dfdaf5 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3270970 Reviewed-by: Jonah Ryan-Davis <jonahr@google.com> Commit-Queue: Jonah Ryan-Davis <jonahr@google.com>
Jamie Madill 67a8cf07 2021-11-09T15:10:50 Revert "Metal: Reintroduce GPU power preference selection code." This reverts commit 017161701b7dbf70a13f3c180a39e1fa45c27d9f. Reason for revert: Blocking roller, please re-land with the fix. Original change's description: > Metal: Reintroduce GPU power preference selection code. > > This CL re-introduces the GPU power preference code to > the metal backend. It also reworks EGLDisplay caching > in the frontend to cache based on the native display > as well as the power preference attribute. > A new extension, EGL_ANGLE_display_power_preference is > added based on EGL_ANGLE_power_preference. This extension > is a client extension that allows selection of GPU on > display creation, similar to how GPUs are selected on > context creation in EGL_ANGLE_power_preference. > This CL adds EGLDisplayPowerPreferenceTest and enables it on > the metal backend. > > Bug: angleproject:6143 > Change-Id: I0a081dcd2e3f18ab365fdd3498ddcb6e2ba35212 > Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3231986 > Reviewed-by: Kenneth Russell <kbr@chromium.org> > Reviewed-by: Jamie Madill <jmadill@chromium.org> > Reviewed-by: Gregg Tavares <gman@chromium.org> > Commit-Queue: Jonah Ryan-Davis <jonahr@google.com> TBR=kbr@chromium.org,gman@chromium.org,jonahr@google.com,jmadill@chromium.org,angle-scoped@luci-project-accounts.iam.gserviceaccount.com Change-Id: I4f775bf7139253a87b033a30e0da2100b3c1bb02 No-Presubmit: true No-Tree-Checks: true No-Try: true Bug: angleproject:6143 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3270749 Reviewed-by: Jamie Madill <jmadill@chromium.org> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Jonah Ryan-Davis 01716170 2021-10-14T13:24:41 Metal: Reintroduce GPU power preference selection code. This CL re-introduces the GPU power preference code to the metal backend. It also reworks EGLDisplay caching in the frontend to cache based on the native display as well as the power preference attribute. A new extension, EGL_ANGLE_display_power_preference is added based on EGL_ANGLE_power_preference. This extension is a client extension that allows selection of GPU on display creation, similar to how GPUs are selected on context creation in EGL_ANGLE_power_preference. This CL adds EGLDisplayPowerPreferenceTest and enables it on the metal backend. Bug: angleproject:6143 Change-Id: I0a081dcd2e3f18ab365fdd3498ddcb6e2ba35212 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3231986 Reviewed-by: Kenneth Russell <kbr@chromium.org> Reviewed-by: Jamie Madill <jmadill@chromium.org> Reviewed-by: Gregg Tavares <gman@chromium.org> Commit-Queue: Jonah Ryan-Davis <jonahr@google.com>
Jamie Madill 666fcf38 2021-10-08T12:19:15 Rename functions that overlap with Windows APIs. Bug: angleproject:6283 Change-Id: Ifcd9ea9e3bf729fd2066178eb9429050b2f10518 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3212894 Reviewed-by: Bruce Dawson <brucedawson@chromium.org> Reviewed-by: Jamie Madill <jmadill@chromium.org> Commit-Queue: Jamie Madill <jmadill@chromium.org>
angle-autoroll 66c89b0f 2021-10-06T16:28:42 Fix and disable MSVC warnings Needed because some warnings are no longer disabled after http://crrev.com/c/3189512. Also includes https://github.com/KhronosGroup/OpenCL-Headers/pull/179, needed after clang upgrade to llvmorg-14-init-5410-gd0473681 Bug: chromium:1257173 Change-Id: I4f844aa972362c488cb6d37244439e2126f2c1c3 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3210629 Reviewed-by: Yuly Novikov <ynovikov@chromium.org> Commit-Queue: Yuly Novikov <ynovikov@chromium.org>
Jamie Madill 0004ecc2 2021-09-22T13:39:55 Write GL info to test artifact in end2end_tests. These will show up as glinfo.json in the "cas outputs" of the shard that runs these tests. We currently have information for Pixel 4, NVIDIA and Intel Win/Linux, and SwiftShader Windows. GLES 1.x extensions are treated separately. Some WebGL/Mac-only/D3D-only extensions show as unsupported anywhere. In order to capture artifacts on Android, we use a render test output parameter. Bug: angleproject:6379 Change-Id: I560b219b5a586cdbb68d272090f38eedc76aba20 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3176780 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Ian Elliott <ianelliott@google.com> Reviewed-by: Tim Van Patten <timvp@google.com> Reviewed-by: Jonah Ryan-Davis <jonahr@google.com>
Jamie Madill 247ee1c3 2021-09-10T09:38:25 Perf Tests: Output results in "msBestFitFormat" again. The histogram bins for "ms" would cause excessive rounding for tests that run very quickly. For example, 0.0012354 ms would be rounded to 0.001 ms. This would produce very flat graphs that don't accurately represent the real results. Bug: angleproject:6090 Change-Id: I9293b2083d7e54dc2663648c361f4883278806ec Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3152746 Reviewed-by: Yuly Novikov <ynovikov@chromium.org> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Jamie Madill e1bc8de3 2021-09-08T07:59:37 Change links from 'master' to 'main' branch. Bug: chromium:1226949 Change-Id: Ie7b28b2fa094cf0c0a407968d681e580519cbb88 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3148210 Reviewed-by: Geoff Lang <geofflang@chromium.org> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Zequan Wu 178b8481 2021-09-03T23:30:52 Revert "Skip validation of stderr in TestSuiteTest.RunFlakyTests" This reverts commit 89dbbb2ac6875c92c99feb24353641142bbd49b1. Reason for revert: The test failure was gone in latest clang roll. Original change's description: > Skip validation of stderr in TestSuiteTest.RunFlakyTests > > There is a bug (crbug.com/1234124) causing the profile runtime to print > errors about failing to write the profraw file. Don't validate stderr in > these builds until that is fixed. > > Bug: chromium:1234124 > Change-Id: Ie1e3a3eaf7dc0e2c20064822f774cc85fe79d3bc > Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3076479 > Reviewed-by: Jamie Madill <jmadill@chromium.org> > Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org> > Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org> Bug: chromium:1234124 Change-Id: If33e92b533c99677d2b8f55de618ecf15e3f19d2 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3141973 Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org> Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org>
Jamie Madill 5e1bd524 2021-08-26T10:28:57 Perf Tests: Fix histogram units. There was another place where we needed to set "smallerIsBetter". Also prefer "ms" instead of "msBestFitFormat". Bug: angleproject:6090 Change-Id: I489089ae03f46b4c427821019917294d2d01dbcd Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3123225 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Yuly Novikov <ynovikov@chromium.org>
Jamie Madill f65d6acc 2021-08-18T12:20:40 Test Runner: Determine slow tests from test expectations. This updates the test harness to handle slow test based on the TIMEOUT test expectation. It removes the "registerSlowTests" API in favor of using the test expectation files. Also updates some of the timeout handling for various angle tests, including on SwiftShader for multithreading tests. Increases the timeout for tests in a batch to attempt to fix the problem of the first test being much slower. Bug: angleproject:6261 Change-Id: I7427344da5a0c1ea26d8c2d6eb0e9be2557b56e6 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3104007 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Geoff Lang <geofflang@chromium.org> Reviewed-by: Yuly Novikov <ynovikov@chromium.org>
Jamie Madill d25c9d7f 2021-07-07T12:15:12 Add perf tests runner script. This script will be responsible for running the perf tests multiple times to try and stabilize measurements. We'll use it on the bots instead of just running the perf tests directly. Because the script invokes the binary multiple times, this slows down execution. Most significantly on Android, where we now need to use 20 shards, up from 6. Also marks one test as flaky on OpenGL. Bug: angleproject:6090 Change-Id: I5280035cb0bdb290a68dc6961a384eaf4b40dd4b Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3011422 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Cody Northrop <cnorthrop@google.com> Reviewed-by: Tim Van Patten <timvp@google.com>
Reid Kleckner 89dbbb2a 2021-08-05T13:14:12 Skip validation of stderr in TestSuiteTest.RunFlakyTests There is a bug (crbug.com/1234124) causing the profile runtime to print errors about failing to write the profraw file. Don't validate stderr in these builds until that is fixed. Bug: chromium:1234124 Change-Id: Ie1e3a3eaf7dc0e2c20064822f774cc85fe79d3bc Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3076479 Reviewed-by: Jamie Madill <jmadill@chromium.org> Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org> Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org>
Jamie Madill 26329cb9 2021-07-30T09:55:24 Test Runner: Allow listing tests on Android. Because of quirks with the Chromium-provided Android test runner, we need to use a few tricks to get the test list output. We add placeholder output for a single test to trick the test runner into thinking it ran the tests successfully. We also add an end marker for the tests list so we can parse the list from the more spammy Android stdout log. Will enable a Python wrapper script to list and run performance tests outside of the GTest harness. Bug: angleproject:6090 Change-Id: I810d4722e9a6efa03ef94208e516401890af1c37 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3062240 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Cody Northrop <cnorthrop@google.com> Reviewed-by: Tim Van Patten <timvp@google.com>
Shahbaz Youssefi 22ae4ce0 2021-07-28T16:06:49 Revert "Disable RunAppAsync and RunAppAsyncRedirectStderrToStdout" This reverts commit f4f866ce289423d691a0edc0d44d13f4de16f897. Reason for revert: These tests should not be disabled. The culprit is fixed per crbug.com/1233361 Original change's description: > Disable RunAppAsync and RunAppAsyncRedirectStderrToStdout > > The tests are blocking the CQ due to flakiness. > > Bug: chromium:1233361 > Change-Id: Ib0283e201c271379364a177bea130d63322eee53 > Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3058233 > Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org> > Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org> Bug: chromium:1233361 Change-Id: I24682f74d790d40f58a9526aaaaa842c3df23ec0 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3058502 Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com> Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org> Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org>
Kenneth Russell e74638fe 2021-07-28T19:03:43 Disable Metal shader cache again in angle_end2end_tests. This hooking was removed in https://chromium-review.googlesource.com/3010486 and sped up local test runs, but seems to have reintroduced flakiness on the Mac AMD Retina bots. Bug: chromium:1233864 Bug: angleproject:5505 Change-Id: I46d95545fbbedc0e881e3a2832a5ac5d5384f405 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3057823 Commit-Queue: ccameron <ccameron@chromium.org> Reviewed-by: ccameron <ccameron@chromium.org>
Shahbaz Youssefi 8593e0a2 2021-07-28T13:52:06 TestSuite tests: Don't validate stderr until fixed Bug: chromium:1233361 Change-Id: Ibcd66520d7b842211eafb26f9188298c281cbe64 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3058503 Reviewed-by: Tim Van Patten <timvp@google.com> Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org>
Jamie Madill 03cf7888 2021-07-22T08:32:25 Test Runner: Remove extra quotes from histograms. We were double-quoting the test names. Bug: angleproject:6090 Change-Id: I69ce089ef20068b408de7156bdbade922cdefe95 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3043890 Reviewed-by: Jamie Madill <jmadill@chromium.org> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Kenneth Russell b3cbfd55 2021-07-07T12:31:57 Fix multiple end2end crashes in direct-to-MSL compiler. This set of changes: - Adds a single place (CompilerMtl::useDirectToMSLCompiler) where the direct-to-MSL compiler can be enabled, still at compile time rather than run time. It is still disabled by default. - Initializes MTLWaitableCompileEventImpl::mShader, without which all shader compiles crash. - Fixes a mismatch of the ANGLERasterizerDisabled variable name in the generated MSL. - Removes the transform feedback bindings as an argument to the vertex shader's main0(). Transform feedback support needs to be reimplemented in this compiler backend. - Added an option to the DriverUniform class to emit as either an interface block or a struct. The direct-to-MSL backend assumes it is a struct. - Disable variable reference validation when referencing either sample mask or rasterizer discard functionality in the shader. - Disable struct usage validation when inserting references to ANGLE_TextureEnv. This occurs during pipeline rewriting and it's infeasible to disable it at that point, so disable it during pipeline rewriting - which means it's disabled for all shaders. - For angle_end2end_tests, disable the file API hooking which disables the Metal shader cache. This speeds up the tests significantly - by roughly a factor of 5. With these changes, several hundred angle_end2end_tests run without any assertion failures, though still with many test failures. The next crash is in: GLSLTest.NamelessScopedStructs/ES2_Metal and will likely require a larger bug fix, or more changes to be upstreamed from WebKit's repository. Bug: angleproject:5505 Change-Id: Ia1ea9a13867f00035d4aeccf907fd032255588e3 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3010486 Commit-Queue: Kenneth Russell <kbr@chromium.org> Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org> Reviewed-by: Jonah Ryan-Davis <jonahr@google.com>
Torne (Richard Coles) d84d1288 2021-07-16T12:08:03 android: Explicitly specify exported activities. Android 12 requires all components with an <intent-filter> to explicitly specify whether they are exported, instead of being exported by default. Update various remaining cases in our tests/sample code/tools. Bug: chromium:1223110 Change-Id: Iab37881d947da25d771bf757d2f7581ad782c73f Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3035442 Reviewed-by: Geoff Lang <geofflang@chromium.org> Commit-Queue: Geoff Lang <geofflang@chromium.org>
Pujun Lun 2a713d92 2021-07-15T14:58:54 Avoid recreating activity when rotating the display. Reference: https://developer.android.com/guide/topics/resources/runtime-changes#HandlingTheChange Bug: b/193828422 Change-Id: I4caa80941e4d32bdf5c830985a19536efd041069 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3032150 Reviewed-by: Cody Northrop <cnorthrop@google.com> Reviewed-by: Charlie Lao <cclao@google.com> Reviewed-by: Yuly Novikov <ynovikov@chromium.org> Commit-Queue: Cody Northrop <cnorthrop@google.com>
Jamie Madill 15623ada 2021-06-23T08:46:21 Test Runner: Omit results files when too many failures. This will cause the test shards to be re-tried due to an invalid result. The goal is to retry flaky configs where whole shards fail. Bug: angleproject:6075 Change-Id: I76d0f08da29ffc4bcad7f282f7be38618c384bce Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2982500 Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org> Reviewed-by: Jonah Ryan-Davis <jonahr@google.com> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Shahbaz Youssefi 09bd5578 2021-06-21T23:47:15 Test runner: Capture test stderr With this change, stderr is also output when a test fails, including logs such as those originating from UNIMPLEMENTED(). Bug: angleproject:6077 Change-Id: Ice1e1f9aff26acfb630aef99865503643f0a9658 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2976184 Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org> Reviewed-by: Jamie Madill <jmadill@chromium.org>
Shahbaz Youssefi 72d2bd0c 2021-06-21T09:59:21 Allow capturing process stdout and stderr interleaved The test utils are enhanced to allow redirecting stderr to stdout. This is in preparation for a change that makes the test runner capture stderr together with stdout. Currently, on failure logs originating from UNIMPLEMENTED() and other such macros are not captured. Bug: angleproject:6077 Change-Id: I7a3c6c4732a59dac3ff0cc20a7835d5ed6f0f22e Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2976183 Reviewed-by: Jamie Madill <jmadill@chromium.org> Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org>
Jamie Madill 4194d400 2021-06-21T13:08:56 Test Runner: Add maximum failure count. This will make the test runner early exit after a certain count of failures. This will prevent the bots from running tens of thousands of tests when the whole test run is corrupt. Bug: angleproject:6075 Change-Id: I8f5c19ea694a5ad3a5d5df1429d2ba4b1fe868e3 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2976179 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org> Reviewed-by: Jonah Ryan-Davis <jonahr@google.com>
Shahbaz Youssefi c51c59c7 2021-06-15T13:43:50 Test for missing index dirty bit bug Bug fixed in https://chromium-review.googlesource.com/c/angle/angle/+/2961690 triggers only in the following situation: - Context 1: draw indexed -> clears index dirty bit - Context 1: change state in such a way that closing the render pass is deferred to dirty bit handling (for example, change FBO) - Context 1: issue a non-indexed draw call. This closes the render pass and starts a new one -> bug was that the index dirty bit was not set - Context 2: flush the command buffer, which submits the previous render pass of context 1 (which contained vkCmdBindIndexBuffer). The primary command buffer is now reset. - Context 1: issue an indexed draw call. Since the index dirty bit was not set, this was missing the vkCmdBindIndexBuffer call. This change implements a regression test based on the above scenario. Bug: chromium:1183068 Bug: chromium:1190493 Change-Id: I729bd48cd6df2621ca763f6231023a52ac08b0fb Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2963836 Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org> Reviewed-by: Charlie Lao <cclao@google.com> Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org>
Jamie Madill bc9d5223 2021-06-02T18:19:08 Experimental fix for stdout freopen race. Bug: chromium:1213184 Change-Id: I2e2741f4100b2a5516210f2250dc82511fafd05b Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2933602 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Yuly Novikov <ynovikov@chromium.org>
Jamie Madill dd686e48 2021-05-11T19:08:01 Test Runner: Add test expectations parser. Moves the test expectations from dEQP into the test runner. Also updates angle_end2end_tests to take an expectations file. Includes some very simple angle_end2end_tests expectations. Note that the expectations in the file are less expressive than the skips we use in the cpp. Bug: angleproject:5951 Change-Id: Ib92235575bc3ea5f3a977ce416b0e78fe806e39b Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2892274 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Geoff Lang <geofflang@chromium.org> Reviewed-by: Jonah Ryan-Davis <jonahr@google.com>
Jamie Madill 1d019f49 2021-05-13T10:33:06 Test Runner: Fixes to skipped tests. Uses a different token for skipped and not run tests in the json. The token isn't standard but should only be visible when batches report to the parent process that a test wasn't run. It shouldn't be visible to the scripts that consume test results on the bots. Also fixes the expected result for skipped tests to also be "SKIP". Bug: angleproject:5951 Change-Id: I0bd7d13b2b25b583f9c42718786cb588bacd4bc6 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2894524 Reviewed-by: Yuly Novikov <ynovikov@chromium.org> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Jamie Madill 9a219a14 2021-05-12T09:52:16 Test Runner: Handle skipped gtests. GTEST_SKIP() allows us to skip tests using the GoogleTest APIs. This CL updates our test runner code to handle these skips. Bug: angleproject:5951 Change-Id: Idef5802c7c81bd92226f5a93281a1cea0f1b24ee Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2892272 Reviewed-by: Geoff Lang <geofflang@chromium.org> Reviewed-by: Jonah Ryan-Davis <jonahr@google.com> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Cody Northrop 74177f31 2021-05-09T21:15:09 Tests: Run Android tests fullscreen In order to hide the status bar and hopefully prevent any layers from rendering on top of our traces, make all Android tests run fullscreen. Test: angle_perftests Bug: angleproject:5937 Change-Id: I649104a7d95b7f056b411e180632524a85b7c4f2 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2880335 Reviewed-by: Jamie Madill <jmadill@chromium.org> Reviewed-by: Tim Van Patten <timvp@google.com> Commit-Queue: Cody Northrop <cnorthrop@google.com>
Peter Kotwicz bf0c6ae1 2021-03-09T13:21:58 Improve Angle Android gtest support This CL makes angle_test() use an angle-specific gtest launcher - AngleUnitTestActivity.java. This enables building standalone Android angle without //base Chromium-Bug: 1149922 Bug: None Change-Id: Id84f3e2bd84c5017ed1988ca07534f424ebfa596 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2745535 Reviewed-by: Jamie Madill <jmadill@chromium.org> Reviewed-by: Geoff Lang <geofflang@chromium.org> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Jamie Madill e5c5efc0 2021-02-05T02:39:55 Revert "Test Runner: Pick single/multi-process mode automatically." This reverts commit 573d7f34a462fa5b2ca767dd84773ae6fa460d8a. Reason for revert: Breaking ANGLE roll. No-try: true Original change's description: > Test Runner: Pick single/multi-process mode automatically. > > This enables multi-process when running with more than one test > config. This saves developers from remembering to add the arg. > It can also simplify our test configurations by allowing us to omit > the default arguments. On Android by default we stay with single > process. It also changes the --bot-mode flag into --single-process > and --multi-process which override the default behaviour. > > Bug: angleproject:5585 > Change-Id: I3dd885a8a8e6604c224693c546ff5d15224e016e > Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2664246 > Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org> > Reviewed-by: Jonah Ryan-Davis <jonahr@google.com> > Reviewed-by: Kenneth Russell <kbr@chromium.org> > Commit-Queue: Jamie Madill <jmadill@chromium.org> TBR=kbr@chromium.org,syoussefi@chromium.org,jonahr@google.com,jmadill@chromium.org,jonahr@chromium.org # Not skipping CQ checks because original CL landed > 1 day ago. Bug: angleproject:5585 Change-Id: I11ee1034c301aa246c4b03e29ff9b56235a1005c Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2676247 Reviewed-by: Jamie Madill <jmadill@chromium.org>
Jamie Madill 573d7f34 2021-02-01T10:36:56 Test Runner: Pick single/multi-process mode automatically. This enables multi-process when running with more than one test config. This saves developers from remembering to add the arg. It can also simplify our test configurations by allowing us to omit the default arguments. On Android by default we stay with single process. It also changes the --bot-mode flag into --single-process and --multi-process which override the default behaviour. Bug: angleproject:5585 Change-Id: I3dd885a8a8e6604c224693c546ff5d15224e016e Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2664246 Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org> Reviewed-by: Jonah Ryan-Davis <jonahr@google.com> Reviewed-by: Kenneth Russell <kbr@chromium.org> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Jamie Madill 85efb9d5 2021-01-26T14:39:19 Log dEQP QPA files as test artifacts. This adds artifact output to the test runner. We add a fake test at the start of a test run that owns the artifacts. Bug: angleproject:5236 Change-Id: Ice8001bf1f2aafbd8123fee76e0e7fcc3e5a8a0c Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2657535 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org> Reviewed-by: Mohan Maiya <m.maiya@samsung.com> Reviewed-by: Yuly Novikov <ynovikov@chromium.org>
Jamie Madill feb470cc 2021-01-19T10:10:18 Test Runner: Prevent race by not resetting timeout. This instead only sets the timeout when tests start to run. That way the timeout should always be at least the default timeout and we won't get into a situation where the watchdog times out the test immediately when the test starts. Bug: angleproject:5562 Change-Id: I6b12bb8fe8edcf35f46ba4fb106fdf80ff9402a1 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2637182 Reviewed-by: Yuly Novikov <ynovikov@chromium.org> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Jamie Madill e4b4ff7a 2021-01-12T13:48:20 Test Runner: Add a slow tests list. Each slow test in the list will use a 3x longer timeout. The list is implemented using the same filter wildcard we use in the test expectations logic. We can test this out using a slow D3D11 varying test. Bug: angleproject:5076 Bug: angleproject:5496 Change-Id: I31cf45e6ee8a8bbd6e460d675ff8a0cf5f19a504 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2625172 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org> Reviewed-by: Yuly Novikov <ynovikov@chromium.org>
Le Hoang Quyen cb8903b1 2020-12-08T01:08:00 Metal: Ignore OS's internal shader cache when testing. Internal shader cache caused timeout in some dEQP tests. Work-around: ignore the cache by hooking fopen function and return null when the cache related files are accessed. Bug: angleproject:5354 Change-Id: I12ca228540925e67454bf24ce1ba83d703882c87 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2580918 Commit-Queue: Le Hoang Quyen <le.hoang.q@gmail.com> Reviewed-by: Jamie Madill <jmadill@chromium.org> Reviewed-by: Kenneth Russell <kbr@chromium.org>
Tim Van Patten c859c0ac 2020-11-24T17:21:38 Batch replace std::unordered_map with angle::HashMap in src/ There are a few places that will remain std::unordered_map due to build or run-time errors, which will need to be evaluated more closely to determine if they should remain std::unordered_map or if there is another Abseil data structure that would be more efficient while still working correctly. Bug: angleproject:4873 Change-Id: Ib04253e3ad6398e63f4cc2bfe12c0f9e57cb112b Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2558873 Commit-Queue: Tim Van Patten <timvp@google.com> Reviewed-by: Jamie Madill <jmadill@chromium.org> Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org> Reviewed-by: Charlie Lao <cclao@google.com>
Jamie Madill 9fe2d211 2020-12-01T13:24:33 Test Runner: Handle flaky failures. When a test mark as flaky fails consistently we need to avoid marking the test as flaky. We treat is as a deterministic failure. This means we output "PASS" as the expected result instead of "PASS FAIL". That change will produce the correct behaviour with the test scripts. Also adds missing test runner output when we retry a flaky test. Bug: chromium:1152515 Change-Id: I9e3cbaf04ea57a198717ee81505b5d63a9183410 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2568231 Reviewed-by: Yuly Novikov <ynovikov@chromium.org> Reviewed-by: Jamie Madill <jmadill@chromium.org> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Jamie Madill 3e5b6f81 2020-10-31T12:33:28 Enable writing histogram-set-json-format. This uses the protobuf histogram functionality to build up a histogram set. The test suites then output the histograms to JSON. This is only implemented for angle_perftests. Bug: angleproject:5161 Change-Id: Ia5a7868e8d8dcf4f13d83115ae622828c63ef0d9 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2482295 Reviewed-by: Brian Sheedy <bsheedy@chromium.org> Reviewed-by: Yuly Novikov <ynovikov@chromium.org> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Jamie Madill c06a424f 2020-11-02T21:13:39 Test Runner: Robustly handle unexpected crashes. This adds the ability for the test runner to parse unexpected crashes from the stdout. It also processes the stdout to determine which tests failed. Tests that run after the crash are re-tried in a follow-up child process. Will allow for the test runner to handle very crashy test suites and also processes crashes from win-asan and other configs where it is harder to intercept crashes. Bug: angleproject:5251 Change-Id: Iee03130622571580cb7910f4fb097fe3659d75ec Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2513288 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Yuly Novikov <ynovikov@chromium.org> Reviewed-by: Tim Van Patten <timvp@google.com>
Jamie Madill 59fbb989 2020-11-02T13:27:52 Test Runner: Add ability to retry flaky tests. Bug: angleproject:5273 Change-Id: Ie89559bb0897a04213981aa8fe4e2f2bfe78959a Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2513287 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Yuly Novikov <ynovikov@chromium.org> Reviewed-by: Tim Van Patten <timvp@google.com>
Jamie Madill 47e66f22 2020-10-28T14:24:37 Revert "Re-enable crash testing in angle_unittests." This reverts commit 21add1240950c2e2a6ec997f9a128005d84266b6. Reason for revert: Fails on Windows 7 x64 Chromium config. https://ci.chromium.org/p/chromium/builders/ci/Win%207%20Tests%20x64%20%281%29/73194 Original change's description: > Re-enable crash testing in angle_unittests. > > This is an experimental test to see if this works now. > Disabled on Win/ASAN because of conflicts with the > crash callback. > > Bug: angleproject:5239 > Change-Id: I280b18ab8846de79b561f26716df9ba374cfd9a5 > Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2496040 > Reviewed-by: Jamie Madill <jmadill@chromium.org> > Commit-Queue: Jamie Madill <jmadill@chromium.org> TBR=ynovikov@chromium.org,jmadill@chromium.org # Not skipping CQ checks because original CL landed > 1 day ago. Bug: angleproject:5239 Change-Id: I83c150c2c2090b7d1aeb78e53e96115a167a7491 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2505479 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Jamie Madill <jmadill@chromium.org>
Jamie Madill 5c4c37dc 2020-10-25T15:55:45 Capture/Replay: Test improvements. Streamlines some arguments. Use a common subdirectory in out/. Show process name when logging. Clean up environment variable handling. Bug: angleproject:5247 Change-Id: I85eb5b8194a64f90691c4d03db3ee65c5e9b2313 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2497560 Reviewed-by: Cody Northrop <cnorthrop@google.com> Reviewed-by: Courtney Goeltzenleuchter <courtneygo@google.com> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Jamie Madill 21add124 2020-10-25T15:55:45 Re-enable crash testing in angle_unittests. This is an experimental test to see if this works now. Disabled on Win/ASAN because of conflicts with the crash callback. Bug: angleproject:5239 Change-Id: I280b18ab8846de79b561f26716df9ba374cfd9a5 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2496040 Reviewed-by: Jamie Madill <jmadill@chromium.org> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Jamie Madill 4968245c 2020-10-25T12:58:15 Adjust TestSuite test timeout values. This should prevent flakiness in this test. Bug: angleproject:5245 Change-Id: I841de2cee093ea04b35763314f0a4f5497bcf778 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2497223 Reviewed-by: Jamie Madill <jmadill@chromium.org> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Jamie Madill 31868665 2020-10-23T10:54:02 Test Runner: Only run watchdog in child processes. For single test execution we don't need to be running the watchdog thread. This only serves to cause timeouts in slow running tests. Adds a new method to keep the --batch-id argument in the list of parameters so other functions can process them. Bug: angleproject:5124 Bug: angleproject:5218 Change-Id: I958230149b7d79830802cd55f67f0190de607200 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2495282 Reviewed-by: Tim Van Patten <timvp@google.com> Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Jamie Madill cd88bccf 2020-10-21T08:44:32 Test Runner: Command line sharding args override env. Previously ANGLE would error out when both the enviornment and the command line args are specified at the same time. Because the perf bots use both at once we need to handle the conflict in the same way as the prior test runner does. In this case the command line takes precedence over the environment. Bug: angleproject:5124 Change-Id: I1ba765b4e75759922bf9fe2db9f153cfc5995f85 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2489722 Reviewed-by: Yuly Novikov <ynovikov@chromium.org> Reviewed-by: Courtney Goeltzenleuchter <courtneygo@google.com> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Jamie Madill 26cd1cc6 2020-10-18T11:44:27 Perf Tests: Add stdout printing arg to bot mode. This can be useful for debugging and also for perf tests. The perf tests output gets scraped from stdout. Bug: angleproject:5124 Change-Id: Id5005df3f2b66be9497a9ba422abd47b95a1a330 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2486097 Reviewed-by: Tim Van Patten <timvp@google.com> Reviewed-by: Courtney Goeltzenleuchter <courtneygo@google.com> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Jamie Madill 556d812a 2020-10-18T11:48:45 Test Runner: Disable watchdog when debugger attached. Bug: angleproject:3162 Change-Id: I3685a4fe068b6e82b8bb04448c212788f563ab8b Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2483463 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Yuly Novikov <ynovikov@chromium.org>
Jamie Madill b4fb7cc9 2020-10-15T08:58:06 Test Runner: Print reproduction steps on a batch failure. This prints out a large gtest_filter of the tests in a batch. Makes reproducing a particular failure much easier. In the future we could potentially look at using an encoding to reproduce a specific batch: e.g. a flag that acceps a batch / shard number and runs those tests based on using the same internal counts. Bug: angleproject:5172 Change-Id: Iefad8f5ca23761417ad1ceb11fd664c9cf23c8be Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2475455 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Courtney Goeltzenleuchter <courtneygo@google.com> Reviewed-by: Yuly Novikov <ynovikov@chromium.org>
Jamie Madill 9580f3b4 2020-10-10T10:06:04 Test Runner: Accept GTEST shard environment vars. This makes interoperation with Chromium tests easier. For instance when triggering tests with scripts when using sharding. And also using the run/ wrapper scripts instead of run_gtest_angle_test. Bug: angleproject:5158 Change-Id: Ia5b1f94c02a081caf576615175791fff4e0d3fb3 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2464030 Reviewed-by: Courtney Goeltzenleuchter <courtneygo@google.com> Reviewed-by: Yuly Novikov <ynovikov@chromium.org> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Jamie Madill ad80b184 2020-10-13T17:15:59 Test Runner: List tests explicitly without GTest. This dramatically speeds up Android shards that use batches of hudreds of tests. Because ANGLE already has the test list, it no longer needs to call GTest with an enourmous gtest_filter. Bug: angleproject:5164 Change-Id: I28fee2f36c50006f2a35a4dcd90f44f8ebe4f78c Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2468464 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Yuly Novikov <ynovikov@chromium.org>
Jamie Madill 336202d7 2020-10-09T16:43:32 Test Runner: Pass batch ID to child processes. This will let both the ANGLETest and dEQP test runners handle child processes correctly. The existing ANGLETest runner will reuse displays, and the dEQP test runner will write to a batch-unique filename. This solves the problem of the multi-process --bot-mode writing to the same file from multiple children simultaneously. Long-term it will be good to include the dEQP QPA files into the test artifacts. Also disables flushing after every log write when running as a child process. This hugely improves performance on machines with no SSD. Test: https://chromium-swarm.appspot.com/task?id=4f2b87c4f8234910 Bug: angleproject:5157 Change-Id: I226d24adf55e0f8b98e5a8e7947862e6906b4c40 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2464424 Reviewed-by: Courtney Goeltzenleuchter <courtneygo@google.com> Reviewed-by: Ian Elliott <ianelliott@google.com> Commit-Queue: Jamie Madill <jmadill@chromium.org>
kmx.io     mail     discord kmxgit v0.4.0