infra/specs/test_suite_exceptions.pyl


Log

Author Commit Date CI Message
Yuly Novikov b778f1ce 2025-08-07T15:31:01 Balance test shards to 10 minutes Shards increased due to googletest roll: angle_end2end_tests on Win ASAN: 10 -> 14 angle_deqp_gles3_tests on Linux ASAN and TSAN: 8 -> 11 Drive-by shards increase for slowness which was not investigated: angle_end2end_tests: on Linux Intel and Windows and Linux NVIDIA: 2 -> 3 on Win Intel experiment (UHD 770, 2 processes): 3 -> 5 on Win Intel (UHD 630, 1 process): 4 -> 10 angle_restricted_trace_gold_tests on Linux Intel and Windows and Linux NVIDIA: 1 -> 3 on Win Intel UHD 770: 1 -> 4 on Win Intel UHD 630: 1 -> 5 angle_trace_perf_vulkan_tests on Win Intel UHD 770: 2 -> 4 on Win Intel UHD 630: 2 -> 3 Skip EGLDisplayTest.ContextLeakAfterTerminate on Linux Intel, which started failing with the new sharding. Bug: angleproject:428697844, angleproject:437259106 Change-Id: I3ba458aa7b68bddf9786bb9fcbc2a3ed7479556c Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/6827389 Auto-Submit: Yuly Novikov <ynovikov@chromium.org> Reviewed-by: Cody Northrop <cnorthrop@google.com> Commit-Queue: Cody Northrop <cnorthrop@google.com>
Yuly Novikov 9e3467c8 2025-04-04T09:58:05 Run end2end tests single process on Linux Intel Multi-process suspected for flaky WebGPU backend test failures. Bug: angleproject:408276172 Change-Id: I2813d49f1263eeefea7db6999b5452d4cf61db92 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/6432449 Auto-Submit: Yuly Novikov <ynovikov@chromium.org> Reviewed-by: Roman Lavrov <romanl@google.com> Commit-Queue: Roman Lavrov <romanl@google.com> Commit-Queue: Yuly Novikov <ynovikov@chromium.org>
Yuly Novikov 4d613ca7 2024-09-20T14:25:27 Skip angle_deqp_egl_vulkan_tests on linux-exp-nvidia Occasionally hangs the machines. Bug: angleproject:368553850 Change-Id: Ic9d948f3e69c31c3c6f3494dd8a32759df1bb6f4 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/5873135 Reviewed-by: Brian Sheedy <bsheedy@chromium.org> Commit-Queue: Brian Sheedy <bsheedy@chromium.org> Auto-Submit: Yuly Novikov <ynovikov@chromium.org>
Yuly Novikov e413ecaf 2024-07-15T17:44:31 Increase parallelism Win Intel UHD 770 Crashes weren't reproduced with 2 processes, and this improves performance a bit. Bug: angleproject:352528974 Change-Id: I934e4aacdf38a790cad858e79cae683e2a5ed1d3 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/5709048 Auto-Submit: Yuly Novikov <ynovikov@chromium.org> Reviewed-by: Brian Sheedy <bsheedy@chromium.org> Commit-Queue: Brian Sheedy <bsheedy@chromium.org> Commit-Queue: Yuly Novikov <ynovikov@chromium.org>
Yuly Novikov 2997e354 2024-07-09T17:38:56 Start Win Intel UHD 770 experiment - Suppress failing tests. - Disable UHD 630 workarounds on experimental bot - Reduce deqp_gles2_d3d11 and end2end parallelism Use-Permissive-Angle-Pixel-Comparison: True Bug: chromium:331316080, angleproject:352085732, angleproject:352528974 Change-Id: I80eca02f1def11e60b857968a7b0c8fe93a1b4d3 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/5688794 Auto-Submit: Yuly Novikov <ynovikov@chromium.org> Commit-Queue: Yuly Novikov <ynovikov@chromium.org> Reviewed-by: Brian Sheedy <bsheedy@chromium.org>
Shahbaz Youssefi d193d51b 2024-06-17T22:46:08 Replace issue ids post migration to new issue tracker This change replaces anglebug.com/NNNN links. Bug: None Change-Id: I8ac3aec8d2a8a844b3d7b99fc0a6b2be8da31761 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/5637912 Reviewed-by: Geoff Lang <geofflang@chromium.org> Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org>
Roman Lavrov a76432f0 2024-06-07T15:00:18 Android perf tests: increase custom-throttling-temp to 38C (applies to pixel6 perf bots only) Multiple bots running hotter than expected: b/345514719#comment10 Bug: b/345514719 Change-Id: I4e4a20cf2872a73ff7594b8a1edabdb1b4cfd153 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/5607830 Commit-Queue: Roman Lavrov <romanl@google.com> Reviewed-by: Amirali Abdolrashidi <abdolrashidi@google.com> Reviewed-by: Kenneth Russell <kbr@chromium.org>
Roman Lavrov da0b30f0 2024-06-07T10:00:37 Android perf tests: add temp-based throttling to pixel6 bots. Seems like we don't have sufficient airflow to run heavy tests at full throttle. Some shards start >10C hotter, so add custom throttling to avoid aggressive throttling by the phone itself skewing perf results. This might noticeably increase shard runtime on bots with poor airflow. Implementation is similar to https://crrev.com/c/5288455 but relies on data available on current bot OS build (no IThermal) and does _not_ disable phone throttling. Bug: b/345514719 Change-Id: I582838c2448c9b014381a4f8ba847343b68e4d3a Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/5608312 Commit-Queue: Roman Lavrov <romanl@google.com> Reviewed-by: Cody Northrop <cnorthrop@google.com>
Roman Lavrov 0c87d0ed 2024-05-27T14:35:12 Tests: Increase win-intel shards due to --max-processes=1 Currently bottlenecking win-test CI latency Bug: angleproject:7874 Change-Id: I7aceade04175ecc9b89a59363c26060167ed846c Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/5570283 Commit-Queue: Yuly Novikov <ynovikov@chromium.org> Reviewed-by: Yuly Novikov <ynovikov@chromium.org>
Roman Lavrov 1249fc1f 2023-09-07T14:39:26 Tests: Support capture_tests on Android and enable on CI Enables angle_with_capture_by_default = true on Android bots similarly to Linux and Windows. Enables the capture test on Pixel 4 and 6 bots (it takes only a few seconds to run) Enables support for capture in android_helper in general. For example, this now works the same way it would on say Linux, but runs the capture on the device an pulls files to OUT_DIR. % ANGLE_CAPTURE_LABEL=gacha_club ANGLE_CAPTURE_OUT_DIR=/tmp/capt2 ANGLE_CAPTURE_FRAME_START=16 ANGLE_CAPTURE_FRAME_END=16 out/AndroidPerformance/angle_trace_tests --gtest_filter=TraceTest.gacha_club --use-angle=vulkan --retrace-mode --max-steps-performed=20 For end2end tests it's a bit trickier as we're still using the Chromium wrappers for e.g. out/AndroidPerformance/angle_end2end_tests - which is why capture_tests.py is currently using angle_android_test_runner.py Bug: b/296397568 Change-Id: I2606e74a17c90e381bb49ec8d555a6d24851b4f7 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4850889 Reviewed-by: Yuly Novikov <ynovikov@chromium.org> Reviewed-by: Cody Northrop <cnorthrop@google.com> Commit-Queue: Roman Lavrov <romanl@google.com>
Roman Lavrov 46ff02f8 2023-05-18T13:52:29 Capture/Replay: Initial setup for angle_capture_tests Implements the first part (FrameCapture) of the proposal go/frame-capture-and-interpreter-testing Adds a basic test (CapturedTest) with a few frames. This test gets captured by capture_tests.py into a temporary directory and the resulting files are diff'ed with the files under expected/ A diff fails the test. When capture changes, the workflow would be to run the command indicated by the error message in the test which will overwrite the files with new ones so that they can be added to the CL. Example test failure on capture change: https://chromium-swarm.appspot.com/task?id=62b5f4034527c610 when testing https://crrev.com/c/4598046/3 Tests in CI: https://screenshot.googleplex.com/77o8vZVuj8AbFRj Also adds a "angle_capture_tests_trace" lib with the trace just to test that this capture also builds, the lib is not currently loaded by anything. Bug: b/286067106 Change-Id: I7d5f6eed088d84f9e3eb8a72b24b1d92515fff38 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4545408 Reviewed-by: Cody Northrop <cnorthrop@google.com> Commit-Queue: Roman Lavrov <romanl@google.com> Reviewed-by: Yuly Novikov <ynovikov@chromium.org>
Yuly Novikov 2678b830 2023-02-23T18:19:30 Reenable GLES 1.0 conformance tests on Win/Intel Hopefully blue screen and timeouts are fixed by the new driver. Bug: angleproject:7750 Change-Id: Ic84f3080aeac689fa04f4fc8fc2f7f9c582abc10 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4286568 Auto-Submit: Yuly Novikov <ynovikov@chromium.org> Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org> Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org>
Yuly Novikov 6382dd74 2023-01-04T13:49:50 Switch Win Intel to Optiplex 7080 by rolling Chromium Optiplex 7050 had HD 630 GPU PCI ID 8086:5912. Optiplex 7060 had UHD 630 GPU PCI ID 8086:3e92. Optiplex 7080 has UHD 630 GPU PCI ID 8086:9bc5. Also upgrading the driver from 26.20.100.8141 to 31.0.101.2111. And the Windows 10 version from 1909 to 21H1. Re-enable angle_restricted_trace_gold_tests on Intel, since they looked stable on the experimental bot. Stop running tests on the experimental win10-x64-exp-intel. Roll chromium_revision 87b7ed3032..9b6c0a64c3 (1088609:1088686) Change log: https://chromium.googlesource.com/chromium/src/+log/87b7ed3032..9b6c0a64c3 Full diff: https://chromium.googlesource.com/chromium/src/+/87b7ed3032..9b6c0a64c3 Changed dependencies * testing: https://chromium.googlesource.com/chromium/src/testing/+log/9c112b5654..c5a91cd488 * third_party/catapult: https://chromium.googlesource.com/catapult.git/+log/{catapult_..a404e6d108 * tools/mb: https://chromium.googlesource.com/chromium/src/tools/mb/+log/eb0a826405..2c8d802803 * tools/perf: https://chromium.googlesource.com/chromium/src/tools/perf/+log/7e2765e6fa..0f2155f845 DEPS diff: https://chromium.googlesource.com/chromium/src/+/87b7ed3032..9b6c0a64c3/DEPS No update to Clang. Bug: chromium:1349828, angleproject:7708 Change-Id: I8af85de8334c01a5cdd62149ab30be9eaba5d07a Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4133266 Auto-Submit: Yuly Novikov <ynovikov@chromium.org> Reviewed-by: Jamie Madill <jmadill@chromium.org> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Yuly Novikov 104faff5 2022-12-09T15:01:54 Enable angle_restricted_trace_gold_tests retries on Win Intel UHD630 Flaky 4x8 pixel artifacts like with older GPU/driver. Also expand flaky Texture2DTestES3.NonZeroBaseEmulatedClear skip. Bug: angleproject:5415, angleproject:7866 Change-Id: Iea11f7dcb75e71f89dcf83f0cdde7ff74d6385f9 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4092005 Commit-Queue: Yuly Novikov <ynovikov@chromium.org> Reviewed-by: Jamie Madill <jmadill@chromium.org> Commit-Queue: Jamie Madill <jmadill@chromium.org> Auto-Submit: Yuly Novikov <ynovikov@chromium.org>
Yuly Novikov 81a3c71e 2022-12-01T18:51:39 Enable tests on Win Intel Optiplex 7080 experimental bot Suppress failing end2end tests and run single process Suppress VUID-VkDescriptorImageInfo-imageView-06711 Bug: chromium:1349828, angleproject:7874 Bug: angleproject:7865, angleproject:7866, angleproject:7872 Change-Id: I18c9099af6635cf41f5368960dc1d35d9decc48c Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4071925 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Jamie Madill <jmadill@chromium.org> Auto-Submit: Yuly Novikov <ynovikov@chromium.org>
Jamie Madill dc62b3ee 2022-10-10T21:00:16 Capture/Replay: Add trace interpreter. Also adds a self-test using the retrace script. Bug: angleproject:7752 Change-Id: I1985b47250bef99726d2ca2d90bef859208e357e Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3965128 Reviewed-by: Cody Northrop <cnorthrop@google.com> Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Yuxin Hu <yuxinhu@google.com>
Yuly Novikov 96bbc80b 2022-10-13T15:09:12 Restore default angle_gles1_conformance_tests Win Intel batch size Low batch size suspect for causing blue screen on this platform. Bug: angleproject:7750 Change-Id: I331f3aef22e313e318aeabb84b1277f4bf1678c9 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3953493 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Jamie Madill <jmadill@chromium.org> Auto-Submit: Yuly Novikov <ynovikov@chromium.org> Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org>
Yuly Novikov 115d8361 2022-07-21T13:59:48 Switch Linux Intel testing to UHD 630 GPU Replacing 0x5912 HD 630 with 0x9bc5 UHD 630. Also updating Mesa from 19.0.2 to 20.0.8. Since no problems were observed on the new configuration during the experiment, removing max-processes and flaky-retries workarounds. Bug: chromium:1342945 Change-Id: I51150092c67802ffd54796dd3b067cd6473dfb86 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3780922 Reviewed-by: Roman Lavrov <romanl@google.com> Auto-Submit: Yuly Novikov <ynovikov@chromium.org> Commit-Queue: Yuly Novikov <ynovikov@chromium.org> Commit-Queue: Roman Lavrov <romanl@google.com>
Jamie Madill d99864c0 2021-06-07T12:48:01 Gold Tests: Fix flaky test handling. Indead of just retrying the batch, retry individual tests. This should correctly handle the Intel flake pattern. Bug: angleproject:5415 Change-Id: I029c2514ff8e7a45184c5c105bf2d0350171da29 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2940049 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Jonah Ryan-Davis <jonahr@google.com> Reviewed-by: Yuly Novikov <ynovikov@chromium.org>
Jamie Madill 1bdd34ff 2021-04-26T09:04:59 infra: Add test specs for standalone tests. Includes test specs for Linux/Mac/Win7/Win10. Android is left out for a second pass to keep the changes a bit simpler. Bug: angleproject:5114 Change-Id: I94e7f7f273073f85488ec982e89b4b68f45627ac Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2805371 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Yuly Novikov <ynovikov@chromium.org>
Jamie Madill 29dc2b6e 2021-04-13T09:32:30 infra: Move and check gn_isolate_map. In preparation for more source-side configuration. Also adds a presubmit check and some empty files to enable the presubmit to function. Bug: angleproject:5114 Change-Id: Id9a0cbe1e78949720be39c65efab73c1345d4f39 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2819942 Reviewed-by: Yuly Novikov <ynovikov@chromium.org> Commit-Queue: Jamie Madill <jmadill@chromium.org>