infra/specs/generate_test_spec_json.py


Log

Author Commit Date CI Message
Yuly Novikov 5e443fd4 2025-05-16T12:35:15 Target Android 15 on Samsung S24 phones Phones were upgraded to AP3A.240905.015.A2 Bug: b/416745553 Change-Id: I8c1afd2526bb3e166c06264316e026b3d3f650b7 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/6556182 Reviewed-by: Yuxin Hu <yuxinhu@google.com> Commit-Queue: Yuly Novikov <ynovikov@chromium.org> Reviewed-by: Brian Sheedy <bsheedy@chromium.org>
Yuly Novikov 1ee585fd 2025-04-09T10:03:26 Rename ANGLE S22 builders to S24 Also start targeting S24 phones on the new builders. Bug: angleproject:394893136 Change-Id: I50496a5a5b114034bccf326001aa8ebc8a601398 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/6439516 Reviewed-by: Cody Northrop <cnorthrop@google.com> Commit-Queue: Yuly Novikov <ynovikov@chromium.org>
Garrett Beaty 2fa5e818 2024-08-15T16:39:12 Load //testing/buildbot/mixins.pyl directly. //infra/spec/generate_test_spec_json.py only needs the mixins used in the chromium //testing/buildbot directory. Rather than load all of the configuration files, loading mixins.pyl directly will remove the need for gn_isolate_map.pyl, test_suites.pyl and variants.pyl to be synced over to //testing/buildbot in the chromium/src repo. Bug: chromium:360187776 Change-Id: I67106785e9242e9eca8a42c1f0dea26d529b5e02 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/5792035 Reviewed-by: Yuly Novikov <ynovikov@chromium.org> Commit-Queue: Yuly Novikov <ynovikov@chromium.org>
Roman Lavrov 8007cad4 2024-06-14T09:46:48 Android tests: run perf tests on temp_band < 30C bots Applying to both pixel 6 and pixel 4 bots. This should improve the odds of avoiding overheating bots. Bug: b/345514719 Change-Id: Iee3a3efc7ccf93baee56de8b72e05d333f513e3a Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/5630172 Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org> Commit-Queue: Roman Lavrov <romanl@google.com> Reviewed-by: Yuly Novikov <ynovikov@chromium.org>
Yuly Novikov 6d10e966 2024-05-07T15:14:02 Switch ANGLE to new GPU Android mixins Bug: chromium:338536265 Change-Id: I25b7ff35ccea3fe8cde1661715f86fc341d2b82e Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/5522773 Commit-Queue: Brian Sheedy <bsheedy@chromium.org> Reviewed-by: Brian Sheedy <bsheedy@chromium.org> Auto-Submit: Yuly Novikov <ynovikov@chromium.org>
Yuly Novikov f8f2a10b 2024-05-03T18:11:22 Prepare ANGLE to use new Android mixins Bug: chromium:338536265 Change-Id: I225b79cb1a26b63cb74f0d2bd2c1f14048882722 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/5514150 Auto-Submit: Yuly Novikov <ynovikov@chromium.org> Reviewed-by: Brian Sheedy <bsheedy@chromium.org> Commit-Queue: Brian Sheedy <bsheedy@chromium.org>
Cody Northrop 057db6ef 2024-02-26T10:59:20 Add ANGLE experimental S22 build and test Builders: android-arm64-exp-s22-test Testers: android-arm64-exp-s22 Bug: b/267953710 Change-Id: I28b1fe022589a2b75c90e26fc5e2aa7536b8138f Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/5324785 Reviewed-by: Yuly Novikov <ynovikov@chromium.org> Commit-Queue: Cody Northrop <cnorthrop@google.com>
Yuly Novikov 55279449 2023-07-03T19:42:52 Roll chromium_revision 2e0371f07e..ad19957265 (1162850:1165395) Don't update testing beyond https://chromium.googlesource.com/chromium/src/testing/+/8e38c8e8a6b4251a2f9584d9d9bfbf261d0c1835 Remove $mixin_append usage due to https://chromium.googlesource.com/chromium/src/testing/+/a5415f3c674a065cc541c618df12262073d4d10e Change log: https://chromium.googlesource.com/chromium/src/+log/2e0371f07e..ad19957265 Full diff: https://chromium.googlesource.com/chromium/src/+/2e0371f07e..ad19957265 Changed dependencies * build: https://chromium.googlesource.com/chromium/src/build.git/+log/e4262406c0..e0690c7c78 * buildtools: https://chromium.googlesource.com/chromium/src/buildtools.git/+log/f089c59d7c..6d00f8d8c5 * buildtools/third_party/libc++/trunk: https://chromium.googlesource.com/external/github.com/llvm/llvm-project/libcxx.git/+log/b272a1c128..84fb809dd6 * buildtools/third_party/libunwind/trunk: https://chromium.googlesource.com/external/github.com/llvm/llvm-project/libunwind.git/+log/b5a43ecdac..f1c687e0aa * testing: https://chromium.googlesource.com/chromium/src/testing/+log/46c0754180..ac3cb62a5a * third_party/abseil-cpp: https://chromium.googlesource.com/chromium/src/third_party/abseil-cpp/+log/66a5ea107e..7d4c4bf473 * third_party/android_build_tools/lint: kqQgSDAgfuKTUIT8gpYlhM9gvajbtIDs_ApDbh3wdNIC..DjoXGsdMeTwnppye9Nwe5ozhXIqKyJQDzgb0nTFH8SAC * third_party/android_build_tools/manifest_merger: pRHDE8UAgipcDQINCUsRz94lgm5BHQjiL-BLF6d6xC4C..MN3CF2GQ8xeB6obj4qf5J6l15-NoA43u4__RQTTe8I4C * third_party/android_toolchain: version:2@r25c.cr1..R_8suM8m0oHbZ1awdxGXvKEFpAOETscbfZxkkMthyk8C * third_party/catapult: https://chromium.googlesource.com/catapult.git/+log/{catapult_..48b31edbfc * third_party/depot_tools: https://chromium.googlesource.com/chromium/tools/depot_tools.git/+log/f0fba1d307..872ac9bd3d * third_party/fuchsia-sdk/sdk: version:13.20230626.3.1..version:13.20230629.2.1 * third_party/turbine: G8Cku1fztaDd9to_8gk3PNWM2iRacAxD9zcUAgUPUEAC..9rSngS4o9OIlNubLcU1zXF_w0dPYif6AS-_k1pPAV1kC * third_party/zlib: https://chromium.googlesource.com/chromium/src/third_party/zlib/+log/f81f385906..f999826d2d * tools/android: https://chromium.googlesource.com/chromium/src/tools/android/+log/5c2008090e..39d4cc4ec8 * tools/clang: https://chromium.googlesource.com/chromium/src/tools/clang.git/+log/053092c268..36a80749cf * tools/luci-go: git_revision:39f255d5875293d3e1d978888b819ac124a8b0cc..git_revision:15be1e7f953cb4ef6d88aba86f57ca28724d1c9a * tools/luci-go: git_revision:39f255d5875293d3e1d978888b819ac124a8b0cc..git_revision:15be1e7f953cb4ef6d88aba86f57ca28724d1c9a * tools/mb: https://chromium.googlesource.com/chromium/src/tools/mb/+log/f80577ef22..33899440a1 * tools/perf: https://chromium.googlesource.com/chromium/src/tools/perf/+log/daed2f7981..422351564c * tools/skia_goldctl/linux: tVDltUu_xw7_tk4k-rnp7Ua8pKhf3EEHmTwX5c6BS4AC..209WoWuXITgke_BJDnvM7aTy7wu2jFOUPYs4uIoJ3CAC * tools/skia_goldctl/mac_amd64: ov_DuOSHCneWQSTW6AZ9gWHEeoJxXEYDX9dl_0rA_d0C..vEHTSSZiGWhNdCjMGZ22sj4PfyPzgoIIkV97YXOJeKEC * tools/skia_goldctl/mac_arm64: 3wDkZAVD4vV1dNhXfRNMOUBvODbzJ23AVmmsFbTsbkQC..CcHM6-LZ7_b-Api718RTCYQPD0v17lWKuQBaz42b8r4C * tools/skia_goldctl/win: yW7os_HVMoE-wq04olb6B7GNgEmaYZYPN0ecTzPPBdkC..u5j7yik1xZdFc9dECFfFDX9vRP9go2GpqukfJwOLCbgC DEPS diff: https://chromium.googlesource.com/chromium/src/+/2e0371f07e..ad19957265/DEPS No update to Clang. Bug: chromium:1456553 Change-Id: I1c18e6c6f68c7d24ae72a6754c707397a4c739d2 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4660052 Reviewed-by: Ian Elliott <ianelliott@google.com> Commit-Queue: Yuly Novikov <ynovikov@chromium.org>
Roman Lavrov f102184d 2023-06-13T14:34:07 Make Test spec JSON generator hashless. Removes scripts/code_generation_hashes/Test_spec_JSON.json Instead, run codegen in --verify-only mode and compare content. This run during presubmits and is fast (~0.2s in my tests) Similar to https://crrev.com/c/4604579 With this, should be able to auto-resolve conflicts in infra/specs/angle.json Also testing/buildbot/mixins.pyl seems to have had hashes routinely updated by autorolls. Bug: angleproject:8193 Change-Id: Ic1a657dbf464e6f4a8066ea8c5e18297e27a3b4c Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4605467 Reviewed-by: Yuly Novikov <ynovikov@chromium.org> Auto-Submit: Roman Lavrov <romanl@google.com> Commit-Queue: Roman Lavrov <romanl@google.com> Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org>
Brian Sheedy 9566011b 2023-02-23T15:01:59 Add support for permissive pixel comparison Adds support for running ANGLE pixel tests with a more permissive inexact matching algorithm. This is done by passing in --use-permissive-pixel-comparison=1 to the test runner. This is intended to reduce the amount of manual triage work required on CLs that are expected to have a larger amount of differences, such as SwiftShader rolls. On the bots, this will normally be disabled, but will be enabled if "Use-Permissive-Angle-Pixel-Comparison: True" is present as a CL footer. This footer is not yet included automatically anywhere, so there should be no functional change as a result of this CL yet. Bug: angleproject:7985 Change-Id: Ie815fac42edb2198dd4d115fc50650504df136c0 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4288612 Auto-Submit: Brian Sheedy <bsheedy@chromium.org> Commit-Queue: Yuly Novikov <ynovikov@chromium.org> Reviewed-by: Yuly Novikov <ynovikov@chromium.org>
Jamie Madill 8bac3563 2022-10-07T17:36:53 infra: Update timeout for perf tests. Changes 200 minutes to 120 minutes. Also applies the timeout to the Pixel 4 perf tests which are now timing out with the latest increase in warmup time. Bug: angleproject:7738 Change-Id: Ib43d61cb218aac24a5d9ae190813256a72b9eccc Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3937050 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Yuxin Hu <yuxinhu@google.com>
Jamie Madill d8489db5 2022-10-05T12:57:06 Perf Tests: Reduce resource usage. This halves the amount of time spent running the tests on desktop platforms. It also reduces the number of shards used on desktop platforms, to reduce resource utilization. It also bumps up the test timeout to keep from getting infra errors. Bug: angleproject:7738 Change-Id: Ie487c06452a39bb9ad1fe29b0d682e9c1bf297d5 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3936448 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Yuly Novikov <ynovikov@chromium.org>
Jamie Madill 0b5485c5 2022-03-04T08:26:06 Call generate_test_spec_json with vpython3. Bug: chromium:1302648 Change-Id: If071d8f98b28b0dc3d279d35abd2cecde0976a58 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3503688 Reviewed-by: Jamie Madill <jmadill@google.com> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Cody Northrop 00c4dc93 2022-02-18T09:10:34 Python3: Update generate_test_spec_json.py Bug: angleproject:5707 Change-Id: Ifa84ecfc4f718f2798b232da30613b8af4940a7a Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3475171 Auto-Submit: Cody Northrop <cnorthrop@google.com> Reviewed-by: Jamie Madill <jmadill@chromium.org> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Jamie Madill 3b891062 2022-02-16T11:38:31 Roll Chromium DEPS manually. Includes a fix for the removed Gold service account mixin. Also upgrades roll_chromium_deps to Python 3. Changed dependencies * build: https://chromium.googlesource.com/chromium/src/build.git/+log/021b19e04d..47f630a2c5 * buildtools/third_party/libc++abi/trunk: https://chromium.googlesource.com/external/github.com/llvm/llvm-project/libcxxabi.git/+log/01efcb5d51..738dc100c6 * buildtools/third_party/libunwind/trunk: https://chromium.googlesource.com/external/github.com/llvm/llvm-project/libunwind.git/+log/b323ac8f4c..1e0047c09e * testing: https://chromium.googlesource.com/chromium/src/testing/+log/2062399745..e5e34cbf4b * third_party/abseil-cpp: https://chromium.googlesource.com/chromium/src/third_party/abseil-cpp/+log/47dfacfcce..ad002f6051 * third_party/android_deps: https://chromium.googlesource.com/chromium/src/third_party/android_deps/+log/86630a4fe3..00852ba1c5 * third_party/catapult: https://chromium.googlesource.com/catapult.git/+log/{catapult_..6fbe58035d * third_party/depot_tools: https://chromium.googlesource.com/chromium/tools/depot_tools.git/+log/f72361d313..d6a3040bfd * third_party/zlib: https://chromium.googlesource.com/chromium/src/third_party/zlib/+log/4e87a80d55..901a5d97c2 * tools/clang: https://chromium.googlesource.com/chromium/src/tools/clang.git/+log/ad74e59c0f..0a9dd7740c * tools/mb: https://chromium.googlesource.com/chromium/src/tools/mb/+log/aa8103bd9b..25c9dfaa86 * tools/perf: https://chromium.googlesource.com/chromium/src/tools/perf/+log/6d4e79909d..715f5e95c1 No update to Clang. Bug: angleproject:5707 Change-Id: I781dc018443fab6bbde1351c6c0ab426338be4d8 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3469215 Auto-Submit: Jamie Madill <jmadill@chromium.org> Reviewed-by: Yuly Novikov <ynovikov@chromium.org> Commit-Queue: Yuly Novikov <ynovikov@chromium.org>
Jamie Madill 9a36c3b6 2021-11-15T13:46:06 infra: Update NVIDIA to 1660 testing. Bug: angleproject:6694 Change-Id: I2580b53e45e633af98be757d9bb081461c06de95 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3283163 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Brian Sheedy <bsheedy@chromium.org> Reviewed-by: Yuly Novikov <ynovikov@chromium.org>
Yuly Novikov 166dd0c7 2021-06-10T16:09:01 Roll chromium_revision 6c5859c895..399855f4bb (890175:891326) Manual changes: 1. Fix generate_test_spec_json.py to correctly handle ADDITIONAL_MIXINS Needed after angle_skia_gold_test mixin removed from Chromium in crrev.com/890333 2. Copy third_party/logdog from Chromium Needed after crrev.com/890539 in order to create isolates on Android and for tools/perf/process_perf_results.py Change log: https://chromium.googlesource.com/chromium/src/+log/6c5859c895..399855f4bb Full diff: https://chromium.googlesource.com/chromium/src/+/6c5859c895..399855f4bb Changed dependencies * build: https://chromium.googlesource.com/chromium/src/build.git/+log/8870cb4120..2192a63c23 * buildtools: https://chromium.googlesource.com/chromium/src/buildtools.git/+log/c793cca886..9d8449e380 * buildtools/linux64: git_revision:39a87c0b36310bdf06b692c098f199a0d97fc810..git_revision:393dab000d704a4364d085fa4c01ec7af176c8fa * buildtools/mac: git_revision:39a87c0b36310bdf06b692c098f199a0d97fc810..git_revision:393dab000d704a4364d085fa4c01ec7af176c8fa * buildtools/third_party/libc++abi/trunk: https://chromium.googlesource.com/external/github.com/llvm/llvm-project/libcxxabi.git/+log/7e3b76855b..2c53623d59 * buildtools/win: git_revision:39a87c0b36310bdf06b692c098f199a0d97fc810..git_revision:393dab000d704a4364d085fa4c01ec7af176c8fa * testing: https://chromium.googlesource.com/chromium/src/testing/+log/a62f8260df..ab567bcc84 * third_party/abseil-cpp: https://chromium.googlesource.com/chromium/src/third_party/abseil-cpp/+log/7949d87093..bd17c406ba * third_party/android_deps: https://chromium.googlesource.com/chromium/src/third_party/android_deps/+log/887e8d9009..fa47598982 * third_party/android_sdk: https://chromium.googlesource.com/chromium/src/third_party/android_sdk/+log/1cfc90728e..816daa2545 * third_party/catapult: https://chromium.googlesource.com/catapult.git/+log/{catapult_..17cf72ca75 * third_party/depot_tools: https://chromium.googlesource.com/chromium/tools/depot_tools.git/+log/b508ecd932..6d099d543d * third_party/nasm: https://chromium.googlesource.com/chromium/deps/nasm.git/+log/19f3fad68d..e9be5fd6d7 * third_party/protobuf: https://chromium.googlesource.com/chromium/src/third_party/protobuf/+log/82f8803671..f4241bd0f2 * third_party/turbine: _iPtB_ThhxlMOt2TsYqVppwriEEn0mp-NUNRwDwYLUAC..Om6yIEXgJxuqghErK29h9RcMH6VaymMbxwScwXmcN6EC * tools/clang: https://chromium.googlesource.com/chromium/src/tools/clang.git/+log/09481f56be..fd14318cc9 * tools/luci-go: git_revision:2cc9805d5ad186367461ef1c4f0c59098b450418..git_revision:725192cc79f07ea946e10a64baac06625c206968 * tools/luci-go: git_revision:2cc9805d5ad186367461ef1c4f0c59098b450418..git_revision:725192cc79f07ea946e10a64baac06625c206968 * tools/luci-go: git_revision:2cc9805d5ad186367461ef1c4f0c59098b450418..git_revision:725192cc79f07ea946e10a64baac06625c206968 * tools/mb: https://chromium.googlesource.com/chromium/src/tools/mb/+log/94630dfc19..12c8fa872b * tools/perf: https://chromium.googlesource.com/chromium/src/tools/perf/+log/5c84710692..2e6477d751 DEPS diff: https://chromium.googlesource.com/chromium/src/+/6c5859c895..399855f4bb/DEPS Clang version changed llvmorg-13-init-11999-g50c0aaed:llvmorg-13-init-12491-g055770d5 Details: https://chromium.googlesource.com/chromium/src/tools/clang/+/09481f56be7b1bbaf5a466be5d81691902825fcf..fd14318cc948b67e5ec8a0f2f7563142ccd6dac2/scripts/update.py Bug: angleproject:4483, angleproject:6037 Change-Id: I9035126bce55642d4dfce54eeace85093bdd1782 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2954241 Reviewed-by: Jamie Madill <jmadill@chromium.org> Commit-Queue: Yuly Novikov <ynovikov@chromium.org>
Jamie Madill 2917ab34 2021-06-07T11:41:54 infra: Override Gold mixin. This mixin is being removed from the Chromium version of the mixins. Therefore we have to move it into the additional mixins option of the generator. Bug: angleproject:6037 Change-Id: I605b7cbf35d6282754bf5fb26246e32b2edcea10 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2939466 Reviewed-by: Yuly Novikov <ynovikov@chromium.org> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Jamie Madill 18c9aa0d 2021-04-29T15:25:32 Run test spec update as part of codegen. Test specs now will be updated when the Chromium build files change. It will be automatically included as part of the Chromium->ANGLE roll. Bug: angleproject:5114 Change-Id: If99c2a20033d417a5999295f425a3bb203f5da3a Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2860962 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Yuly Novikov <ynovikov@chromium.org>
Jamie Madill 7c53d6af 2021-04-26T15:20:27 infra: Auto-generate mixins.pyl. The generate_test_spec_json.py file now reads ANGLE's waterfalls.pyl and Chromium's mixins.pyl to find the referenced mixins and copy them into ANGLE's mixins.pyl. This way we can update ANGLE's mixins easily without needing to manually copy them from Chromium's. In a follow-up we could make this run on the Chromium auto-roller. Bug: angleproject:5114 Change-Id: I332b3182732985b261e107dfe9a5c7c413c410a0 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2849184 Reviewed-by: Yuly Novikov <ynovikov@chromium.org> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Jamie Madill 1bdd34ff 2021-04-26T09:04:59 infra: Add test specs for standalone tests. Includes test specs for Linux/Mac/Win7/Win10. Android is left out for a second pass to keep the changes a bit simpler. Bug: angleproject:5114 Change-Id: I94e7f7f273073f85488ec982e89b4b68f45627ac Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2805371 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Yuly Novikov <ynovikov@chromium.org>