kc3-lang/angle/src/tests/capture_replay_tests.py

src/tests/capture_replay_tests.py

Log

Author Commit Date CI Message
Mark Lobodzinski 7181239d 2025-04-04T14:13:58 Add long ANGLE traces feature Enables very long Android captures by swapping binary data chunked buffers to/from disk. Bug: b/425728227 Change-Id: I539f72590eece03cfc69d42fc34be9825a9ff1fe Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/6476924 Reviewed-by: Cody Northrop <cnorthrop@google.com> Commit-Queue: Mark Łobodziński <mark@lunarg.com>
Junji Watanabe 214b7ce2 2025-05-09T11:35:31 Remove --use-reclient flag from capture_replay_tests.py The flag isn't used anymore. Bug: b/401959048 Change-Id: I6bedc9fcb2c5a5ed152b3da4bcbce0ba4785c883 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/6528770 Reviewed-by: Cody Northrop <cnorthrop@google.com> Reviewed-by: Yuly Novikov <ynovikov@chromium.org> Auto-Submit: Junji Watanabe <jwata@google.com> Commit-Queue: Yuly Novikov <ynovikov@chromium.org>
angle-autoroll 6bdbe9cc 2025-05-01T12:33:41 Manual roll Chromium from dd275389e023 to cf3a960742c3 (1975 revisions) Use third_party/depot_tools/gn.py instead of gn to accommodate https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6504176 Manual roll requested by ynovikov@google.com https://chromium.googlesource.com/chromium/src.git/+log/dd275389e023..cf3a960742c3 If this roll has caused a breakage, revert this CL and stop the roller using the controls here: https://autoroll.skia.org/r/chromium-angle-autoroll Please CC angle-team@google.com,geofflang@google.com,ynovikov@google.com on the revert to ensure that a human is aware of the problem. To file a bug in Chromium: https://bugs.chromium.org/p/chromium/issues/entry To file a bug in ANGLE: https://bugs.chromium.org/p/angleproject/issues/entry To report a problem with the AutoRoller itself, please file a bug: https://issues.skia.org/issues/new?component=1389291&template=1850622 Documentation for the AutoRoller is here: https://skia.googlesource.com/buildbot/+doc/main/autoroll/README.md Changed dependencies * build: https://chromium.googlesource.com/chromium/src/build.git/+log/88030b3203..aaf578b4fc * buildtools: https://chromium.googlesource.com/chromium/src/buildtools.git/+log/0f32cb9025..6d4e129ac6 * buildtools/linux64: git_revision:90478db6b59b9bebf7ca4cf912d860cf868e724c..git_revision:487f8353f15456474437df32bb186187b0940b45 * buildtools/mac: git_revision:90478db6b59b9bebf7ca4cf912d860cf868e724c..git_revision:487f8353f15456474437df32bb186187b0940b45 * buildtools/win: git_revision:90478db6b59b9bebf7ca4cf912d860cf868e724c..git_revision:487f8353f15456474437df32bb186187b0940b45 * testing: https://chromium.googlesource.com/chromium/src/testing/+log/fe092931f1..35b04da37d * third_party/android_build_tools/error_prone/cipd: -IrkxgAkj3RxGNBPp6b4r9OWZC6_dTbI-jC3c3UPgq8C..2t_u1XEjof609BQE4czlCQzEqRliKS_qY35FOu_VckgC * third_party/android_build_tools/lint/cipd: gx4NrAApWUVG5HG3WJRIIzZrTssD6H5uGbKe0g77mucC..TEDRB0F7VgCgyepd8FsIxPRqvKjvWQ_YkLV-ybx-GQYC * third_party/android_build_tools/manifest_merger/cipd: HhXknpcPosMUDHZQD-8Ogve-HaG4HmJ4p082zKKAcnMC..fPGWiT0LVG2_ifxOpHb5KNwTL-_7lb7tYOsB_ZoIiskC * third_party/android_deps: https://chromium.googlesource.com/chromium/src/third_party/android_deps/+log/1ea2115b17..a86aaa9531 * third_party/catapult: https://chromium.googlesource.com/catapult.git/+log/000f47cfa3..1591e813b6 * third_party/depot_tools: https://chromium.googlesource.com/chromium/tools/depot_tools.git/+log/1fcc527019..5982a1aeb3 * third_party/fuchsia-sdk/sdk: version:27.20250424.2.1..version:27.20250501.3.1 * third_party/libc++/src: https://chromium.googlesource.com/external/github.com/llvm/llvm-project/libcxx.git/+log/917609c669..62724eb55f * third_party/llvm-libc/src: https://chromium.googlesource.com/external/github.com/llvm/llvm-project/libc.git/+log/912274164f..bf95cc5da6 * third_party/r8/cipd: bA3htCoEd_EArHekDGQSNpmBzQrcby2ioG6SFyl3AtwC..sxgLKZyJNZMVi8cot5yvRzqCFQxX_HbbR0psaeto5RcC * third_party/r8/d8/cipd: 6qLey8EBp9ivhThnqVPWy2ZDGpsf5Y29EsTbi_rZ1pMC..bA3htCoEd_EArHekDGQSNpmBzQrcby2ioG6SFyl3AtwC * third_party/rust: https://chromium.googlesource.com/chromium/src/third_party/rust/+log/926ec54499..7c098b791b * third_party/zlib: https://chromium.googlesource.com/chromium/src/third_party/zlib/+log/1e85c01b15..6f9b4e6192 * tools/android: https://chromium.googlesource.com/chromium/src/tools/android/+log/60443ebac3..e4c0da2094 * tools/clang: https://chromium.googlesource.com/chromium/src/tools/clang.git/+log/6c4f037a98..5ae6e4bfb4 * tools/luci-go: git_revision:fd48b8efe637ea5f431fc371f1617b215b7ba1a2..git_revision:a9a10995e2889126ace91faf4052949e38c69d33 * tools/luci-go: git_revision:fd48b8efe637ea5f431fc371f1617b215b7ba1a2..git_revision:a9a10995e2889126ace91faf4052949e38c69d33 * tools/luci-go: git_revision:fd48b8efe637ea5f431fc371f1617b215b7ba1a2..git_revision:a9a10995e2889126ace91faf4052949e38c69d33 * tools/mb: https://chromium.googlesource.com/chromium/src/tools/mb/+log/3b0c718c2d..e3589a5831 * tools/perf: https://chromium.googlesource.com/chromium/src/tools/perf/+log/9c8188cae9..1a0049ce09 * tools/rust: https://chromium.googlesource.com/chromium/src/tools/rust.git/+log/fa679ed68e..7e70cdf64e * tools/skia_goldctl/linux: M9CSmdri8G3miQpGPQeqstBvQ3TtlngrsiF5GdSTYs0C..srGbc-O6bVQmBoT6VLvP66h6Be4A-nffx4UhixUJ9w0C * tools/skia_goldctl/mac_amd64: Txs7Izu5kRGbcoAc9qESynBnBHo8RdRIyim5zF9rWbgC..tZvHxy8AcDJhO9A-5UuZKL_-5vfE_wC-RBkElqWZpM4C * tools/skia_goldctl/mac_arm64: LTzU4UbKV4zw61tWyttEX62vyq7nErr2gD8-jiH8IhgC..2br8hIWKHDAaOt77siYyRUclHBYUZX6dj8jtogL90nwC * tools/skia_goldctl/win: UKiySjywM8juGVuMwWbOdaBXFD1TbzUOWcGGVTYfem0C..zasiUhQIHGZ2e60_M6g41oz8mYgPta5JmOWV5LwFjGMC No update to Clang. Bug: chromium:414330998 Change-Id: I5ce14c5574391c1dd638787c09789f0577ee26a2 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/6503456 Reviewed-by: Geoff Lang <geofflang@chromium.org> Commit-Queue: Yuly Novikov <ynovikov@chromium.org>
Junji Watanabe e93c143a 2025-04-28T15:39:35 tests: Prepare capture_replay_tests for Siso migration - Adds --use_remoteexec as an alias of --use_reclient. - Adds --use_siso to add `use_siso=true` GN arg. - Changes the out dir to out_CaptureReplayTest, so that the build dir depth will be 2 for Siso remote execution rules. - Cleans up build dir when switching from Ninja to Siso. Bug: b/401959048 Change-Id: Id698644f16c161e86d1d1b42828281cb5fb690ca Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/6489896 Auto-Submit: Junji Watanabe <jwata@google.com> Reviewed-by: Yuly Novikov <ynovikov@chromium.org> Commit-Queue: Yuly Novikov <ynovikov@chromium.org> Reviewed-by: Cody Northrop <cnorthrop@google.com>
Roman Lavrov e6d9854b 2025-01-09T09:12:53 Capture/replay tests: fix logging on capture error Introduced by https://crrev.com/c/6042183. Also remove unneeded trailing \n in a few spots. Bug: angleproject:380296979 Change-Id: Ie6438409533855b22bc85a02ec0017279be4880a Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/6163683 Auto-Submit: Roman Lavrov <romanl@google.com> Reviewed-by: Cody Northrop <cnorthrop@google.com> Commit-Queue: Cody Northrop <cnorthrop@google.com>
Roman Lavrov b856059e 2024-11-22T09:58:13 Tests: linux-trace logs return code with OOM note In case on the bug the replay was terminated with an empty stdout and signal handler wouldn't trigger, making it difficult to diagnose. This happened due to OOM killer (system low on memory) sending SIGKILL to the process, which can't be handled and results in termination with exit code -9. Add a note to the logs when this happens. Bug: angleproject:380296979 Change-Id: Id4db4b12c680701511430d90eee847baded03bcb Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/6042183 Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org> Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org>
Roman Lavrov 99d5d4d5 2024-11-21T09:02:32 Tests: fix replay failure handling in linux/win-trace Bug: angleproject:42264614 Change-Id: I752fb914b82222392bf88b51351217cd4d063cd6 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/6039443 Reviewed-by: Cody Northrop <cnorthrop@google.com>
Roman Lavrov 3cda3527 2024-07-30T14:28:17 Capture/Replay: fix CaptureProducedRequiredFiles bottleneck Turns out repeatedly listing/checking files was a huge bottleneck, especially on Windows. Example from win-trace GLES 3.0. Before this change almost 10 minutes!: I09:59:03.198422Z Running 2255 capture tests, worker_count=8 batch_count=1 I10:00:36.627151Z RunCaptureInParallel finished I10:10:20.380077Z CaptureProducedRequiredFiles finished I10:10:20.396174Z Building replay tests After: I11:02:57.679244Z Running 2255 capture tests, worker_count=8 batch_count=1 I11:04:33.078232Z RunCaptureInParallel finished I11:04:35.331878Z CaptureProducedRequiredFiles finished I11:04:35.344066Z Building replay tests win-trace timing expected to improve from ~30 minutes to ~15 minutes. Bug: angleproject:42264614 Change-Id: Ie1903d89d54ab5c6464d3ddb993ec3007e7604ea Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/5749745 Commit-Queue: Roman Lavrov <romanl@google.com> Reviewed-by: Amirali Abdolrashidi <abdolrashidi@google.com>
Roman Lavrov da864294 2024-07-18T18:44:55 Capture/Replay: rework capture_replay_tests, enable linux-trace Changing the way capture_replay_tests orders its stages. Previously, build, capture and replay would all run concurrently in multiple processes sometimes causing difficult to understand interaction between those stages. This CL changes it so that the test follows stages sequentially: * Build end2end tests * Run all capture tests (in parallel) * Check captured files and produce a single build for replay * Build replay binary and all tests * Run all replay tests (in parallel, but _one by one_) Note that running replay tests one by one avoids the confusing batch failures altogether. Now when a replay test is broken, the logs will specifically identify which one caused the issue. This also noticeably improves CI time as ninja builds of bigger batches are more efficient and possibly less linking is happening. Additionally, xvfb is supported directly in the script without relying on testing/xvfb.py, in a way that reuses xvfb displays instead of starting/stopping xvfb for each test and avoids races that are possible when running testing/xvfb.py concurrently (such as picking which display to use). This appears to significantly improve stability on linux-trace which have had to be disabled due to the flakiness. Bug: angleproject:42264614 Change-Id: I38a5547250cf5f63b8b9e4879f02cd715d8df0a9 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/5721996 Commit-Queue: Roman Lavrov <romanl@google.com> Reviewed-by: Cody Northrop <cnorthrop@google.com> Reviewed-by: Amirali Abdolrashidi <abdolrashidi@google.com>
Shahbaz Youssefi d193d51b 2024-06-17T22:46:08 Replace issue ids post migration to new issue tracker This change replaces anglebug.com/NNNN links. Bug: None Change-Id: I8ac3aec8d2a8a844b3d7b99fc0a6b2be8da31761 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/5637912 Reviewed-by: Geoff Lang <geofflang@chromium.org> Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org>
Roman Lavrov 7047d820 2024-03-19T17:17:22 win-trace (capture_replay_tests): cleanup after replay Windows builds produce a large .pdb file for each test, e.g. ClearTest_ClearIsClamped_ES2_Vulkan_SwiftShader.dll.pdb As there are many tests, these accumulate and run out of disk Remove files that have test labels in the filename, in the case above that's 'ClearTest_ClearIsClamped_ES2_Vulkan_SwiftShader' Bug: angleproject:8621 Change-Id: I995b561409f7ae714da81d246e7b7152d936e9d2 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/5378178 Reviewed-by: Amirali Abdolrashidi <abdolrashidi@google.com> Commit-Queue: Roman Lavrov <romanl@google.com> Auto-Submit: Roman Lavrov <romanl@google.com> Commit-Queue: Amirali Abdolrashidi <abdolrashidi@google.com>
Yuly Novikov 239e8caa 2024-02-12T11:29:20 Capture/Replay: Disallow concurrent ninja processes. To confirm whether reclient flaky failures are due to concurrency. Bug: b/323830218 Change-Id: I79ed257675624393135f9072fd207a46b4d2a30e Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/5285434 Auto-Submit: Yuly Novikov <ynovikov@chromium.org> Commit-Queue: Roman Lavrov <romanl@google.com> Reviewed-by: Roman Lavrov <romanl@google.com>
Bani Singh cf77126a 2024-02-06T15:50:15 Turn off ADC explicitly Prevent ADC to be used for windows bots, use gce instead. Bug: b/323830218 Change-Id: I15e0faeb2a22b6bcb933f8cecbe986489e4f2991 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/5273482 Reviewed-by: Yuly Novikov <ynovikov@chromium.org> Commit-Queue: Yuly Novikov <ynovikov@chromium.org>
Bani Singh 0640f651 2024-01-30T17:42:31 USE GCE Credentials for Chrome Bot In case of bot, use GCE instead of automatic_auth which will no longer fall back to ADC (and hence GCE). Bug: b/319246651 Change-Id: I86942c30bea911b1cc89870a748bf564584af660 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/5249329 Commit-Queue: Cody Northrop <cnorthrop@google.com> Reviewed-by: Cody Northrop <cnorthrop@google.com>
Bani Singh 66517b0b 2024-01-26T18:18:05 Revert "Use ADC for GCE VM" This reverts commit 2dfbedee7d4f9c84f278818217a1fa0908d6cbee. Reason for revert: May be causing credentials errors Original change's description: > Use ADC for GCE VM > > In case of bot, use ADC instead of automatic_auth which will in the > future be done through credshelper and will not support ADC. > > Test: Locally ran script with no errors, and tested that running > autoninja respects environment variables > > Bug: b/319246651 > Change-Id: I58a282047d387ef54659dbb195cc117b3459d2e8 > Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/5212967 > Reviewed-by: Roman Lavrov <romanl@google.com> > Reviewed-by: Cody Northrop <cnorthrop@google.com> > Commit-Queue: Cody Northrop <cnorthrop@google.com> Bug: b/319246651 Change-Id: I315651d99e937c331183b28ea7d7d665b43fc6d9 No-Presubmit: true No-Tree-Checks: true No-Try: true Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/5236466 Auto-Submit: Bani Singh <banisingh@google.com> Commit-Queue: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com> Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Bani Singh 2dfbedee 2024-01-18T16:52:10 Use ADC for GCE VM In case of bot, use ADC instead of automatic_auth which will in the future be done through credshelper and will not support ADC. Test: Locally ran script with no errors, and tested that running autoninja respects environment variables Bug: b/319246651 Change-Id: I58a282047d387ef54659dbb195cc117b3459d2e8 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/5212967 Reviewed-by: Roman Lavrov <romanl@google.com> Reviewed-by: Cody Northrop <cnorthrop@google.com> Commit-Queue: Cody Northrop <cnorthrop@google.com>
Takuto Ikuta 7b9b95f8 2023-11-13T11:42:26 drop goma support from capture_replay_tests.py https://ci.chromium.org/ui/p/angle/builders/try/win-trace/7316/overview failed, but it seems unrelated to this CL. Bug: angleproject:8309 Change-Id: Id4fd9cad341973c87570adbea63809b5aa84c4a0 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/5019418 Reviewed-by: Roman Lavrov <romanl@google.com> Auto-Submit: Takuto Ikuta <tikuta@chromium.org> Commit-Queue: Roman Lavrov <romanl@google.com>
Takuto Ikuta 1bb2c5ec 2023-11-07T15:44:41 support reclient in capture_replay_tests This also uses autoninja to start and stop reclient automatically. This will be used with https://crrev.com/c/4797373. Bug: angleproject:8309 Change-Id: I2e17c87c8ff0e253f16201297ef979337bf30292 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4796170 Reviewed-by: Yuly Novikov <ynovikov@chromium.org> Commit-Queue: Yuly Novikov <ynovikov@chromium.org> Auto-Submit: Takuto Ikuta <tikuta@chromium.org>
Yuly Novikov 918028a2 2023-10-31T17:53:07 Update bot detection in capture_replay_tests Since GOMA is going away, check username instead. Bug: angleproject:6085, angleproject:8309 Change-Id: Ie4fa4ec7fad8b52450c1ef8be9e5099f30c8fc7c Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4994667 Reviewed-by: Roman Lavrov <romanl@google.com> Commit-Queue: Yuly Novikov <ynovikov@chromium.org>
Roman Lavrov d6bf8e0d 2023-10-02T18:10:36 Improve capture_replay_tests (win-trace) logging * Get rid of the CRASH expectation as it is always confusing. Replace items in expectations with SKIP_FOR_CAPTURE. This will no longer run the corresponding tests (which was making logs even more confusing) * Use CaptureFailed and ReplayFailed instead of Crashed status (the actual meaning is a non-zero return code from the test harness) * Introduce batch names by indexing them as they are created: batch_000, batch_001, ... This enables easy searching in logs. * When logging a batch (result) include its name and why it is logged * Failed batches are printed with full output instead of "abbreviated" output. One issue with b/303065176 was that an important part of the output (errors) was truncated by this "abbreviation" * Remove "regression log" and "Summary" as these are often just confusing: regression log just duplicates information printed earlier and summary may not be clear w.r.t test expectations. The part that prints what was unexpected is still there. It also now includes batch names so it's much more obvious what comes from where. Example: Unexpected 'CaptureFailed' (8): != Pass: batch_000 FramebufferTest_ES3.RenderAndInvalidateImmutableTextureWithBellowBaseLevelLOD/ES3_Vulkan_SwiftShader != Pass: batch_000 FramebufferTest_ES3.RenderAndInvalidateImmutableTextureWithBellowBaseLevelLOD/ES3_Vulkan_SwiftShader_AsyncCommandQueue != Pass: batch_000 FramebufferTest_ES3.RenderAndInvalidateImmutableTextureWithBellowBaseLevelLOD/ES3_Vulkan_SwiftShader_EnableParallelCompileAndLink != Pass: batch_000 FramebufferTest_ES3.RenderAndInvalidateImmutableTextureWithSubImageWithBeyondMaxLevel/ES3_Vulkan_SwiftShader != Pass: batch_000 FramebufferTest_ES3.RenderAndInvalidateImmutableTextureWithSubImageWithBeyondMaxLevel/ES3_Vulkan_SwiftShader_AsyncCommandQueue != Pass: batch_000 FramebufferTest_ES3.RenderAndInvalidateImmutableTextureWithSubImageWithBeyondMaxLevel/ES3_Vulkan_SwiftShader_EnableParallelCompileAndLink != Pass: batch_000 FramebufferTest_ES3.RenderImmutableTextureWithSubImageWithBeyondMaxLevel/ES3_Vulkan_SwiftShader != Pass: batch_000 FramebufferTest_ES3.RenderImmutableTextureWithSubImageWithBeyondMaxLevel/ES3_Vulkan_SwiftShader_AsyncCommandQueue Unexpected 'FailedToTrace' (1): != Pass: batch_001 FramebufferTest_ES3.RenderSampleDepthTextureWithExcludedLevel/ES3_Vulkan_SwiftShader Bug: angleproject:7519 Bug: b/303065176 Change-Id: Ie0c4734c7c4ac965aa6fd38b0c1783e0b92af7a5 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4908349 Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org> Commit-Queue: Roman Lavrov <romanl@google.com> Reviewed-by: Cody Northrop <cnorthrop@google.com>
Shahbaz Youssefi e9719f5c 2023-10-02T12:15:35 Capture/Replay: Disable VK_EXT_host_image_copy during capture Bug: b/303065176 Change-Id: I87f9dd11cd87115f097bb35d44f024db19274d51 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4903077 Reviewed-by: Roman Lavrov <romanl@google.com> Commit-Queue: Roman Lavrov <romanl@google.com> Auto-Submit: Shahbaz Youssefi <syoussefi@chromium.org>
Roman Lavrov f382b244 2023-05-19T14:28:26 Add ci/linux-trace disabling logic to ANGLE script Currently disabled by this line in chromium/tools/build repo: https://crsrc.org/b/recipes/recipe_modules/angle/api.py;drc=e768c9b9c2a9b34346f9c009341761a034df7a00;l=91 I will remove that logic after this CL lands. This way we can decide whether to run this test inside ANGLE repo. Bug: angleproject:6085 Change-Id: I8efafb11149cda1fd47a5e9a95612e521823493a Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4549681 Auto-Submit: Roman Lavrov <romanl@google.com> Reviewed-by: Amirali Abdolrashidi <abdolrashidi@google.com> Commit-Queue: Amirali Abdolrashidi <abdolrashidi@google.com>
Junji Watanabe 3cfdb191 2022-11-28T14:42:34 Change ninja path to third_party/ninja/ninja It also removes `.exe` extension as it seems not necessary. Bug: chromium:1338373 Change-Id: Ic94426771d56c54b8c03627cdff6dccccec1a267 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4060128 Auto-Submit: Junji Watanabe <jwata@google.com> Commit-Queue: Yuly Novikov <ynovikov@chromium.org> Reviewed-by: Takuto Ikuta <tikuta@chromium.org> Reviewed-by: Yuly Novikov <ynovikov@chromium.org>
Jamie Madill 06ecc918 2022-10-10T20:59:38 Capture/Replay: Optionally emit C sources. This adds an option that controls if we can write out simplified C replays. Once the work is finished we can remove the option and only allow emitting C instead of both C and CPP. Required emitting multi-line strings differently, as well as conditionalizing a few other language differences. Bug: angleproject:7713 Change-Id: I3303134316ed3fc1b4286bcd32961e8f7ecfbb06 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3953339 Reviewed-by: Cody Northrop <cnorthrop@google.com> Reviewed-by: Yuxin Hu <yuxinhu@google.com> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Jamie Madill 11f8f537 2022-10-17T16:59:50 Capture/Replay: Improvements to self-test. - print errors in the replay test - catch missing serialization - clean stale json output files before each replay - add flag to show replay stdout - fix help message Bug: angleproject:7731 Change-Id: Ibeb780f417bdbed4b249038920963dd1bd68f4aa Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3962248 Reviewed-by: Yuxin Hu <yuxinhu@google.com> Reviewed-by: Cody Northrop <cnorthrop@google.com> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Gert Wollny ffd662e9 2022-08-01T10:49:19 capture/replay: Add MEC support to capture-replay script Running the capture/replay script in MEC mode is done by adding a parameter --mec <n> with n being the start frame of the capturing. Exactly one frame will be captured. Bug: angleproject:7564 Change-Id: I93153356d75712ba81ca26ccfc7f40785ba3e4ef Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3819845 Reviewed-by: Cody Northrop <cnorthrop@google.com> Reviewed-by: Jamie Madill <jmadill@chromium.org> Commit-Queue: Gert Wollny <gert.wollny@collabora.com>
Roman Lavrov adeb750d 2022-07-21T18:49:00 Validate that skipped tests are skipped by suite. Before this change, any test failing to produce trace files would be marked as "Skipped" regardless of whether it was skipped by suite or just failed to produce trace files for some other reason. This change adds FailedToTrace status to differentiate between the two cases. If a test is FailedToTrace it will need to be explicitly added to the capture_replay expectation file (probably SKIP_FOR_CAPTURE). This change caught that OcclusionQueriesNoSurfaceTestES3.SwitchingContextsWithQuery/* was incorrectly specified as NOT_RUN when the intent was to SKIP. Bug: angleproject:5133 Bug: angleproject:7516 Change-Id: I451b4d5fd799d15ef92fab678aa20d2b6cc42c9f Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3781902 Reviewed-by: Yuxin Hu <yuxinhu@google.com> Commit-Queue: Yuxin Hu <yuxinhu@google.com> Auto-Submit: Roman Lavrov <romanl@google.com> Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org>
Roman Lavrov 3a0d7249 2022-07-21T13:38:21 Ignore tests skipped by the suite. This way we don't have to add skipped tests to capture_replay_expectations as well. Bug: angleproject:5133 Bug: angleproject:7512 Bug: angleproject:7516 Change-Id: Iabc5f4738b51ce23890c07b8a0548cbc6bec1274 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3780583 Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org> Commit-Queue: Roman Lavrov <romanl@google.com>
Alexey Knyazev 1d67951e 2022-04-20T15:58:35 Avoid IOError in capture_replay_tests.py On macOS, psutil iterator may return a zombie process that causes IOError when accessing its name. Bug: angleproject:6102 Change-Id: I44692982c8dd78b4c5153077b5d7dccc7f87883a Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3596040 Reviewed-by: Jamie Madill <jmadill@chromium.org> Commit-Queue: Alexey Knyazev <lexa.knyazev@gmail.com>
Gert Wollny 5236a767 2022-02-23T12:15:26 Capture/Replay: enable running GLES 3.2 tests with script In addition add annotations for the tests that fail (crash) Bug: angleproject:5366 Change-Id: I92747e70b4c0a494fb7f67ed0c282bd76788f505 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3484057 Reviewed-by: Jamie Madill <jmadill@chromium.org> Reviewed-by: Cody Northrop <cnorthrop@google.com> Commit-Queue: Gert Wollny <gert.wollny@collabora.com>
Jamie Madill f60b152c 2021-12-06T10:19:53 Frame Capture: Merge small frames into single sources. By buffering up the replay functions before writing them out into sources, we can reduce the number of replay files by a large margin. Bug: angleproject:5133 Change-Id: I0a556fd1a3a5f2dcc8a26b761e885c9b7e464cf5 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3316200 Reviewed-by: Cody Northrop <cnorthrop@google.com> Reviewed-by: Tim Van Patten <timvp@google.com> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Gert Wollny 2020ec2b 2021-11-18T16:49:35 Capture/Replay: run tests that will crash or fail compilation These tests have to run in one-sized batches so that they do not bring down other tests run in the same batch, but it is useful to run these tests to see whether chenges to the code fix these tests. In addition, update the expectations to reflect that more tests do not crash or even pass. Mark one more test as FLAKY because a change of the batch allocation seems to change this test. Leave timeouts as skip to not burden the CI too much. Bug: angleproject:6180 Bug: angleproject:6267 Bug: angleproject:6267 Bug: angleproject:6313 Bug: angleproject:6314 Bug: angleproject:6315 Bug: angleproject:6324 Bug: angleproject:6329 Bug: angleproject:6333 Bug: angleproject:6334 Bug: angleproject:6335 Bug: angleproject:6399 Bug: angleproject:6400 Bug: angleproject:6605 Bug: angleproject:6611 Bug: angleproject:6631 Change-Id: I973ec43feff8ca51de698e0aade3d7102b193166 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3291643 Reviewed-by: Jamie Madill <jmadill@chromium.org> Reviewed-by: Cody Northrop <cnorthrop@google.com> Commit-Queue: Gert Wollny <gert.wollny@collabora.com>
Gert Wollny ff89ba1b 2021-11-19T11:52:55 Capture/Replay: Repeat logs for unexpected results The Ci logs are quite long and it is difficult to find the log for the actual CI failures, so repeat the logs for unexpected failures, crashes, compile errors, and timeouts at the end of the log. Bug: angleproject:6542 Change-Id: Id9dff4cc70649d0d9cc8bd4509477fbe3ef10de8 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3291653 Commit-Queue: Gert Wollny <gert.wollny@collabora.com> Reviewed-by: Jamie Madill <jmadill@chromium.org> Reviewed-by: Cody Northrop <cnorthrop@google.com>
Gert Wollny 2dd6ea51 2021-11-08T16:22:18 Capture/Replay: Limit the sleep time based on unfinished jobs When running only a few tests the script has the tendency to sleep a long time after the tests were finished. So limit the sleep time by the number of remaining jobs. Bug: None Change-Id: I52ad39bf047abf93cd900f687bd3aae3962e9e20 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3263999 Reviewed-by: Cody Northrop <cnorthrop@google.com> Reviewed-by: Jamie Madill <jmadill@chromium.org> Commit-Queue: Gert Wollny <gert.wollny@collabora.com>
Jamie Madill b930b9d3 2021-09-08T09:08:28 Capture/Replay: Remove redundant "capture" from label old: trex_200_capture_context1_frame001.cpp new: trex_200_context1_frame001.cpp The 'capture' word wasn't really adding any useful information. Take it out so the filenames are a bit shorter. Bug: angleproject:5133 Change-Id: Ibd7d47df2aa1f24a641c5d5b2a6dd14d6ce240e1 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3149370 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Tim Van Patten <timvp@google.com> Reviewed-by: Cody Northrop <cnorthrop@google.com>
Jamie Madill 5650273a 2021-09-07T09:41:59 Capture/Replay: Capture trace file list to JSON. Instead of using a text file, we can use a JSON file to store the list of files in a trace. We can use this list in the GN to pull the trace files directly instead of using auto-generated code. It also consolidates the trace metadata in a single location. Also captures the Windows Surface ID to the JSON file. Bug: angleproject:5133 Change-Id: I236b614f7d0c844e7c4a5276dd6ac5964e5c393d Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3144214 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Cody Northrop <cnorthrop@google.com> Reviewed-by: Tim Van Patten <timvp@google.com>
Jamie Madill 7e8fa429 2021-09-08T11:56:45 Capture/Replay Tests: Script cleanups. Minor cleanup changes and removes some outdated parameters. Includes logging information about how much time the script spends running each child process. Bug: angleproject:6102 Bug: angleproject:6299 Change-Id: I0b4da535b858ced3cdc21c341c2c001ffe9c8192 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3149190 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Cody Northrop <cnorthrop@google.com> Reviewed-by: Tim Van Patten <timvp@google.com>
Jamie Madill ac78d723 2021-09-08T12:19:40 Capture/Replay Tests: Allow 3 concurrent ninja processes. This is an attempt to fix the flaky ninja error when running multiple processes while still getting pretty good test times. The previous fix eliminated the flakiness but doubled the test time. In local testing this is a bit slower but not 2x slower. Bug: angleproject:6299 Change-Id: Ibcd462712d3eab759c5923ca229e009046783e9b Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3149570 Reviewed-by: Cody Northrop <cnorthrop@google.com> Reviewed-by: Tim Van Patten <timvp@google.com> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Jamie Madill 2f3e4db8 2021-09-02T10:31:13 Capture/Replay: Serialize trace metadata to a JSON file. This will allow us to remove more code auto-generation for the trace tests. The trace info now can be loaded directly from JSON instead of from the autogenerated information. Bug: angleproject:5133 Change-Id: I04e22b9279b19282df274bc8defcd363d0449111 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3140218 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Tim Van Patten <timvp@google.com> Reviewed-by: Cody Northrop <cnorthrop@google.com>
Jamie Madill b6e99bb3 2021-09-02T13:47:34 Rename and expand shader variable init feature. This feature now forces both output and unitialized locals to be initialized by the shader translator. This feature is needed by the trace validator to ensure we get deterministic behaviour in traces that exhibit some undefined results. Bug: angleproject:5133 Change-Id: Id1242cd077a57e891eed217f7671976ce1631a58 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3140216 Reviewed-by: Cody Northrop <cnorthrop@google.com> Reviewed-by: Tim Van Patten <timvp@google.com> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Gert Wollny ea609fba 2021-08-30T10:43:29 Capture/Replay: Change workdir for debug runs and annotation Append "Debug" to the capture_replay_tests workdir if it wasn't overridden to avoid having to recompile everyithing when running with or without the --debug flag given. Also add annotation for test that only fail when run with the --debug flag. Bug: angleproject:6282 Change-Id: I7c5251d8a6b1a010f3c0950909db60feeb58c611 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3129360 Commit-Queue: Gert Wollny <gert.wollny@collabora.com> Reviewed-by: Jamie Madill <jmadill@chromium.org> Reviewed-by: Cody Northrop <cnorthrop@google.com>
Gert Wollny ec50e22b 2021-08-27T10:17:12 Capture/Replay: Show some expectation when skipped tests are run These tests were marked "PASS or new tests", when failing which is misleading. Bug: angleproject:6282 Change-Id: I2adf4a29690f8442682e65e5326d173e4e157842 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3129142 Reviewed-by: Jamie Madill <jmadill@chromium.org> Reviewed-by: Cody Northrop <cnorthrop@google.com> Commit-Queue: Gert Wollny <gert.wollny@collabora.com>
Jamie Madill dd5994df 2021-08-25T18:39:31 Capture/Replay Tests: Fix Renderbuffer readback. Renderbuffer readback would use an unimplemented size function, ending up with a zero sized buffer. Also cleans up some of the code for reading back both Renderbuffers and Textures. One Depth/Stencil test that was crashing due to a recent change to read back depth data is now failing comparison because of missing combined depth/stencil support. Also updates the capture/reply self-test to handle ASAN-only test expectations. Bug: angleproject:4688 Change-Id: Idb6c05d15329f405f44502c2e5282764c7f780fb Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3119057 Reviewed-by: Cody Northrop <cnorthrop@google.com> Reviewed-by: Tim Van Patten <timvp@google.com> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Jamie Madill 6290a86b 2021-08-25T09:34:19 Capture/Replay Tests: Add lock around ninja calls. This should prevent huge numbers of calls to goma from causing test flakiness. Uses the Python multiprocessing.lock. Bug: angleproject:6299 Change-Id: I60fb38aaad363204de12d06f9d155bacb7ddd928 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3118269 Reviewed-by: Cody Northrop <cnorthrop@google.com> Reviewed-by: Tim Van Patten <timvp@google.com> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Gert Wollny 72ecb371 2021-08-24T11:17:12 Capture/Replay: further rework of replay script: * Simplify the code by using dictionaries to store the batch and final results * Use unified result specifiers * Group the unexpected result output by result type Bug: angleproject:6282 Change-Id: I90107efdeb51e183aa122ef67d1a646c8c1fd86a Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3114655 Commit-Queue: Gert Wollny <gert.wollny@collabora.com> Reviewed-by: Jamie Madill <jmadill@chromium.org> Reviewed-by: Cody Northrop <cnorthrop@google.com>
Gert Wollny 637e543f 2021-08-23T13:28:52 Capture/Replay: beautify the result output Instead of writing a diff between expected and real output list the tests that have results that differ from the expectations with their real and expected results. While at it also clean up the result printing a bit. Bug: angleproject:6282 Change-Id: Id95de9aa659b39f3a2fe3f63402e285ad875a4e7 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3113553 Reviewed-by: Jamie Madill <jmadill@chromium.org> Reviewed-by: Cody Northrop <cnorthrop@google.com> Commit-Queue: Gert Wollny <gert.wollny@collabora.com>
Jamie Madill 2f45d93d 2021-08-18T16:58:44 Capture/Replay: Init shader outputs during self-tests. This forces all uninitialized variables to have default values. For instance if the application doesn't initialize the output color, or a varying that's use in the output, this will ensure we don't use any undefined values in the computation. Found when working on a re-trace of T-Rex, which doesn't write to the alpha channel in the final rendering pass. Also fixes undefined values in GLSLTest.InactiveVaryingInVertexActiveInFragment. Bug: angleproject:5133 Change-Id: Ia291338e5adf23dab5263cb2ebe737dc05852d3e Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3110225 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Cody Northrop <cnorthrop@google.com> Reviewed-by: Tim Van Patten <timvp@google.com>
Jamie Madill 762db3c0 2021-08-18T09:06:42 Capture/Replay: Improvements to testing script. Using new Python APIs for multiprocessing logging. Also updates some logging to be simpler / more consistent. Also a few other small fixes. Bug: angleproject:5133 Change-Id: I4acbc9173a3891989f409cc499c8e6431e712c0c Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3104006 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Cody Northrop <cnorthrop@google.com> Reviewed-by: Tim Van Patten <timvp@google.com>
Jamie Madill 839d5318 2021-08-16T08:29:28 Capture/Replay: Changes to MEC first frame capture. Recent changes made it so it was impossible to trigger MEC *after* the first frame and *before* the second frame. Instead it was only possible to capture *before* the first frame or *after* the second frame. Makes it possible for the retracing script to do identical captures. Also includes a number of refactorings to make it simpler to follow the code, including renaming methods, and removing some extra output files. Bug: angleproject:5133 Change-Id: Ice6a189eb9f4d53e8ee1ba39beb537af2ef5fd9f Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3097807 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Tim Van Patten <timvp@google.com> Reviewed-by: Cody Northrop <cnorthrop@google.com>
Tim Van Patten 06b19ed8 2021-08-06T17:02:36 Framecapture: Update file naming Update the file naming to add "shared" and remove the unnecessary frame numbering: angle.angledata.gz angle_capture_context1.cpp angle_capture_context1.h angle_capture_context1_files.txt angle_capture_context1_frame001.cpp angle_capture_shared.cpp angle_capture_shared.h Bug: angleproject:6256 Change-Id: I6454a3dc412896557f29faee1269dfa55146d43b Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3078908 Reviewed-by: Jamie Madill <jmadill@chromium.org> Reviewed-by: Cody Northrop <cnorthrop@google.com> Commit-Queue: Tim Van Patten <timvp@google.com>
Gert Wollny 6c415487 2021-07-26T13:08:46 Capture/Replay: Add option to override test disabling In order to be able to easily run disabled tests, add a flag --force-run-capture that makes it possible to ignore the SKIP_FOR_CAPTURE designation. Bug: angleproject:6166 Change-Id: I195248045d5c928d965f16b5ae17681d78686374 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3053363 Commit-Queue: Gert Wollny <gert.wollny@collabora.com> Reviewed-by: Tim Van Patten <timvp@google.com> Reviewed-by: Jamie Madill <jmadill@chromium.org>
Gert Wollny af35fd22 2021-07-12T14:58:01 Capture/Replay: rework test runner to use expectations This CL changes the test runner to use a expectation file for all the tests including failing tests. The expectation file is also used to disable running certain tests. e.g. because they are known to crash or result in compilation failures, which then take down all the test in the corresponding batch. The capture_replay_tests.py script will report failure if the test result doesn't meet the expectation. In order to keep the current behaviour to run subsets of tests in tact, the test filter is also applied to the expectation. In addition to the "normal" expectations, FLAKY was added to tag tests that can be captured and compiled correctly but have unstable results. This CL also adds the expectation for ES2_Vulkan_SwiftShader and ES2_Vulkan Bug: angleproject:6166 Change-Id: Ica05303e6ae3b7c6d2b6749c2c9c6593b659cc1a Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3024209 Reviewed-by: Jamie Madill <jmadill@chromium.org> Reviewed-by: Tim Van Patten <timvp@google.com> Reviewed-by: Cody Northrop <cnorthrop@google.com> Commit-Queue: Gert Wollny <gert.wollny@collabora.com>
Jamie Madill 26329cb9 2021-07-30T09:55:24 Test Runner: Allow listing tests on Android. Because of quirks with the Chromium-provided Android test runner, we need to use a few tricks to get the test list output. We add placeholder output for a single test to trick the test runner into thinking it ran the tests successfully. We also add an end marker for the tests list so we can parse the list from the more spammy Android stdout log. Will enable a Python wrapper script to list and run performance tests outside of the GTest harness. Bug: angleproject:6090 Change-Id: I810d4722e9a6efa03ef94208e516401890af1c37 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3062240 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Cody Northrop <cnorthrop@google.com> Reviewed-by: Tim Van Patten <timvp@google.com>
Jamie Madill 60d76ec1 2021-06-25T17:35:22 Trace Tests: Autodetect Goma. Checks for the existence of the compiler_proxy process. Bug: angleproject:6102 Change-Id: I58848dc7cd62aeb65c28990212f29df6ff66db1a Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2989672 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Cody Northrop <cnorthrop@google.com>
Jamie Madill 2b86e2d6 2021-07-19T10:30:57 GN: Update dcheck/assert conditions. In preparation for a Chromium change that makes dchecks enabled by default in Release builds, we'll keep ASSERT disabled in Release by default. We add a new flag 'angle_assert_always_on' for controlling ANGLE ASSERT behaviour. Also slightly refactors the direct SPIR-V generation GN. Chromium CL: https://crrev.com/c/2893204 Bug: chromium:1225701 Change-Id: I01a6def28d7c21e771e9800889a6b9c274972de2 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3035447 Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org> Reviewed-by: Dirk Pranke <dpranke@google.com> Reviewed-by: Geoff Lang <geofflang@chromium.org> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Jamie Madill dd0e8a1f 2021-07-16T10:33:57 Capture/Replay tests: Enable dcheck_always_on. This will enable release ASSERTs. ASSERTs were unintentionally disabled when switching to release builds. Bug: angleproject:5133 Change-Id: Ie19e9fd02624b10283192467c4e22e3cf2b382c6 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3035521 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Cody Northrop <cnorthrop@google.com>
Jamie Madill ac61386d 2021-07-15T11:08:48 Capture/Replay Tests: Swap before TearDown. Swapping before TearDown lets the capture and serialization logic see all the test resources that will be cleaned up by the fixture. This will increase coverage quite a bit because many tests do automatic cleanup which would previously skip serialization. Bug: angleproject:6175 Change-Id: I85aa3f6d9bcf2fd66836523e55862a2d5f0d8e32 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3031702 Reviewed-by: Tim Van Patten <timvp@google.com> Reviewed-by: Cody Northrop <cnorthrop@google.com> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Tim Van Patten 223cd0ac 2021-06-15T18:46:07 Capture/Replay: Refactor shared context handling This is the initial CL to enable capture/replay of multi-context applications. This CL refactors FrameCapture and FrameCaptureShared to move much of the functionality into FrameCaptureShared, since most everything is shared by Contexts in the share group. For example, the setup of the majority of the GL objects is done in the new SetupReplayContextShared() function in the new $LABEL_capture_context_shared_frame001.cpp file. The setup is performed by (for example): void SetupReplay() { $LABEL::InitReplay(); $LABEL::SetupReplayContextShared(); SetupReplayContext2(); } This performs the shared setup first, followed by the context-specific setup, which may reference shared objects careated by LABEL::SetupReplayContextShared(). No re-capturing is required with this change, since the external APIs (i.e., SetupReplay()) are still the same. Bug: angleproject:5878 Test: Manual MEC and replay of Magic Tiles 3, Candy Crush Soda Saga, Temple Run 2 Change-Id: Iab7bfe651437e9be1dee83514cd97acc20c61d1d Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2965780 Commit-Queue: Tim Van Patten <timvp@google.com> Reviewed-by: Cody Northrop <cnorthrop@google.com> Reviewed-by: Jamie Madill <jmadill@chromium.org>
Jamie Madill 68d4e392 2021-07-02T14:18:54 Trace Tests: Add option for ASAN builds. ASAN builds can detect OOB memory accesses. This is very useful for diagnosing some kinds of flaky OOB crashes. Removes a test that was calling BufferSubData with a null data argument. This behaviour is undefined and we shouldn't be testing it. Also includes some minor refactorings and script changes. Bug: angleproject:5133 Change-Id: I5a7d9a5500c50a51f6369e944a5f0a533709f00d Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3002510 Reviewed-by: Jamie Madill <jmadill@chromium.org> Reviewed-by: Gert Wollny <gert.wollny@collabora.com> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Gert Wollny 27e9883b 2021-06-22T12:20:08 Capture/Replay: Limit number of compile processes based on load When replaying, for each batch a limited number of compile processes is launched, but no all-over limit is set, so that on computers with a low processor count and/or limited working memory the machine may become un-responsive. Therefore, tell ninja to not spawn new processes when the load is larger than the number of available processors. Bug: angleproject:6078 Change-Id: I9982a139ef2346d0cf25fbd510c7aff038e48071 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2976658 Reviewed-by: Jamie Madill <jmadill@chromium.org> Reviewed-by: Cody Northrop <cnorthrop@google.com> Commit-Queue: Gert Wollny <gert.wollny@collabora.com>
Jamie Madill da523f6b 2021-06-25T17:25:24 Trace Tests: Use hermetic depot_tools. We no longer need to use the depot_tools in the system path. Bug: angleproject:6102 Change-Id: I6772ceeb4c61e686d15b2638fb4348e93179e321 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2989671 Reviewed-by: Cody Northrop <cnorthrop@google.com> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Jamie Madill da17d561 2021-06-23T13:50:15 Trace Tests: Use xvfb consistently on Linux. Previously there were two places where we missed the xvfb script. This was causing the replay to fail. Bug: angleproject:6085 Change-Id: I833916fa0cdacc163ec2bdd08831249807f319c5 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2979353 Reviewed-by: Cody Northrop <cnorthrop@google.com> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Gert Wollny b6009f64 2021-06-17T14:53:03 Capture/Replay: Add feature for robust resource init Query the feature when creating the display and the context, and pass the flag to the created context and also to durfaces. With that we create surfaces that will be marked "MayNeedInit" and are initialized if the resource is cleared or invalidated. Bug: angleproject:6041 Change-Id: I292f2e3f931736a18db93695441407e17d2265b6 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2976656 Commit-Queue: Gert Wollny <gert.wollny@collabora.com> Reviewed-by: Jamie Madill <jmadill@chromium.org> Reviewed-by: Cody Northrop <cnorthrop@google.com>
Jamie Madill b80f1377 2021-06-22T14:42:00 Trace Tests: Print serialization diff on failure. Instead of only printing this with the "verbose" flag, print the diff of the serialization as an "info" message. This will make the diff show up on the bots. Bug: angleproject:5715 Change-Id: Ic9684b43cdf2d6ae248b5beab4317b0e939567c1 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2979854 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Cody Northrop <cnorthrop@google.com>
Gert Wollny 1af7caa4 2021-05-14T11:46:54 Capture/Replay: Add suffix to label for test file search When tests are run with a wildcard filter on the backend, e.g. "TEST/ES2_Vulkan*" the same tests from different backends may end up in the same trace file directory, e.g. TEST_ES2_Vulkan and TEST_ES2_Vulkan_SwiftShader. So far, when searching for the ES2_Vulkan trace files the check for trace files would them also pick up all the _SwiftShader files, and the skript would assume that the file count is wrong and skip the replay. By adding "_capture" the start of the commonly used suffix, this is avoided and replays are no longer skipped because mis-counting capture files. Bug: angleproject:5972 Change-Id: I7af54aaf51ee7070333ffe81974d866bd89dbd6f Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2897236 Commit-Queue: Gert Wollny <gert.wollny@collabora.com> Reviewed-by: Jamie Madill <jmadill@chromium.org> Reviewed-by: Cody Northrop <cnorthrop@google.com>
Gert Wollny 022401f9 2021-05-12T09:21:04 Capture/Replay: Print context diff also with frame gaps The capture/replay context differences may not be in each frame, so if the test signals that comparisons fail, don't bail if the context JSON files are not found for one frame, but continue the search until all possibly captured frames have been checked Bug: angleproject:5953 Change-Id: I8af34cc0fe4de5615fc43ef6c45915055546f507 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2891772 Commit-Queue: Gert Wollny <gert.wollny@collabora.com> Reviewed-by: Jamie Madill <jmadill@chromium.org> Reviewed-by: Cody Northrop <cnorthrop@google.com>
Gert Wollny 31cd0e98 2021-05-10T10:37:54 Capture/Replay: track robustResourceInit Bug: angleproject:5938 Change-Id: I7c9ab9bebd1ec2dc572a071844a860f99d0befe6 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2882580 Reviewed-by: Cody Northrop <cnorthrop@google.com> Reviewed-by: Jamie Madill <jmadill@chromium.org> Commit-Queue: Cody Northrop <cnorthrop@google.com>
Gert Wollny 39cef942 2021-04-27T08:28:13 Capture/Replay: capture WebGL compatibility With that we can also enable the tests DifferentStencilMasksTest.* Bug: angleproject:5851 Change-Id: If2c992e456716f606320e3992f389823d0af050f Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2853584 Commit-Queue: Gert Wollny <gert.wollny@collabora.com> Reviewed-by: Cody Northrop <cnorthrop@google.com> Reviewed-by: Jamie Madill <jmadill@chromium.org>
Gert Wollny 078a2ba6 2021-04-19T11:52:38 Capture/Replay: Print capture and replay exe in verbose mode When running the test script to capture and replay it is helpful to know the locations of the executables that are run to make it simpler to locate them for debugging. Bug: None Change-Id: Ia43bd9863e94058515d9023981863ad6c711285e Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2850640 Commit-Queue: Gert Wollny <gert.wollny@collabora.com> Reviewed-by: Jamie Madill <jmadill@chromium.org> Reviewed-by: Cody Northrop <cnorthrop@google.com>
Gert Wollny 3e1af85a 2021-04-19T18:12:30 Capture/Replay: Handle bindGeneratesResource in trace meta data Bug: angleproject:5844 Change-Id: I0243cab70b5833efd83b0d75a1801f68025ee291 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2835889 Commit-Queue: Gert Wollny <gert.wollny@collabora.com> Reviewed-by: Jamie Madill <jmadill@chromium.org> Reviewed-by: Cody Northrop <cnorthrop@google.com>
Gert Wollny c1da2794 2021-04-19T11:49:41 Capture/Replay: Handle ClientArraysEnabled in trace meta data Bug: angleproject:5848 Change-Id: Ieba325e2b5efa3242b24cafa8363e562c258546c Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2835888 Commit-Queue: Gert Wollny <gert.wollny@collabora.com> Reviewed-by: Cody Northrop <cnorthrop@google.com> Reviewed-by: Jamie Madill <jmadill@chromium.org>
Gert Wollny 41c66442 2021-04-12T16:46:47 Capture/Replay: Print context diff on failure when verbose When run with the --verbose flag, and a test fails because the context comparison fails, then print the unified diff between the JSON serialization of context that was captured and the context when it was replayed. Bug: angleproject:5715 Change-Id: I86210d96e5bbdf63f1ed663ddf8dfcb33764cbea Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2821535 Commit-Queue: Gert Wollny <gert.wollny@collabora.com> Reviewed-by: Cody Northrop <cnorthrop@google.com> Reviewed-by: Jamie Madill <jmadill@chromium.org>
Jamie Madill 78dde332 2021-03-29T17:31:52 Move restricted traces to CIPD. All traces are now stored as DEPS entries in CIPD. The auto-generation script generates the DEPS entries. Note that we don't include DEPS in the list of generated outputs to simplify other rollers. Also we update auto-generation to include full sources list to allow 'gn analyze' to work successfully. Usees a trace fixture for common code. This will enable a more compact trace without as much repeated code. We must land a set of re-trace traces to avoid breakage. Also includes a python script for uploading new traces to CIPD. The script first checks if traces are already present in the cloud, and if so it skips the upload. It will take a while to complete as the number of traces grows larger as it takes a few seconds per trace. The traces in this patch are also re-traced to use the common fixture code instead of including duplicated code in each trace. They now form a simple common interface and the autogenerated cpp is now simply a list of properties. I've also updated the capture/replay tests to use the simpler common entry point integration. There is less auto-generated glue code now. We now use a new serialized string query extension instead of calling directly into ANGLE's internals. Also includes a docs update. The capture/replay sample is broken and we'll need to update it in a follow-up CL. Also includes a few necessary fixes to the retracing script. Bug: angleproject:5811 Change-Id: I977bc6dc56843c8966377fc445ae97e91e17319a Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2797833 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Cody Northrop <cnorthrop@google.com> Reviewed-by: Tim Van Patten <timvp@google.com>
Gert Wollny cee0bfd6 2021-03-15T14:39:30 Capture/Replay: serialize context to JSON Bug: angleproject:5715 Change-Id: Ibc6897d7567fedab4a2ba7bf8d2bd5a7e496bd4b Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2760325 Reviewed-by: Jamie Madill <jmadill@chromium.org> Reviewed-by: Cody Northrop <cnorthrop@google.com> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Jamie Madill 9a025fd4 2021-03-13T22:48:13 Capture/Replay: add frontend feature to force capture limits Some of the limits set when FrameCapture is enabled are reflected in the serialized context. In order for capture/replay tests to pass these limits may also be needed to be enforced when replaying. Add a fronten feature enable_capture_limits for this and use it in the capture replay test script. Patch authored by gert.wollny@collabora.com. Bug: angleproject:5750 Change-Id: I3d333d22fe65c12dd5aa5f263d19a2c1568541c2 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2790301 Reviewed-by: Jamie Madill <jmadill@chromium.org> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Jamie Madill eac2397b 2021-03-25T13:36:50 Trace Tests: Work around autoninja.bat change on Win. A breaking change to autoninja.bat was causing the script to fail to find 'python3.bat' in this configuration. Work around the issue by instead invoking 'ninja.exe' directly with the correct '-j' argument. Bug: angleproject:5766 Change-Id: Ie01d7e8c4caf80661f420ccfc55a8d951a4e1b4f Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2787186 Reviewed-by: Cody Northrop <cnorthrop@google.com> Reviewed-by: Tim Van Patten <timvp@google.com> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Lubosz Sarnecki 36770173 2021-03-10T15:23:03 capture_replay_tests: Don't shadow subprocess import. Rename local variable, so using subprocess.Timeout does not result in an exception. This patch fixes the test timeout counter. Bug: angleproject:5742 Change-Id: Ib6991bc58df6ea653f765dfe9f6117add75b85db Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2748339 Commit-Queue: Lubosz Sarnecki <lubosz.sarnecki@collabora.com> Reviewed-by: Cody Northrop <cnorthrop@google.com> Reviewed-by: Jamie Madill <jmadill@chromium.org>
Lubosz Sarnecki 1330aa2c 2021-02-12T15:21:31 capture_replay_tests: Fix repository path retrieval. Use relation to retrieve angle repository path. The previous way of detecting the root repository folder failed to detect a correct path if the angle directory was in a path that contained the string angle. e.g. `/home/user/code/angle/angle` Other paths that would fail with the current implementation: `/home/user/my-angle-workspace/angle` `/home/tangled/workspace/angle/` `/home/user/code/angle-android` This way the path is constructed with the known relation of the script path inside the repository. Another advantage of this way is that the `angle` repository could be checked out with a different name. The disadvantage of this method is that whenever the script was moved, the relative path must be updated. Bug: angleproject:5645 Change-Id: I9a56aeba449b248273087382adceb0d6fba5a494 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2715323 Reviewed-by: Jamie Madill <jmadill@chromium.org> Reviewed-by: Cody Northrop <cnorthrop@google.com> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Jamie Madill f6925344 2021-01-25T13:13:33 Trace Tests: Update expectations. Bug: angleproject:5530 Change-Id: I48a6dc5753406e6dd6cd1845f1f28fcbfc485e34 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2647871 Reviewed-by: Cody Northrop <cnorthrop@google.com> Reviewed-by: Tim Van Patten <timvp@google.com> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Jamie Madill 3623868e 2021-01-22T12:24:50 Trace Tests: Add xvfb command line option. This will enable running the tests on Linux. Bug: angleproject:5530 Change-Id: I7cd2155fac89d395a8bebacb2f1a44be0e7d3739 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2644723 Reviewed-by: Cody Northrop <cnorthrop@google.com> Reviewed-by: Tim Van Patten <timvp@google.com> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Jamie Madill e60ce3da 2021-01-21T16:42:17 Capture/Replay: Allow testing with multiple versions. This embeds the context version and device type info into the replay. The self-tests then can create the correct display and context types. This fixes testing against SwiftShader which is necessary for the bots. Bug: angleproject:4759 Change-Id: If9da6bfdc1c2b315ccd7e453872fc84063277054 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2643363 Reviewed-by: Tim Van Patten <timvp@google.com> Reviewed-by: Cody Northrop <cnorthrop@google.com> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Jamie Madill b745f6df 2021-01-21T14:41:00 Capture/Replay tests: Fixes for goma & depot tools. Adds the depot tools path to be specified via the command line. This will allow the bots to specify working directory locations. Also adds quotes around the goma path in the gn args. Bug: angleproject:5530 Change-Id: I1291a8cdf4e36e55786403b4d9a6ce8565093352 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2642894 Reviewed-by: Cody Northrop <cnorthrop@google.com> Reviewed-by: Tim Van Patten <timvp@google.com> Reviewed-by: Jamie Madill <jmadill@chromium.org> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Jamie Madill 0ae61c25 2021-01-20T10:40:24 Capture/Replay tests: Use failure exit code. This will allow the test step to fail when run in CI. Bug: angleproject:5530 Change-Id: I7c156438ded82aa7191b8b597a9a52200c0efe57 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2638989 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Tim Van Patten <timvp@google.com> Reviewed-by: Cody Northrop <cnorthrop@google.com>
Jamie Madill 738cb55b 2021-01-14T17:33:20 Capture/Replay tests: Fixes and improvements. The script was broken after the change to build to a separate shared library. Also fixes issues on Windows and cleans up the code. State comparison is still broken on every test from what I can see. Bug: angleproject:5530 Change-Id: I23e77e56fadedfb85d7afdb59dddac5558a71a56 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2630232 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Tim Van Patten <timvp@google.com> Reviewed-by: Cody Northrop <cnorthrop@google.com>
Jamie Madill 39b777c6 2020-10-29T16:55:06 Capture/Replay: Two cleanups. This changes from returning a vector to directly returning a pointer to the binary data for the serialized state. The second cleanup is to use a ContextID as a wrapped type which simplifies the output formatting code. Bug: angleproject:5247 Change-Id: Ieb8afdb9326a12968dd2d69c05e1ed811b93abff Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2506198 Reviewed-by: Courtney Goeltzenleuchter <courtneygo@google.com> Reviewed-by: Cody Northrop <cnorthrop@google.com> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Jamie Madill 1c53df78 2020-10-26T13:42:14 Capture/Replay Tests: Expand suppressions. Updates the suppression list to include Linux tests and other new failures. Also sort the test list output to make it more readable. Bug: angleproject:5134 Change-Id: I38b5d95ceab3270d1cab13e3fb3c19d3137d10dc Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2497568 Reviewed-by: Cody Northrop <cnorthrop@google.com> Reviewed-by: Courtney Goeltzenleuchter <courtneygo@google.com> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Jamie Madill 5c4c37dc 2020-10-25T15:55:45 Capture/Replay: Test improvements. Streamlines some arguments. Use a common subdirectory in out/. Show process name when logging. Clean up environment variable handling. Bug: angleproject:5247 Change-Id: I85eb5b8194a64f90691c4d03db3ee65c5e9b2313 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2497560 Reviewed-by: Cody Northrop <cnorthrop@google.com> Reviewed-by: Courtney Goeltzenleuchter <courtneygo@google.com> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Jamie Madill 5ffd778e 2020-10-23T13:16:50 Make capture replay tests exectuable. This makes them runnable directly on Linux. Bug: angleproject:5134 Change-Id: Icf5ccf9d82b8cf09b16c5c60312e33752a9724e4 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2495288 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Cody Northrop <cnorthrop@google.com>
Jamie Madill be73728d 2020-10-03T17:52:25 Improve capture replay tests. Removes the custom logger class and uses the python logging class. Moves the argument descriptions into argparse help. Keep build directories around by default. Essential for normal use. Print more debug messages. Bug: angleproject:5133 Change-Id: Ida48828512abe60ee7d16873cbdb87c58529a60a Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2446898 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Cody Northrop <cnorthrop@google.com> Reviewed-by: Courtney Goeltzenleuchter <courtneygo@google.com>
Manh Nguyen c65517d6 2020-08-12T17:43:53 Add tests' expectations to capture replay tests Skips failing tests so that they do not affect the results of passing tests. Bug: angleproject:4817 Change-Id: I80678cb62b0b81d6a1aa811908019022efbe1268 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2351394 Reviewed-by: Cody Northrop <cnorthrop@google.com> Reviewed-by: Jamie Madill <jmadill@chromium.org> Commit-Queue: Manh Nguyen <nguyenmh@google.com>
Manh Nguyen ce5943c5 2020-08-05T23:24:17 Change test result types + add additional logging New result types: Passed, Failed, TimedOut, Crashed, CompileFailed, Skipped. They divide the results of tests more clearly. Results are now immediately logged to the stdout once a test batch completes. Flags, with the exception of --gtest_filter now use "-" instead of "_". Bug: angleproject:4817 Change-Id: I9616b23ef70026818e22126a251e1e2341de641d Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2340787 Commit-Queue: Manh Nguyen <nguyenmh@google.com> Reviewed-by: Cody Northrop <cnorthrop@google.com> Reviewed-by: Jamie Madill <jmadill@chromium.org>
Manh Nguyen 8956bfb9 2020-08-05T13:34:01 Use striping when making test batches When grouping tests to test batches, instead of grouping neighboring tests together, use striping so that slow tests are fanned out into multiple batches. This way, no test batch is too slow. Bug: angleproject:4817 Change-Id: I7825539e401ea53b5a31167f206d1f5bfb07bdbf Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2339540 Reviewed-by: Jamie Madill <jmadill@chromium.org> Reviewed-by: Cody Northrop <cnorthrop@google.com> Commit-Queue: Manh Nguyen <nguyenmh@google.com>
Manh Nguyen 6aed2832 2020-07-31T10:52:21 Batch capture run + change how results are logged Before, the run stages of tests in a batch were run separately by multiple subprocesses. Now the run stages of tests in a batch are batched together and run by a single subprocess. Changes how results are logged. Tests in a batch are logged together. Within a batch, tests that fail at the same stage are also logged together. Bug: angleproject:4817 Change-Id: Ie3f992c081de914f1f1f521bec2d72f06ccca238 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2331738 Commit-Queue: Manh Nguyen <nguyenmh@google.com> Reviewed-by: Jamie Madill <jmadill@chromium.org> Reviewed-by: Cody Northrop <cnorthrop@google.com>
Manh Nguyen 2c41931c 2020-07-21T14:21:27 Batch-compile and batch-run-replay multiple tests Multiple tests are batch-compiled into 1 replay application instead of multiple replay applications. Replay application now runs generated code of multiple tests instead of 1 test. This reduces overhead cost and brings down runtime. Main process now receives messages sent by workers via a message queue and prints them to the main stdout so that user can know if workers are hanging. Add handle for user interrupt (Ctrl-C) so that processes are properly destroyed and cleaned up. Trace files now have the option not to be deleted. Bug: angleproject:4817 Change-Id: Ic90ae0f430e1d3c261ffea5f963be5a4e94b0ad2 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2310909 Reviewed-by: Cody Northrop <cnorthrop@google.com> Reviewed-by: Jamie Madill <jmadill@chromium.org> Commit-Queue: Manh Nguyen <nguyenmh@google.com>
Manh Nguyen 3b0c9918 2020-07-21T11:52:13 Update CaptureAndReplay.md and Python script with links Links to the Python script in CaptureAndReplay.md Links to CaptureAndReplay.md's testing section in the Python script Bug: angleproject:4817 Change-Id: I584f8508661950e4907dae1adb8f0fe300387198 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2309226 Commit-Queue: Manh Nguyen <nguyenmh@google.com> Reviewed-by: Cody Northrop <cnorthrop@google.com> Reviewed-by: Jamie Madill <jmadill@chromium.org>
Manh Nguyen 10e9e9a1 2020-07-13T17:02:38 Apply multiprocessing to capture_replay_tests.py Multiple tests are run in paralell to squeeze the most out of CPUs. Tests are stored in a work queue. Whenever a CPU becomes available, the next unrun test is grabbed and run on a spawn process on that CPU. Each cpu gets their own environment and build folders and trace folders so that tests don't overwrite each other. Bug: angleproject:4817 Change-Id: Ifd35c9c75522e480b0257d090d5af70f2a3428ba Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2296040 Commit-Queue: Manh Nguyen <nguyenmh@google.com> Reviewed-by: Cody Northrop <cnorthrop@google.com> Reviewed-by: Jamie Madill <jmadill@chromium.org>
Manh Nguyen e5c18475 2020-07-06T09:50:49 Multi-frames serialization + comparision for capture replay test Changes how context states are serialized during capture run. Now, context states are serialized after each frame until the end frame is reached, or context is destroyed. Adds an api to the trace files to query serialization results of multiple frames. Makes change to CaptureReplayTest to serialize multiple frames then compares the serialization results with the ones retrieved from said api. Adds to capture replay the ability to produce working, compilable trace files even when end frame is not reached. Adds to the generated trace files config information and makes CaptureReplayTest utilize said config information to reproduce the exact environment of the captured application. Bug: angleproject:4817 Change-Id: Ie3d487af2bacf349dc3ff6f6b1b5f89e1169dc84 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2282885 Commit-Queue: Manh Nguyen <nguyenmh@google.com> Reviewed-by: Cody Northrop <cnorthrop@google.com> Reviewed-by: Jamie Madill <jmadill@chromium.org>
Manh Nguyen dce37b7d 2020-06-21T22:27:06 Serialize framebuffers + compare contexts for CaptureReplayTests Adds to frame capture the ability to serialize a frame's pre-swap GL state and store it in the binary data file Adds to CaptureReplayTests the ability to compare its serialized GL state with the serialized state pulled from the binary data file Adds a serialization module that serializes framebuffers' GL states and the contents of their color attachments Adds checks to automation script so that it would skips tests that do not produce the expected trace files Adds exception handling to automation script so that it will not crash when a replay build crashes Bug: angleproject:4779 Change-Id: I40a02e018073749e79f0ddbfd3d4065745548f46 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2258295 Commit-Queue: Manh Nguyen <nguyenmh@google.com> Reviewed-by: Jamie Madill <jmadill@chromium.org> Reviewed-by: Tim Van Patten <timvp@google.com>
Manh Nguyen 9dee8e3f 2020-06-15T10:59:18 Automate testing of capture_replay automation script will: 1. compile all tests in angle_end2end with framecapture enabled 2. run each test with frame capture 3. compile CaptureReplayTest with cpp trace files 4. Run CaptureReplayTest Test: run_tests_with_framecapture.py Bug: angleproject:4749 Change-Id: Ieb21c6137653632d73beedea48712e46fc15a0f3 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2246321 Reviewed-by: Jamie Madill <jmadill@chromium.org> Reviewed-by: Cody Northrop <cnorthrop@google.com> Commit-Queue: Manh Nguyen <nguyenmh@google.com>
kmx.io     mail     discord kmxgit v0.4.0