src/tests/restricted_traces/retrace_restricted_traces.py


Log

Author Commit Date CI Message
Cody Northrop 645c192a 2024-12-17T09:45:41 retrace: Fix assumption in get_min_reqs Only try to remove implied extension if it is present. Test: retrace_restricted_traces.py get_min_reqs --traces riptide_gp2 Bug: b/384714388 Change-Id: Ic0240c387012923e1918fccd2ea6c341d9a84609 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/6101167 Reviewed-by: Roman Lavrov <romanl@google.com> Commit-Queue: Mark Łobodziński <mark@lunarg.com>
Mark Lobodzinski 99aac4b9 2024-11-20T09:24:07 Capture/Replay: Remove implied exts from RequiredExtensions Support for the EXT_compressed_ETC1_RGB8_sub_texture extension can be implied from the presence of OES_cmpressed_ETC1_RGB8_texture, so remove it from the trace's RequiredExtensions entry in its json file. Test: angle_trace_tests --gtest_filter=*need_for_speed_no_limits* Bug: b/380026310 Change-Id: Ia80ae4d78c1f7f7f246ce7a5cfd00bfc0e9734a4 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/6037487 Reviewed-by: Cody Northrop <cnorthrop@google.com> Commit-Queue: Mark Łobodziński <mark@lunarg.com>
Mark Lobodzinski 0ec8a7f1 2024-09-26T11:06:25 Prevent multiple solutions when retracing with get_min_reqs Skip extensions that implicitly enable other extensions in the retracer script when running to determine minimum requirements. Test: Retrace cod_mobile with get_min_reqs Bug: b/42266279 Change-Id: Iee6f629d1b338362809ff8880d8a5d9ed7c574de Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/5893959 Reviewed-by: Cody Northrop <cnorthrop@google.com> Commit-Queue: Mark Łobodziński <mark@lunarg.com>
Mark Lobodzinski 04c2879e 2024-09-18T11:30:07 Modify retracer script to output min requirements json diff After the ANGLE retracer script runs with the get_min_reqs option, ouptut the json file diff. Bug: b/42266279 Change-Id: Ia744e4e36fb9f8b7e8d498d44412af733c7ed9b1 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/5874161 Commit-Queue: Cody Northrop <cnorthrop@google.com> Reviewed-by: Cody Northrop <cnorthrop@google.com>
Mark Lobodzinski 27738fe4 2024-09-04T19:53:01 Fix retracer script error due to blank argument Traces fail when running the retracer script if the --request-extensions argument is passed with a blank value. Bug: b/42266279 Change-Id: I1e465caaaa7a5f8d8048c09890b5d01eeb2d0df1 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/5860298 Commit-Queue: Mark Łobodziński <mark@lunarg.com> Reviewed-by: Cody Northrop <cnorthrop@google.com>
Shahbaz Youssefi d193d51b 2024-06-17T22:46:08 Replace issue ids post migration to new issue tracker This change replaces anglebug.com/NNNN links. Bug: None Change-Id: I8ac3aec8d2a8a844b3d7b99fc0a6b2be8da31761 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/5637912 Reviewed-by: Geoff Lang <geofflang@chromium.org> Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org>
Mark Lobodzinski d85de0e9 2023-08-09T14:15:56 Capture/Replay: Add optional replay of trimmed resources This CL: - Adds '--include-inactive-resources' option to angle_trace_tests - Removes the 'trim-enabled' option - Outputs all previously trimmed shaders/programs to trace file in a new Setup function, SetupReplayContextSharedInactive() which is executed only if the new option is specified - Modifies CaptureTest to add inactive resources, but does not set the include-inactive-resources flag Bug: b/296055694 Change-Id: I33b18d5da727d55c90c2012c2bf64b1413521429 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4781552 Reviewed-by: Cody Northrop <cnorthrop@google.com> Commit-Queue: Mark Łobodziński <mark@lunarg.com> Reviewed-by: Roman Lavrov <romanl@google.com>
Roman Lavrov 124e90af 2023-08-17T13:07:46 Retrace: copy get_min_reqs data over when upgrading Bug: angleproject:7639 Change-Id: I9a6e3a5ca75d6dcd5b537123948d2498e6bb47f6 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4789844 Reviewed-by: Cody Northrop <cnorthrop@google.com> Commit-Queue: Roman Lavrov <romanl@google.com>
Roman Lavrov 4e073356 2023-04-13T15:16:53 Trace Interpreter: gzip cpp and run gold tests. New test step: angle_restricted_trace_gold_interpreted_tests similar to existing angle_restricted_trace_gold_tests but runs a subset of newest traces that have the right format. Currently there are 8 of them and the filter is hard-coded in test_suites.pyl. rise_of_empires can also be run interpreted but happened to flake during testing so it's not included for now. Trace cpp files are concatenated and gzipped as a build step producing gen/tracegz_{trace}.gz and these gz files are then distributed as part of the build --trace-interpreter flag changes from boolean to string to support: --trace-interpreter=gz: * this uses the gz file above * gold tests added in this CL use this option --trace-interpreter=c: * using uncompressed c/cpp file * existing angle_trace_interpreter_tests runs a retrace saving .c files and uses this option Bug: b/276742336 Change-Id: I69544f25bda873af191978195d02ffbdd34363c0 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4424690 Reviewed-by: Cody Northrop <cnorthrop@google.com> Commit-Queue: Roman Lavrov <romanl@google.com>
Jamie Madill 2c4de3a7 2023-01-03T16:07:35 Trace Interpreter: Support Genshin Impact. Includes a couple fixes: - parsing hexidecimal values instead of enums - support sharing strings and functions between modules - support 32-bit int pointers Bug: angleproject:7887 Change-Id: I6e20a64a862c45c17ccde78a58d6069d83b31867 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4135797 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Cody Northrop <cnorthrop@google.com>
Jamie Madill 71e89320 2023-01-03T09:10:08 Tests: Support multiple filters with ":". Now we can run multiple traces per invocation, e.g.: "trex_200:among_us" Bug: angleproject:7775 Change-Id: I8e8176d444f20ae97234469ab8c553543b81b331 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4132944 Auto-Submit: Jamie Madill <jmadill@chromium.org> Reviewed-by: Roman Lavrov <romanl@google.com> Commit-Queue: Roman Lavrov <romanl@google.com> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Gert Wollny 98469694 2022-11-02T12:13:35 retrace: better error reporting For a resonable error reporting the output must be captured. Print the output on error only, if it is not written anyway. Bug: angleproject:7800 Change-Id: Ie0ca819b7e269317073f8f1c79819e35a33a4325 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4003220 Reviewed-by: Jamie Madill <jmadill@chromium.org> Reviewed-by: Cody Northrop <cnorthrop@google.com> Commit-Queue: Gert Wollny <gert.wollny@collabora.com>
Gert Wollny e3d0bee6 2022-11-01T13:54:16 retrace: Check for the output json file after upgrade To check the success of writing a new trace we have to look in the output directory, otherwise even skipped traces will report success. Bug: angleproject:7800 Change-Id: Icd2d79c3ebe55cd02f55635b1ccf48d19f46acea Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3996704 Reviewed-by: Jamie Madill <jmadill@chromium.org> Commit-Queue: Gert Wollny <gert.wollny@collabora.com>
Sean Risser f3e047b5 2022-10-12T16:28:32 Add new retrace command get_min_reqs `retrace_restricted_traces.py get_min_reqs` finds the minimum required GLES version and extensions and record them to a trace's json file. TracePerfTest has also been modified to use the new "RequiredExtensions" json field to determine if it should run a test or not. This allows several lines to be removed from TracePerfTest's constructor. get_min_reqs uses a divide-and-conquer strategy to find the extensions. This approach should be faster for traces where the number of required extensions is less than 20. This adds a new command to retrace_restricted_traces.py called get_min_reqs. This runs an existing trace multiple times to find what the minimum required GLES version is, then records that version to the trace's json file. Bug: angleproject:7639 Change-Id: I42530e438054efec1ad79ababebe06d221d5c695 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3884175 Reviewed-by: Geoff Lang <geofflang@chromium.org> Reviewed-by: Cody Northrop <cnorthrop@google.com> Commit-Queue: Sean Risser <srisser@google.com>
Gert Wollny 9737c632 2022-10-24T11:34:51 retrace: Add --show-test-stdout arg to validation This argument is checked when running validation, so we have to define it. Bug: angleproject:7800 Change-Id: Ia24a8c8a82a8bdcefcf707bd9c9183ead4ee095d Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3990622 Reviewed-by: Jamie Madill <jmadill@chromium.org> Commit-Queue: Gert Wollny <gert.wollny@collabora.com>
Gert Wollny 822471f0 2022-10-24T08:08:03 retrace: Add a catch-all clause to report all errors With that running the a batch of tests from an external script can bail out early when a trace fails. Bug: angleproject:7800 Change-Id: I5041e863a8cdc212b7057b88e3f0fd68a5115470 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3990621 Commit-Queue: Gert Wollny <gert.wollny@collabora.com> Reviewed-by: Jamie Madill <jmadill@chromium.org>
Jamie Madill dc62b3ee 2022-10-10T21:00:16 Capture/Replay: Add trace interpreter. Also adds a self-test using the retrace script. Bug: angleproject:7752 Change-Id: I1985b47250bef99726d2ca2d90bef859208e357e Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3965128 Reviewed-by: Cody Northrop <cnorthrop@google.com> Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Yuxin Hu <yuxinhu@google.com>
Jamie Madill e0d2c7f2 2022-10-18T15:07:25 retrace script: Update trace test suite name. Bug: angleproject:7755 Change-Id: I1322543b199aacb4f083b67f0bc4f8bf93ea65b3 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3963359 Reviewed-by: Cody Northrop <cnorthrop@google.com> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Jamie Madill f8d393da 2022-10-05T10:09:34 Capture/Replay: Make retrace restore touch output files. This should force a recompile of modified files. Bug: angleproject:7731 Change-Id: I52c607fdda97c09e557a9ac3144fa67de3a17ee7 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3937048 Reviewed-by: Cody Northrop <cnorthrop@google.com> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Sean Risser 19117567 2022-09-07T19:21:56 Avoid general python exceptions in retrace script Keyboard interrupts are handled as exceptions in Python, so unguarded except statements can prevent a program from properly terminating. Bug: angleproject:7639 Change-Id: I54ee754a40457b924eda0a6c8074f575263528ad Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3856650 Reviewed-by: Cody Northrop <cnorthrop@google.com> Commit-Queue: Sean Risser <srisser@google.com>
Lubosz Sarnecki b3fff1aa 2022-03-31T16:03:11 retrace_restricted_traces: Check for json file. Validate trace upgrades by looking for existing json file instead context header, which was incorrect when traces did not contain a context == 1. Introduce get_trace_json_path function. Use python fstrings. Bug: angleproject:7107 Change-Id: Ib83acb646fb2d9c38d8ca99766170e6c9c7eeb06 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3571884 Reviewed-by: Jamie Madill <jmadill@chromium.org> Reviewed-by: Cody Northrop <cnorthrop@google.com> Commit-Queue: Lubosz Sarnecki <lubosz.sarnecki@collabora.com>
Lubosz Sarnecki e933df51 2022-03-31T15:12:43 retrace_restricted_traces: Remove unused code. Remove unused load_json_metadata, get_trace_metadata and replace_metadata functions. Remove unused re import and trace_path parameter from get_context. Bug: angleproject:7107 Change-Id: I203609ca45145cdced8295d5edf7faa0c8606ae6 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3571883 Reviewed-by: Jamie Madill <jmadill@chromium.org> Commit-Queue: Lubosz Sarnecki <lubosz.sarnecki@collabora.com>
Jamie Madill 9aca4285 2021-12-30T09:18:32 Traces: Fix path in retrace script. This was accidentally regressed when removing the auto-generation. Bug: angleproject:5133 Change-Id: I44e88bd15c9baaca66c49b1e379b2e0dd7206b3f Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3362295 Reviewed-by: Tim Van Patten <timvp@google.com> Reviewed-by: Jamie Madill <jmadill@google.com> Commit-Queue: Jamie Madill <jmadill@google.com>
Jamie Madill 4d893a93 2021-09-07T11:47:23 Re-land: "Trace Tests: Remove auto-generated code." No significant changes in the re-land. Bug: angleproject:5133 Change-Id: Ib5bb8fba3e9e2f1f648d5af251b7b2319c9f37dd Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3352432 Reviewed-by: Cody Northrop <cnorthrop@google.com> Commit-Queue: Cody Northrop <cnorthrop@google.com>
Cody Northrop 709d8727 2021-12-14T16:11:33 Revert "Trace Tests: Remove auto-generated code." This reverts commit 1c0bb44862fbc6107d73d056c2e07c59bcfa8d09. Reason for revert: Causing failures on Intel Linux with native driver. Original change's description: > Trace Tests: Remove auto-generated code. > > Bug: angleproject:5133 > Change-Id: I71a82071b1c667475f7fefcd6c8766a9de71141d > Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3144215 > Reviewed-by: Cody Northrop <cnorthrop@google.com> > Reviewed-by: Tim Van Patten <timvp@google.com> > Commit-Queue: Jamie Madill <jmadill@chromium.org> Bug: angleproject:6816 Change-Id: Ic35145c8c3bb23ccf196b86def42f4a82f07aad5 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3340332 Commit-Queue: Cody Northrop <cnorthrop@google.com> Reviewed-by: Tim Van Patten <timvp@google.com> Commit-Queue: Tim Van Patten <timvp@google.com>
Jamie Madill 1c0bb448 2021-09-07T11:47:23 Trace Tests: Remove auto-generated code. Bug: angleproject:5133 Change-Id: I71a82071b1c667475f7fefcd6c8766a9de71141d Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3144215 Reviewed-by: Cody Northrop <cnorthrop@google.com> Reviewed-by: Tim Van Patten <timvp@google.com> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Jamie Madill 7fc6c07c 2021-09-29T14:02:29 Capture/Replay: Update process for trace upgrading. Includes changes to the retracing script. Also includes documentation on how the process works. Bug: angleproject:5133 Change-Id: I1acfe338f3fe0282a0461c314274c761ed04bd2f Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3193418 Reviewed-by: Cody Northrop <cnorthrop@google.com> Reviewed-by: Tim Van Patten <timvp@google.com> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Jamie Madill b930b9d3 2021-09-08T09:08:28 Capture/Replay: Remove redundant "capture" from label old: trex_200_capture_context1_frame001.cpp new: trex_200_context1_frame001.cpp The 'capture' word wasn't really adding any useful information. Take it out so the filenames are a bit shorter. Bug: angleproject:5133 Change-Id: Ibd7d47df2aa1f24a641c5d5b2a6dd14d6ce240e1 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3149370 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Tim Van Patten <timvp@google.com> Reviewed-by: Cody Northrop <cnorthrop@google.com>
Jamie Madill c5a38976 2021-09-01T07:35:40 Capture/Replay: Add expression trigger validation calls. Setting the environment variable "ANGLE_CAPTURE_VALIDATION_EXPR" will make ANGLE's capture logic evaluate this expression every captured call to see if it should insert a validation checkpoint. The retracing script also accepts --validation-expr as an argument. For instance, the expression: ((frame == 2) && (call < 1189) && (call > 1100) && ((call % 5) == 0)) Will insert validation checkpoints on frame 2, between calls 1100 and 1189 and will validate every 5th call. The 'call' here is the count of captured calls, which are mostly GL calls with a few ANGLE replay calls in the mix. We add a small single-header library that can evaluate arthithmetic expressions in order to parse these expressions, as well as an option to the retracing script. Bug: angleproject:5133 Change-Id: Ic369e85d8e905a3a7a32fa098f7d8ebe7baf4ab9 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3136094 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Tim Van Patten <timvp@google.com> Reviewed-by: Cody Northrop <cnorthrop@google.com>
Jamie Madill 9c05f55a 2021-07-15T10:58:10 Capture/Replay: Add replay state validation. We can use this to validate traces that have slight differences when retracing. The valdation works by embedding the "expected" JSON into a string at the end of each captured frame. The replay also embeds a callback which fires right before the swap in the replay harness. The harness then gets the "actual" JSON and runs a comparison. On a mismatch it calls "diff" externally. Currently the diff call is hard-coded to work on Linux only. Note that when running validation it's important to replay on SwiftShader since that's what we use to retrace. Bug: angleproject:5133 Change-Id: Icbf0031d07be8bd916607c537dec235f9a512c43 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3066008 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Tim Van Patten <timvp@google.com> Reviewed-by: Cody Northrop <cnorthrop@google.com>
Jamie Madill 839d5318 2021-08-16T08:29:28 Capture/Replay: Changes to MEC first frame capture. Recent changes made it so it was impossible to trigger MEC *after* the first frame and *before* the second frame. Instead it was only possible to capture *before* the first frame or *after* the second frame. Makes it possible for the retracing script to do identical captures. Also includes a number of refactorings to make it simpler to follow the code, including renaming methods, and removing some extra output files. Bug: angleproject:5133 Change-Id: Ice6a189eb9f4d53e8ee1ba39beb537af2ef5fd9f Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3097807 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Tim Van Patten <timvp@google.com> Reviewed-by: Cody Northrop <cnorthrop@google.com>
Jamie Madill 3a6e22d5 2021-07-09T12:09:53 Trace Tests: Update retracing script. Adds an option to skip retracing traces that already exist. That makes the script much easier to use on folders that already have traces partially retraced. It also catches exceptions when running the traces so that ANGLE crashes and ASSERTions don't need you to restart the retrace. Bug: angleproject:5133 Change-Id: I059ca545145eadc1aa8c373a2fde762bbeb52b55 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3018302 Reviewed-by: Cody Northrop <cnorthrop@google.com> Reviewed-by: Tim Van Patten <timvp@google.com> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Jamie Madill 78dde332 2021-03-29T17:31:52 Move restricted traces to CIPD. All traces are now stored as DEPS entries in CIPD. The auto-generation script generates the DEPS entries. Note that we don't include DEPS in the list of generated outputs to simplify other rollers. Also we update auto-generation to include full sources list to allow 'gn analyze' to work successfully. Usees a trace fixture for common code. This will enable a more compact trace without as much repeated code. We must land a set of re-trace traces to avoid breakage. Also includes a python script for uploading new traces to CIPD. The script first checks if traces are already present in the cloud, and if so it skips the upload. It will take a while to complete as the number of traces grows larger as it takes a few seconds per trace. The traces in this patch are also re-traced to use the common fixture code instead of including duplicated code in each trace. They now form a simple common interface and the autogenerated cpp is now simply a list of properties. I've also updated the capture/replay tests to use the simpler common entry point integration. There is less auto-generated glue code now. We now use a new serialized string query extension instead of calling directly into ANGLE's internals. Also includes a docs update. The capture/replay sample is broken and we'll need to update it in a follow-up CL. Also includes a few necessary fixes to the retracing script. Bug: angleproject:5811 Change-Id: I977bc6dc56843c8966377fc445ae97e91e17319a Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2797833 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Cody Northrop <cnorthrop@google.com> Reviewed-by: Tim Van Patten <timvp@google.com>
Jamie Madill 3fe652d0 2020-10-29T08:54:57 Add option to disable SwiftShader when retracing. Bug: angleproject:5133 Change-Id: I589252d7eb9648a37a98f896c2a14462b2b34c48 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2506492 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Cody Northrop <cnorthrop@google.com> Reviewed-by: Courtney Goeltzenleuchter <courtneygo@google.com>
Jamie Madill 77defeb0 2020-10-27T18:31:53 Add the ability to re-trace existing traces. This new script runs the ANGLE trace tests with capture enabled to generate updated replay cpp files. This allows us to update our traces files to a new file format in one step. Trace metadata (currently only the default FBO format) is preserved between re-traces. Currently only desktop-based retracing is supported. This means a couple traces that require specific extensions don't run on all platforms. Bug: angleproject:5134 Change-Id: I7c923d89e33c18285ab36a7cee91f2fb735758eb Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2488130 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Courtney Goeltzenleuchter <courtneygo@google.com> Reviewed-by: Cody Northrop <cnorthrop@google.com>