src/tests/capture_replay_tests/CaptureReplayTests.cpp


Log

Author Commit Date CI Message
Jamie Madill 11f8f537 2022-10-17T16:59:50 Capture/Replay: Improvements to self-test. - print errors in the replay test - catch missing serialization - clean stale json output files before each replay - add flag to show replay stdout - fix help message Bug: angleproject:7731 Change-Id: Ibeb780f417bdbed4b249038920963dd1bd68f4aa Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3962248 Reviewed-by: Yuxin Hu <yuxinhu@google.com> Reviewed-by: Cody Northrop <cnorthrop@google.com> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Jamie Madill 5b3781ec 2022-10-03T16:09:35 Remove namespacing from all ANGLE loaders. This will make it easier to work with pure C files. Bug: angleproject:7731 Change-Id: I2fe9af486af5f339d973c9149f082eb1f2efa8c4 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3925426 Reviewed-by: Cody Northrop <cnorthrop@google.com> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Roman Lavrov 4b33633a 2022-08-26T16:16:50 Capture/Replay: Delete binary data in FinishReplay via callback Delete callback fixes b/179188489 as delete[] will get called from the same module where allocation happens. When decompress/delete callbacks not provided, new[] and delete[] are called from the fixture. AFAICT, gBinaryData == nullptr every time, even with a single test and --gtest_repeat=N, so we never delete the uncompressed data. When running multiple tests locally, I see RES usage grow significantly on Linux which this CL fixes. I don't think this matters for bot runs as we run tests one by one there. Bug: b/179188489 Change-Id: Iee3bccb9d6d72c315c4358b70ec9c3c0ac963258 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3858185 Reviewed-by: Jamie Madill <jmadill@chromium.org> Reviewed-by: Cody Northrop <cnorthrop@google.com> Commit-Queue: Roman Lavrov <romanl@google.com>
Geoff Lang 785353fd 2022-05-24T12:40:16 Support Desktop OpenGL context creation in end2end tests Validation of Desktop GL versions and profile masks is unimplemented. Bug: angleproject:7360 Change-Id: Ifae94215b6aada895c2b02318a1d05c9515e9b96 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3664916 Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org> Commit-Queue: Geoff Lang <geofflang@chromium.org> Reviewed-by: Cody Northrop <cnorthrop@google.com>
Gert Wollny fd9301c1 2022-06-17T12:25:10 Capture/Replay: Capture egl surface related calls Creation from a pbuffer and generalized destruction are implemented, as well as binding and releasing a TexImage and making the context current with explicit draw and read surfaces given. Bug: angleproject:4964 Bug: angleproject:6180 Bug: angleproject:6512 Change-Id: Id8be6486125d45341905f3aabdbae4366cd568b7 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3711741 Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org> Commit-Queue: Gert Wollny <gert.wollny@collabora.com> Reviewed-by: Cody Northrop <cnorthrop@google.com>
Gert Wollny a8152447 2022-05-16T12:02:25 Capture/Replay: Override eglCreateImage and eglDestroyImage Bug: angleproject:4964 Change-Id: I1f67e0e32fa901249aa9ee134854afbe1ac29524 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3644579 Reviewed-by: Cody Northrop <cnorthrop@google.com> Reviewed-by: Jamie Madill <jmadill@chromium.org> Commit-Queue: Gert Wollny <gert.wollny@collabora.com>
Jamie Madill 3da24b5c 2022-05-16T11:58:59 Move trace loader code to a common location. With that Capture/Replay will be able to make use of the same code. CL authored by gert.wollny@collabora.com. Bug: angleproject:4964 Change-Id: Ie1ba663169a34929f5a169ab74c7042e9fd5deb0 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3645441 Reviewed-by: Cody Northrop <cnorthrop@google.com> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Shahbaz Youssefi 797e627e 2022-04-08T22:49:51 Autogenerate list of features as enum The WithX() and WithNoX() helpers are removed and replaced with enable() and disable() member functions that take the name of the feature (as a Feature::X enum constant). This has two benefits: - Adding tests that override a feature no longer requires additional helper functions to be written. - There's no mistaking the feature name. This change doesn't yet fix the main issue in anglebug.com/6435, but does fix the following helpers using an old feature name (so they were ineffective): - WithMetalForcedBufferGPUStorage - WithNoVulkanViewportFlip A follow up would remove the old way of overriding features in tests and replaces them with the new way. Bug: angleproject:6435 Change-Id: Ida02b26ec72bc40d7a8938c76a93815bb903ca05 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3580982 Reviewed-by: Jamie Madill <jmadill@chromium.org> Reviewed-by: Yuly Novikov <ynovikov@chromium.org> Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org>
Gert Wollny 29c01d51 2021-09-20T20:49:29 Capture/Replay: Write the context serialization file correctly This is what was done before we switched to not changing the working directory, and what is expected by the test script. In addition, the captured contexts were also written to the wrong files, i.e. the replay context to the CapturedContext file, and the captured context to file ReplayedContext file, so fix that too. Bug: angleproject:6412 Change-Id: I81906e3f64b2caf0002ca6a28cfce2a3d552fc8f Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3171354 Reviewed-by: Cody Northrop <cnorthrop@google.com> Reviewed-by: Jamie Madill <jmadill@chromium.org> Commit-Queue: Gert Wollny <gert.wollny@collabora.com>
Gert Wollny 9121a79d 2021-09-14T11:57:32 Capture/Replay: Don't set CWD and pass path information instead With that we avoid complication when evaluating the module path that is used to load the test modules. Fixes: 2f3e4db838068584a0c15653cb07893683a55300 Capture/Replay: Serialize trace metadata to a JSON file. Bug: angleproject:6385 Change-Id: If4e348237d9b0e99e404f49b28dddba1ac50ef92 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3157907 Reviewed-by: Cody Northrop <cnorthrop@google.com> Reviewed-by: Jamie Madill <jmadill@chromium.org> Commit-Queue: Gert Wollny <gert.wollny@collabora.com>
Jamie Madill 2f3e4db8 2021-09-02T10:31:13 Capture/Replay: Serialize trace metadata to a JSON file. This will allow us to remove more code auto-generation for the trace tests. The trace info now can be loaded directly from JSON instead of from the autogenerated information. Bug: angleproject:5133 Change-Id: I04e22b9279b19282df274bc8defcd363d0449111 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3140218 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Tim Van Patten <timvp@google.com> Reviewed-by: Cody Northrop <cnorthrop@google.com>
Jamie Madill b6e99bb3 2021-09-02T13:47:34 Rename and expand shader variable init feature. This feature now forces both output and unitialized locals to be initialized by the shader translator. This feature is needed by the trace validator to ensure we get deterministic behaviour in traces that exhibit some undefined results. Bug: angleproject:5133 Change-Id: Id1242cd077a57e891eed217f7671976ce1631a58 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3140216 Reviewed-by: Cody Northrop <cnorthrop@google.com> Reviewed-by: Tim Van Patten <timvp@google.com> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Jamie Madill 2f45d93d 2021-08-18T16:58:44 Capture/Replay: Init shader outputs during self-tests. This forces all uninitialized variables to have default values. For instance if the application doesn't initialize the output color, or a varying that's use in the output, this will ensure we don't use any undefined values in the computation. Found when working on a re-trace of T-Rex, which doesn't write to the alpha channel in the final rendering pass. Also fixes undefined values in GLSLTest.InactiveVaryingInVertexActiveInFragment. Bug: angleproject:5133 Change-Id: Ia291338e5adf23dab5263cb2ebe737dc05852d3e Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3110225 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Cody Northrop <cnorthrop@google.com> Reviewed-by: Tim Van Patten <timvp@google.com>
Gert Wollny d67cab22 2021-07-12T13:48:22 Capture/Replay: Handle missing context serialization Some ContextLost* tests do weird things with the context serialization which results in a mis-match of available context serializations. In this case check whether both serializations are missing, if so consider this to be an equal contexts states, otherwise, if only one serialization is available, report failure. While touching this code, also remove an unused parameter from the context serialization compare function. Disable tests that still fail with this change. Bug: angleproject:6157 Change-Id: Iee6cc7d65437112e38919cff59804e2b7f9467da Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3021061 Commit-Queue: Gert Wollny <gert.wollny@collabora.com> Reviewed-by: Jamie Madill <jmadill@chromium.org> Reviewed-by: Cody Northrop <cnorthrop@google.com>
Jamie Madill 0a83d958 2021-07-19T09:52:13 Capture/Replay tests: Fix missing usage of ApplicationDir. This was changed to ModuleDir in http://crrev.com/c/3035444 Fixed the capture/replay self-tests. Bug: chromium:1219969 Change-Id: Iaada421e9748db1e8afe17acf0705d5a89db6edb Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3035248 Reviewed-by: Jamie Madill <jmadill@chromium.org> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Gert Wollny 31cd0e98 2021-05-10T10:37:54 Capture/Replay: track robustResourceInit Bug: angleproject:5938 Change-Id: I7c9ab9bebd1ec2dc572a071844a860f99d0befe6 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2882580 Reviewed-by: Cody Northrop <cnorthrop@google.com> Reviewed-by: Jamie Madill <jmadill@chromium.org> Commit-Queue: Cody Northrop <cnorthrop@google.com>
Gert Wollny ee2f3302 2021-05-12T12:07:42 Capture/Replay: recreate the EGL window if contexts don't match In addition enable affected tests. Bug: angleproject:5955 Change-Id: I4fb72e38a633daa4565d3c0ae7cddb6ace7226f1 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2891775 Commit-Queue: Gert Wollny <gert.wollny@collabora.com> Reviewed-by: Jamie Madill <jmadill@chromium.org> Reviewed-by: Cody Northrop <cnorthrop@google.com>
Gert Wollny 2ba3763e 2021-05-10T10:08:27 Capture/Replay: reset CWD after the test ends As suggested by Jamie Bug: angleproject:5849 Change-Id: I3e548935a9119b7ca7ca392c50bcb0f58ae3e974 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2882578 Commit-Queue: Gert Wollny <gert.wollny@collabora.com> Reviewed-by: Jamie Madill <jmadill@chromium.org>
Gert Wollny 39cef942 2021-04-27T08:28:13 Capture/Replay: capture WebGL compatibility With that we can also enable the tests DifferentStencilMasksTest.* Bug: angleproject:5851 Change-Id: If2c992e456716f606320e3992f389823d0af050f Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2853584 Commit-Queue: Gert Wollny <gert.wollny@collabora.com> Reviewed-by: Cody Northrop <cnorthrop@google.com> Reviewed-by: Jamie Madill <jmadill@chromium.org>
Gert Wollny 3e1af85a 2021-04-19T18:12:30 Capture/Replay: Handle bindGeneratesResource in trace meta data Bug: angleproject:5844 Change-Id: I0243cab70b5833efd83b0d75a1801f68025ee291 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2835889 Commit-Queue: Gert Wollny <gert.wollny@collabora.com> Reviewed-by: Jamie Madill <jmadill@chromium.org> Reviewed-by: Cody Northrop <cnorthrop@google.com>
Gert Wollny c1da2794 2021-04-19T11:49:41 Capture/Replay: Handle ClientArraysEnabled in trace meta data Bug: angleproject:5848 Change-Id: Ieba325e2b5efa3242b24cafa8363e562c258546c Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2835888 Commit-Queue: Gert Wollny <gert.wollny@collabora.com> Reviewed-by: Cody Northrop <cnorthrop@google.com> Reviewed-by: Jamie Madill <jmadill@chromium.org>
Gert Wollny f7891766 2021-04-12T14:24:37 Capture/Replay: write context serializations on failure With this data available as JSON text files further analysis can be executed to drill down to what the actual difference is. Bug: angleproject:5715 Change-Id: I7012668ff9617f6232a6bbe1798a02d8c795321d Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2821534 Commit-Queue: Gert Wollny <gert.wollny@collabora.com> Reviewed-by: Cody Northrop <cnorthrop@google.com> Reviewed-by: Jamie Madill <jmadill@chromium.org>
Jamie Madill 78dde332 2021-03-29T17:31:52 Move restricted traces to CIPD. All traces are now stored as DEPS entries in CIPD. The auto-generation script generates the DEPS entries. Note that we don't include DEPS in the list of generated outputs to simplify other rollers. Also we update auto-generation to include full sources list to allow 'gn analyze' to work successfully. Usees a trace fixture for common code. This will enable a more compact trace without as much repeated code. We must land a set of re-trace traces to avoid breakage. Also includes a python script for uploading new traces to CIPD. The script first checks if traces are already present in the cloud, and if so it skips the upload. It will take a while to complete as the number of traces grows larger as it takes a few seconds per trace. The traces in this patch are also re-traced to use the common fixture code instead of including duplicated code in each trace. They now form a simple common interface and the autogenerated cpp is now simply a list of properties. I've also updated the capture/replay tests to use the simpler common entry point integration. There is less auto-generated glue code now. We now use a new serialized string query extension instead of calling directly into ANGLE's internals. Also includes a docs update. The capture/replay sample is broken and we'll need to update it in a follow-up CL. Also includes a few necessary fixes to the retracing script. Bug: angleproject:5811 Change-Id: I977bc6dc56843c8966377fc445ae97e91e17319a Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2797833 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Cody Northrop <cnorthrop@google.com> Reviewed-by: Tim Van Patten <timvp@google.com>
Jamie Madill 67e4aff5 2021-04-02T13:17:00 Fix rapidjson build error in Skia. Instead of using defines in the header, use the same approach as we do with frame capture by defining a stub "mock" cpp file. Bug: angleproject:5805 Change-Id: Ief1cb6497ddafc9656bb0e7d6a921eff3610a7fb Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2801695 Reviewed-by: Cody Northrop <cnorthrop@google.com> Reviewed-by: Jamie Madill <jmadill@chromium.org> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Gert Wollny cee0bfd6 2021-03-15T14:39:30 Capture/Replay: serialize context to JSON Bug: angleproject:5715 Change-Id: Ibc6897d7567fedab4a2ba7bf8d2bd5a7e496bd4b Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2760325 Reviewed-by: Jamie Madill <jmadill@chromium.org> Reviewed-by: Cody Northrop <cnorthrop@google.com> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Jamie Madill 537cd0d5 2021-02-11T12:38:10 Capture Tests: Update path to capture/. Bug: angleproject:5530 Change-Id: Ibbabab2bcc03c196c6c51e5bf67d81df8bad4925 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2689364 Reviewed-by: Cody Northrop <cnorthrop@google.com> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Jamie Madill 9bbcd86a 2021-01-25T10:52:35 Capture/Replay: Fix instanced array client data. Enables a bunch of self-tests. Bug: angleproject:5530 Change-Id: Idd14574ba0f3d44124e153ccb32fec7318baf217 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2647745 Reviewed-by: Tim Van Patten <timvp@google.com> Reviewed-by: Cody Northrop <cnorthrop@google.com> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Jamie Madill e60ce3da 2021-01-21T16:42:17 Capture/Replay: Allow testing with multiple versions. This embeds the context version and device type info into the replay. The self-tests then can create the correct display and context types. This fixes testing against SwiftShader which is necessary for the bots. Bug: angleproject:4759 Change-Id: If9da6bfdc1c2b315ccd7e453872fc84063277054 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2643363 Reviewed-by: Tim Van Patten <timvp@google.com> Reviewed-by: Cody Northrop <cnorthrop@google.com> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Jamie Madill 39b777c6 2020-10-29T16:55:06 Capture/Replay: Two cleanups. This changes from returning a vector to directly returning a pointer to the binary data for the serialized state. The second cleanup is to use a ContextID as a wrapped type which simplifies the output formatting code. Bug: angleproject:5247 Change-Id: Ieb8afdb9326a12968dd2d69c05e1ed811b93abff Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2506198 Reviewed-by: Courtney Goeltzenleuchter <courtneygo@google.com> Reviewed-by: Cody Northrop <cnorthrop@google.com> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Jamie Madill 5c4c37dc 2020-10-25T15:55:45 Capture/Replay: Test improvements. Streamlines some arguments. Use a common subdirectory in out/. Show process name when logging. Clean up environment variable handling. Bug: angleproject:5247 Change-Id: I85eb5b8194a64f90691c4d03db3ee65c5e9b2313 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2497560 Reviewed-by: Cody Northrop <cnorthrop@google.com> Reviewed-by: Courtney Goeltzenleuchter <courtneygo@google.com> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Manh Nguyen 6aed2832 2020-07-31T10:52:21 Batch capture run + change how results are logged Before, the run stages of tests in a batch were run separately by multiple subprocesses. Now the run stages of tests in a batch are batched together and run by a single subprocess. Changes how results are logged. Tests in a batch are logged together. Within a batch, tests that fail at the same stage are also logged together. Bug: angleproject:4817 Change-Id: Ie3f992c081de914f1f1f521bec2d72f06ccca238 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2331738 Commit-Queue: Manh Nguyen <nguyenmh@google.com> Reviewed-by: Jamie Madill <jmadill@chromium.org> Reviewed-by: Cody Northrop <cnorthrop@google.com>
Manh Nguyen 2c41931c 2020-07-21T14:21:27 Batch-compile and batch-run-replay multiple tests Multiple tests are batch-compiled into 1 replay application instead of multiple replay applications. Replay application now runs generated code of multiple tests instead of 1 test. This reduces overhead cost and brings down runtime. Main process now receives messages sent by workers via a message queue and prints them to the main stdout so that user can know if workers are hanging. Add handle for user interrupt (Ctrl-C) so that processes are properly destroyed and cleaned up. Trace files now have the option not to be deleted. Bug: angleproject:4817 Change-Id: Ic90ae0f430e1d3c261ffea5f963be5a4e94b0ad2 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2310909 Reviewed-by: Cody Northrop <cnorthrop@google.com> Reviewed-by: Jamie Madill <jmadill@chromium.org> Commit-Queue: Manh Nguyen <nguyenmh@google.com>