src/tests/capture_replay_tests


Log

Author Commit Date CI Message
Cody Northrop 135d1483 2021-06-03T13:24:16 Tests: Skip flaky capture_replay_tests Several VulkanExternalImageTest* tests are timing out locally, but not consistently. Bug: angleproject:5721 Change-Id: Iffb6d319bb31206af3287a70259a47857b64f434 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2937736 Reviewed-by: Jamie Madill <jmadill@chromium.org> Reviewed-by: Tim Van Patten <timvp@google.com> Commit-Queue: Cody Northrop <cnorthrop@google.com>
Jamie Madill 16b10a54 2021-05-21T15:35:00 Revert "Capture/Replay: Emit SetupReplay() late" This reverts commit 54ba6f0135715737a55f2dc4fc1cb86874fb98f9. Reason for revert: breaks MEC. Bug: angleproject:5999 Original change's description: > Capture/Replay: Emit SetupReplay() late > > Only emit SetupReplay() after all captured frames have been run, > so that InitializeReplay() is called with values of > readBufferSize and the maxClientArraySize that are sufficiently > large for all frames. > > Bug: angleproject:5965 > Bug: angleproject:5895 > Change-Id: Ic59597dc27f733fb210d7cc5a144a86e54cd2040 > Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2894486 > Commit-Queue: Gert Wollny <gert.wollny@collabora.com> > Reviewed-by: Jamie Madill <jmadill@chromium.org> > Reviewed-by: Cody Northrop <cnorthrop@google.com> Bug: angleproject:5965 Bug: angleproject:5895 Change-Id: Ia901183b2052fa2b4552ad2e09818a2894fbdf6b Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2911878 Reviewed-by: Jamie Madill <jmadill@chromium.org> Commit-Queue: Jamie Madill <jmadill@chromium.org> Commit-Queue: Cody Northrop <cnorthrop@google.com>
Cody Northrop 9ba86e06 2021-06-03T13:41:39 Revert "FrameCapture: Write the frame cpp file even if frame is empty" This reverts commit 6619c464e5f6237d7465ec945d24164372aa4196. Reason for revert: breaks MEC. Bug: angleproject:5999 Original change's description: > FrameCapture: Write the frame cpp file even if frame is empty > > The index file will add a reference to this file, and the replay will > also expect that the frame execution function and the context function > exist, so write the file even if the frame is empty > > Bug: angleproject:5939 > Change-Id: I265d061fdae51b86131d145116ad1c0f9ca8c93f > Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2882581 > Commit-Queue: Gert Wollny <gert.wollny@collabora.com> > Reviewed-by: Cody Northrop <cnorthrop@google.com> > Reviewed-by: Jamie Madill <jmadill@chromium.org> Bug: angleproject:5939 Change-Id: I759d44ba847c69faf1caf97a9818836d1be3b058 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2937737 Reviewed-by: Cody Northrop <cnorthrop@google.com> Reviewed-by: Jamie Madill <jmadill@chromium.org> Reviewed-by: Tim Van Patten <timvp@google.com> Commit-Queue: Cody Northrop <cnorthrop@google.com>
Cody Northrop fd262fdc 2021-05-26T18:10:29 Tests: Update capture_replay_test expectations Bug: angleproject:6006 Change-Id: I56d21a9d9df343fbf805c10307265cab929914c0 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2920087 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Tim Van Patten <timvp@google.com> Reviewed-by: Jamie Madill <jmadill@chromium.org>
Gert Wollny 6619c464 2021-05-10T13:00:48 FrameCapture: Write the frame cpp file even if frame is empty The index file will add a reference to this file, and the replay will also expect that the frame execution function and the context function exist, so write the file even if the frame is empty Bug: angleproject:5939 Change-Id: I265d061fdae51b86131d145116ad1c0f9ca8c93f Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2882581 Commit-Queue: Gert Wollny <gert.wollny@collabora.com> Reviewed-by: Cody Northrop <cnorthrop@google.com> Reviewed-by: Jamie Madill <jmadill@chromium.org>
Gert Wollny 54ba6f01 2021-05-13T20:33:46 Capture/Replay: Emit SetupReplay() late Only emit SetupReplay() after all captured frames have been run, so that InitializeReplay() is called with values of readBufferSize and the maxClientArraySize that are sufficiently large for all frames. Bug: angleproject:5965 Bug: angleproject:5895 Change-Id: Ic59597dc27f733fb210d7cc5a144a86e54cd2040 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2894486 Commit-Queue: Gert Wollny <gert.wollny@collabora.com> Reviewed-by: Jamie Madill <jmadill@chromium.org> Reviewed-by: Cody Northrop <cnorthrop@google.com>
Gert Wollny 45ee87e2 2021-05-17T13:45:25 Capture/Replay: Don't try to serialize blob without data The construct std::vector v; &v[0] doesn't go well with a checked STL, because the element 0 doesn't exists in an empty vector, and obviously it doesn't know that whether the pointer gets de-referenced later. Bug: angleproject:5980 Change-Id: I690c99bd12d74112fe1081ea3134b6ffb11ad23b Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2900226 Reviewed-by: Cody Northrop <cnorthrop@google.com> Reviewed-by: Jamie Madill <jmadill@chromium.org> Commit-Queue: Gert Wollny <gert.wollny@collabora.com>
Gert Wollny 31cd0e98 2021-05-10T10:37:54 Capture/Replay: track robustResourceInit Bug: angleproject:5938 Change-Id: I7c9ab9bebd1ec2dc572a071844a860f99d0befe6 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2882580 Reviewed-by: Cody Northrop <cnorthrop@google.com> Reviewed-by: Jamie Madill <jmadill@chromium.org> Commit-Queue: Cody Northrop <cnorthrop@google.com>
Cody Northrop ec362f9b 2021-05-14T11:46:48 Tests: Update capture_replay_expectations Bug: angleproject:5974 Change-Id: If8f210f569bdd1bf4e81c74cac894049b8c9cb7a Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2896905 Reviewed-by: Cody Northrop <cnorthrop@google.com> Reviewed-by: Jamie Madill <jmadill@chromium.org> Commit-Queue: Cody Northrop <cnorthrop@google.com>
Gert Wollny ee2f3302 2021-05-12T12:07:42 Capture/Replay: recreate the EGL window if contexts don't match In addition enable affected tests. Bug: angleproject:5955 Change-Id: I4fb72e38a633daa4565d3c0ae7cddb6ace7226f1 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2891775 Commit-Queue: Gert Wollny <gert.wollny@collabora.com> Reviewed-by: Jamie Madill <jmadill@chromium.org> Reviewed-by: Cody Northrop <cnorthrop@google.com>
Gert Wollny 2ba3763e 2021-05-10T10:08:27 Capture/Replay: reset CWD after the test ends As suggested by Jamie Bug: angleproject:5849 Change-Id: I3e548935a9119b7ca7ca392c50bcb0f58ae3e974 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2882578 Commit-Queue: Gert Wollny <gert.wollny@collabora.com> Reviewed-by: Jamie Madill <jmadill@chromium.org>
Lubosz Sarnecki a83fe36d 2021-03-09T12:51:27 Capture/Replay: Add GLES1 test expectations. Add failing OpenGL ES 1.0 capture / replay tests to expectations list. The remainging 65 test pass on ES1 SwiftShader. Test: python3 src/tests/capture_replay_tests.py --gtest_filter="*/ES1_Vulkan_SwiftShader" Bug: angleproject:5751 Change-Id: If5157a3ac0ccebc904316ae08c54e670119384e4 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2877233 Reviewed-by: Cody Northrop <cnorthrop@google.com> Reviewed-by: Jamie Madill <jmadill@chromium.org> Commit-Queue: Cody Northrop <cnorthrop@google.com>
Gert Wollny f45dad5e 2021-05-03T11:05:23 Capture/Replay: Enable test for CopyTextureTestWebGL.NPOT Bug: angleproject:5850 Change-Id: I5a8e3ebf6d220251a76f8a428b959b226dca2ca0 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2867077 Commit-Queue: Gert Wollny <gert.wollny@collabora.com> Reviewed-by: Jamie Madill <jmadill@chromium.org> Reviewed-by: Cody Northrop <cnorthrop@google.com>
Gert Wollny 39cef942 2021-04-27T08:28:13 Capture/Replay: capture WebGL compatibility With that we can also enable the tests DifferentStencilMasksTest.* Bug: angleproject:5851 Change-Id: If2c992e456716f606320e3992f389823d0af050f Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2853584 Commit-Queue: Gert Wollny <gert.wollny@collabora.com> Reviewed-by: Cody Northrop <cnorthrop@google.com> Reviewed-by: Jamie Madill <jmadill@chromium.org>
Gert Wollny 422ec18a 2021-04-26T10:40:04 tests: Move DrawAfterFlushWithNoStateChange to ES3 The test uses Vertex Array Objects, and these are only supported from GLES 3.0 on, or by using the extension OES_vertex_array_object, so make it part of the StateChangeTestES3 set. With that we also enable the test. Bug: angleproject:5855 Change-Id: I0cfb5d23d19e3db8eefffd9d52b744d068666606 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2850642 Commit-Queue: Gert Wollny <gert.wollny@collabora.com> Reviewed-by: Jamie Madill <jmadill@chromium.org>
Gert Wollny 3e1af85a 2021-04-19T18:12:30 Capture/Replay: Handle bindGeneratesResource in trace meta data Bug: angleproject:5844 Change-Id: I0243cab70b5833efd83b0d75a1801f68025ee291 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2835889 Commit-Queue: Gert Wollny <gert.wollny@collabora.com> Reviewed-by: Jamie Madill <jmadill@chromium.org> Reviewed-by: Cody Northrop <cnorthrop@google.com>
Gert Wollny c1da2794 2021-04-19T11:49:41 Capture/Replay: Handle ClientArraysEnabled in trace meta data Bug: angleproject:5848 Change-Id: Ieba325e2b5efa3242b24cafa8363e562c258546c Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2835888 Commit-Queue: Gert Wollny <gert.wollny@collabora.com> Reviewed-by: Cody Northrop <cnorthrop@google.com> Reviewed-by: Jamie Madill <jmadill@chromium.org>
Gert Wollny 6f3bd395 2021-04-13T23:53:53 Capture/Replay: Enable InstancingTest.LineLoop/* tests Bug: angleproject:5853 Change-Id: Ia1a7a1cec781dca22ab49a9189dd3560ac148216 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2824435 Commit-Queue: Gert Wollny <gert.wollny@collabora.com> Reviewed-by: Jamie Madill <jmadill@chromium.org>
Gert Wollny fb3173b9 2021-04-13T15:05:44 FrameCapture: write the float values with high precision Because values might be the result of calculations that require some accuracy, writing the float values with the default formatting may not be sufficient to replay correctly, therefore format the values so that they carry more digits. In addition, enable the test that are now passing. Bug: angleproject:5846 Bug: angleproject:5847 Change-Id: I305552a1ffa3cded6326df59883912e14c95f5f5 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2822257 Commit-Queue: Gert Wollny <gert.wollny@collabora.com> Reviewed-by: Jamie Madill <jmadill@chromium.org>
Gert Wollny f7891766 2021-04-12T14:24:37 Capture/Replay: write context serializations on failure With this data available as JSON text files further analysis can be executed to drill down to what the actual difference is. Bug: angleproject:5715 Change-Id: I7012668ff9617f6232a6bbe1798a02d8c795321d Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2821534 Commit-Queue: Gert Wollny <gert.wollny@collabora.com> Reviewed-by: Cody Northrop <cnorthrop@google.com> Reviewed-by: Jamie Madill <jmadill@chromium.org>
Jamie Madill 78dde332 2021-03-29T17:31:52 Move restricted traces to CIPD. All traces are now stored as DEPS entries in CIPD. The auto-generation script generates the DEPS entries. Note that we don't include DEPS in the list of generated outputs to simplify other rollers. Also we update auto-generation to include full sources list to allow 'gn analyze' to work successfully. Usees a trace fixture for common code. This will enable a more compact trace without as much repeated code. We must land a set of re-trace traces to avoid breakage. Also includes a python script for uploading new traces to CIPD. The script first checks if traces are already present in the cloud, and if so it skips the upload. It will take a while to complete as the number of traces grows larger as it takes a few seconds per trace. The traces in this patch are also re-traced to use the common fixture code instead of including duplicated code in each trace. They now form a simple common interface and the autogenerated cpp is now simply a list of properties. I've also updated the capture/replay tests to use the simpler common entry point integration. There is less auto-generated glue code now. We now use a new serialized string query extension instead of calling directly into ANGLE's internals. Also includes a docs update. The capture/replay sample is broken and we'll need to update it in a follow-up CL. Also includes a few necessary fixes to the retracing script. Bug: angleproject:5811 Change-Id: I977bc6dc56843c8966377fc445ae97e91e17319a Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2797833 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Cody Northrop <cnorthrop@google.com> Reviewed-by: Tim Van Patten <timvp@google.com>
Jamie Madill 67e4aff5 2021-04-02T13:17:00 Fix rapidjson build error in Skia. Instead of using defines in the header, use the same approach as we do with frame capture by defining a stub "mock" cpp file. Bug: angleproject:5805 Change-Id: Ief1cb6497ddafc9656bb0e7d6a921eff3610a7fb Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2801695 Reviewed-by: Cody Northrop <cnorthrop@google.com> Reviewed-by: Jamie Madill <jmadill@chromium.org> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Gert Wollny cee0bfd6 2021-03-15T14:39:30 Capture/Replay: serialize context to JSON Bug: angleproject:5715 Change-Id: Ibc6897d7567fedab4a2ba7bf8d2bd5a7e496bd4b Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2760325 Reviewed-by: Jamie Madill <jmadill@chromium.org> Reviewed-by: Cody Northrop <cnorthrop@google.com> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Gert Wollny d00bd105 2021-03-15T11:13:34 Capture/Replay: Enable GetTexImage tests Bug: angleproject:5646 Change-Id: Ic3ffc08edb2504b3f5cf6913e3995f8da8b7a94a Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2791443 Commit-Queue: Gert Wollny <gert.wollny@collabora.com> Reviewed-by: Cody Northrop <cnorthrop@google.com> Reviewed-by: Jamie Madill <jmadill@chromium.org>
Lubosz Sarnecki 8d110222 2021-03-05T18:13:40 capture_replay_expectations: Don't skip passing. Several tests now pass the comparision and do not need to be skipped. Replace the wildcard with specific tests if only one test is failing. Bug: angleproject:5721 Change-Id: I70172965d6e0df9d2c3ac43c391c377f7da12a6e Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2739727 Commit-Queue: Lubosz Sarnecki <lubosz.sarnecki@collabora.com> Reviewed-by: Cody Northrop <cnorthrop@google.com> Reviewed-by: Jamie Madill <jmadill@chromium.org>
Lubosz Sarnecki 9265d1e2 2021-03-05T17:54:59 frame_capture_utils: Do not serialize 0 bindings. Since we are only interested in the difference to non-default states, bindings with a zero ID should not be serialized. This resolves the comparision error in the GetTexLevelParameterTest.* tests. The test now do not need to be skipped. Bug: angleproject:5645 Change-Id: I887bc22b52a201c467c00c42f6ddb3b38895a7c4 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2739726 Reviewed-by: Jamie Madill <jmadill@chromium.org> Reviewed-by: Cody Northrop <cnorthrop@google.com> Commit-Queue: Lubosz Sarnecki <lubosz.sarnecki@collabora.com>
Lubosz Sarnecki 04a641ac 2021-03-04T10:57:37 capture_replay_tests: Change status to crashing. Move both SemaphoreTests to crashing. Don't skip passing VulkanExternalImageTest.*. Explicitly list VulkanExternalImageTest.* tests that still crash. All these tests compile now. Bug: angleproject:5721 Change-Id: Ib529dff8895496925001e4649aabcae78ce77236 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2739725 Commit-Queue: Lubosz Sarnecki <lubosz.sarnecki@collabora.com> Reviewed-by: Cody Northrop <cnorthrop@google.com> Reviewed-by: Jamie Madill <jmadill@chromium.org>
Lubosz Sarnecki c9c75b99 2021-03-04T10:21:03 capture_gles_ext_params: Implement glGetIntegervRobustANGLE. Move RobustClientMemoryTest.* to crashed tests, as it builds now. Bug: angleproject:5721 Change-Id: I590f4bd679374ee3302fc985a53747ddd60dd429 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2739724 Commit-Queue: Lubosz Sarnecki <lubosz.sarnecki@collabora.com> Reviewed-by: Cody Northrop <cnorthrop@google.com> Reviewed-by: Jamie Madill <jmadill@chromium.org>
Lubosz Sarnecki d96b9ffa 2021-03-04T09:57:53 capture_replay_tests: Don't skip tests that pass. These tests were skipped due to compilation problems, but do now fully pass on SwiftShader ES2. Remove PointSpritesTest.PointWithoutAttributesCompliance/* Remove ExplicitContextTest.GetProcAddress/* Remove RobustBufferAccessBehaviorTest.EmptyBuffer/* Remove EGLSyncTest.AndroidNativeFence_WaitSync/* Bug: angleproject:5721 Change-Id: I3f50272c69ba78e2432f4278753e5d48d8601910 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2739723 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Cody Northrop <cnorthrop@google.com> Reviewed-by: Jamie Madill <jmadill@chromium.org>
Lubosz Sarnecki c11186c2 2021-03-02T14:42:08 capture_replay_tests: Fix MemoryObjectTest. Add gles_ext_params Capture functions for MemoryObjectsEXT. Add case for GLCreateMemoryObjectsEXT to MaybeCaptureUpdateResourceIDs. Don't skip MemoryObjectTest, as it passes now on SwiftShader ES2. Bug: angleproject:5703 Change-Id: I930fab675cf0848f46a846388955a6bd3274a587 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2729004 Commit-Queue: Lubosz Sarnecki <lubosz.sarnecki@collabora.com> Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Cody Northrop <cnorthrop@google.com> Reviewed-by: Jamie Madill <jmadill@chromium.org>
Lubosz Sarnecki d24398b3 2021-02-25T15:07:00 gl.xml: Annotate glDiscardFramebufferEXT with param groups. Remove tests from expectations list, as it now passes on SwiftShader for ES2 and ES3. Bug: angleproject:4401 Change-Id: If05b16e40632154a0ba2c22f1c3bf29e12113389 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2720259 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Jamie Madill <jmadill@chromium.org> Reviewed-by: Cody Northrop <cnorthrop@google.com>
Jamie Madill cf8c71de 2021-02-11T16:32:00 Supress failing capture/replay test. StateChangeTest.DrawAfterFlushWithNoStateChange/* Bug: angleproject:5530 Change-Id: I8e9365219a617b8c6dd6d5c767e40ecde100a76d Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2689378 Reviewed-by: Jamie Madill <jmadill@chromium.org> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Jamie Madill 537cd0d5 2021-02-11T12:38:10 Capture Tests: Update path to capture/. Bug: angleproject:5530 Change-Id: Ibbabab2bcc03c196c6c51e5bf67d81df8bad4925 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2689364 Reviewed-by: Cody Northrop <cnorthrop@google.com> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Cody Northrop 41a1920e 2021-02-08T22:04:10 Tests: Reduce capture_replay_test exception list Remove some tests that no longer fail to compile. Test: capture_replay_tests.py --gtest_filter=*ES2_Vulkan_SwiftShader Bug: angleproject:5630 Change-Id: If04787494e58afad0a98d0ee7f37e00d2e77ec1f Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2683543 Reviewed-by: Jamie Madill <jmadill@chromium.org> Commit-Queue: Cody Northrop <cnorthrop@google.com>
Jamie Madill f6925344 2021-01-25T13:13:33 Trace Tests: Update expectations. Bug: angleproject:5530 Change-Id: I48a6dc5753406e6dd6cd1845f1f28fcbfc485e34 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2647871 Reviewed-by: Cody Northrop <cnorthrop@google.com> Reviewed-by: Tim Van Patten <timvp@google.com> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Jamie Madill 9bbcd86a 2021-01-25T10:52:35 Capture/Replay: Fix instanced array client data. Enables a bunch of self-tests. Bug: angleproject:5530 Change-Id: Idd14574ba0f3d44124e153ccb32fec7318baf217 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2647745 Reviewed-by: Tim Van Patten <timvp@google.com> Reviewed-by: Cody Northrop <cnorthrop@google.com> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Jamie Madill cfb9c30c 2021-01-25T10:54:09 Capture/Replay: Apply skips correctly on SwS. The skips listed currently were explicitly for ES2_Vulkan instead of applying to both desktop and SwS Vulkan implementations. This CL swaps the expectations to use wildcards so they will work for both. Bug: angleproject:5530 Change-Id: Iae83207411296cb819bb0a25d2247cb4af863adb Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2645723 Reviewed-by: Jamie Madill <jmadill@chromium.org> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Jamie Madill e60ce3da 2021-01-21T16:42:17 Capture/Replay: Allow testing with multiple versions. This embeds the context version and device type info into the replay. The self-tests then can create the correct display and context types. This fixes testing against SwiftShader which is necessary for the bots. Bug: angleproject:4759 Change-Id: If9da6bfdc1c2b315ccd7e453872fc84063277054 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2643363 Reviewed-by: Tim Van Patten <timvp@google.com> Reviewed-by: Cody Northrop <cnorthrop@google.com> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Jamie Madill a4d05638 2021-01-18T14:58:22 Capture/Replay: Don't serialize compile resources string. The resources string gets changed dynamically during capture. This breaks the regression tests because they were comparing for identity. Bug: angleproject:5530 Change-Id: I0734f735577a5ff4c9083adb8f25a37aad3c0e37 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2634830 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Cody Northrop <cnorthrop@google.com> Reviewed-by: Tim Van Patten <timvp@google.com>
Jamie Madill 738cb55b 2021-01-14T17:33:20 Capture/Replay tests: Fixes and improvements. The script was broken after the change to build to a separate shared library. Also fixes issues on Windows and cleans up the code. State comparison is still broken on every test from what I can see. Bug: angleproject:5530 Change-Id: I23e77e56fadedfb85d7afdb59dddac5558a71a56 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2630232 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Tim Van Patten <timvp@google.com> Reviewed-by: Cody Northrop <cnorthrop@google.com>
Jamie Madill 39b777c6 2020-10-29T16:55:06 Capture/Replay: Two cleanups. This changes from returning a vector to directly returning a pointer to the binary data for the serialized state. The second cleanup is to use a ContextID as a wrapped type which simplifies the output formatting code. Bug: angleproject:5247 Change-Id: Ieb8afdb9326a12968dd2d69c05e1ed811b93abff Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2506198 Reviewed-by: Courtney Goeltzenleuchter <courtneygo@google.com> Reviewed-by: Cody Northrop <cnorthrop@google.com> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Jamie Madill 1c53df78 2020-10-26T13:42:14 Capture/Replay Tests: Expand suppressions. Updates the suppression list to include Linux tests and other new failures. Also sort the test list output to make it more readable. Bug: angleproject:5134 Change-Id: I38b5d95ceab3270d1cab13e3fb3c19d3137d10dc Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2497568 Reviewed-by: Cody Northrop <cnorthrop@google.com> Reviewed-by: Courtney Goeltzenleuchter <courtneygo@google.com> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Jamie Madill 5c4c37dc 2020-10-25T15:55:45 Capture/Replay: Test improvements. Streamlines some arguments. Use a common subdirectory in out/. Show process name when logging. Clean up environment variable handling. Bug: angleproject:5247 Change-Id: I85eb5b8194a64f90691c4d03db3ee65c5e9b2313 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2497560 Reviewed-by: Cody Northrop <cnorthrop@google.com> Reviewed-by: Courtney Goeltzenleuchter <courtneygo@google.com> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Manh Nguyen c65517d6 2020-08-12T17:43:53 Add tests' expectations to capture replay tests Skips failing tests so that they do not affect the results of passing tests. Bug: angleproject:4817 Change-Id: I80678cb62b0b81d6a1aa811908019022efbe1268 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2351394 Reviewed-by: Cody Northrop <cnorthrop@google.com> Reviewed-by: Jamie Madill <jmadill@chromium.org> Commit-Queue: Manh Nguyen <nguyenmh@google.com>
Manh Nguyen 6aed2832 2020-07-31T10:52:21 Batch capture run + change how results are logged Before, the run stages of tests in a batch were run separately by multiple subprocesses. Now the run stages of tests in a batch are batched together and run by a single subprocess. Changes how results are logged. Tests in a batch are logged together. Within a batch, tests that fail at the same stage are also logged together. Bug: angleproject:4817 Change-Id: Ie3f992c081de914f1f1f521bec2d72f06ccca238 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2331738 Commit-Queue: Manh Nguyen <nguyenmh@google.com> Reviewed-by: Jamie Madill <jmadill@chromium.org> Reviewed-by: Cody Northrop <cnorthrop@google.com>
Manh Nguyen 2c41931c 2020-07-21T14:21:27 Batch-compile and batch-run-replay multiple tests Multiple tests are batch-compiled into 1 replay application instead of multiple replay applications. Replay application now runs generated code of multiple tests instead of 1 test. This reduces overhead cost and brings down runtime. Main process now receives messages sent by workers via a message queue and prints them to the main stdout so that user can know if workers are hanging. Add handle for user interrupt (Ctrl-C) so that processes are properly destroyed and cleaned up. Trace files now have the option not to be deleted. Bug: angleproject:4817 Change-Id: Ic90ae0f430e1d3c261ffea5f963be5a4e94b0ad2 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2310909 Reviewed-by: Cody Northrop <cnorthrop@google.com> Reviewed-by: Jamie Madill <jmadill@chromium.org> Commit-Queue: Manh Nguyen <nguyenmh@google.com>
Manh Nguyen 10e9e9a1 2020-07-13T17:02:38 Apply multiprocessing to capture_replay_tests.py Multiple tests are run in paralell to squeeze the most out of CPUs. Tests are stored in a work queue. Whenever a CPU becomes available, the next unrun test is grabbed and run on a spawn process on that CPU. Each cpu gets their own environment and build folders and trace folders so that tests don't overwrite each other. Bug: angleproject:4817 Change-Id: Ifd35c9c75522e480b0257d090d5af70f2a3428ba Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2296040 Commit-Queue: Manh Nguyen <nguyenmh@google.com> Reviewed-by: Cody Northrop <cnorthrop@google.com> Reviewed-by: Jamie Madill <jmadill@chromium.org>
Manh Nguyen e5c18475 2020-07-06T09:50:49 Multi-frames serialization + comparision for capture replay test Changes how context states are serialized during capture run. Now, context states are serialized after each frame until the end frame is reached, or context is destroyed. Adds an api to the trace files to query serialization results of multiple frames. Makes change to CaptureReplayTest to serialize multiple frames then compares the serialization results with the ones retrieved from said api. Adds to capture replay the ability to produce working, compilable trace files even when end frame is not reached. Adds to the generated trace files config information and makes CaptureReplayTest utilize said config information to reproduce the exact environment of the captured application. Bug: angleproject:4817 Change-Id: Ie3d487af2bacf349dc3ff6f6b1b5f89e1169dc84 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2282885 Commit-Queue: Manh Nguyen <nguyenmh@google.com> Reviewed-by: Cody Northrop <cnorthrop@google.com> Reviewed-by: Jamie Madill <jmadill@chromium.org>
Manh Nguyen dce37b7d 2020-06-21T22:27:06 Serialize framebuffers + compare contexts for CaptureReplayTests Adds to frame capture the ability to serialize a frame's pre-swap GL state and store it in the binary data file Adds to CaptureReplayTests the ability to compare its serialized GL state with the serialized state pulled from the binary data file Adds a serialization module that serializes framebuffers' GL states and the contents of their color attachments Adds checks to automation script so that it would skips tests that do not produce the expected trace files Adds exception handling to automation script so that it will not crash when a replay build crashes Bug: angleproject:4779 Change-Id: I40a02e018073749e79f0ddbfd3d4065745548f46 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2258295 Commit-Queue: Manh Nguyen <nguyenmh@google.com> Reviewed-by: Jamie Madill <jmadill@chromium.org> Reviewed-by: Tim Van Patten <timvp@google.com>
Manh Nguyen bdf91e5f 2020-06-29T13:47:13 Refactor frame_capture_utils.h to frame_capture_test_utils.h frame_capture_utils.h is only used in tests so changing the name to be frame_capture_test_utils.h is more appropriate. Also frame_capture_utils.h will now be a file in libANGLE_with_capture library where serialization code is stored. Bug: angleproject:4806 Change-Id: I7482693a75b2d0edda7e84ae9b777fd3f46f1855 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2273917 Commit-Queue: Manh Nguyen <nguyenmh@google.com> Reviewed-by: Jamie Madill <jmadill@chromium.org>
Manh Nguyen 9dee8e3f 2020-06-15T10:59:18 Automate testing of capture_replay automation script will: 1. compile all tests in angle_end2end with framecapture enabled 2. run each test with frame capture 3. compile CaptureReplayTest with cpp trace files 4. Run CaptureReplayTest Test: run_tests_with_framecapture.py Bug: angleproject:4749 Change-Id: Ieb21c6137653632d73beedea48712e46fc15a0f3 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2246321 Reviewed-by: Jamie Madill <jmadill@chromium.org> Reviewed-by: Cody Northrop <cnorthrop@google.com> Commit-Queue: Manh Nguyen <nguyenmh@google.com>