Commit af35fd22de30132b443021da0e5a73de664216e4
2021-07-12T14:58:01
Capture/Replay: rework test runner to use expectations This CL changes the test runner to use a expectation file for all the tests including failing tests. The expectation file is also used to disable running certain tests. e.g. because they are known to crash or result in compilation failures, which then take down all the test in the corresponding batch. The capture_replay_tests.py script will report failure if the test result doesn't meet the expectation. In order to keep the current behaviour to run subsets of tests in tact, the test filter is also applied to the expectation. In addition to the "normal" expectations, FLAKY was added to tag tests that can be captured and compiled correctly but have unstable results. This CL also adds the expectation for ES2_Vulkan_SwiftShader and ES2_Vulkan Bug: angleproject:6166 Change-Id: Ica05303e6ae3b7c6d2b6749c2c9c6593b659cc1a Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3024209 Reviewed-by: Jamie Madill <jmadill@chromium.org> Reviewed-by: Tim Van Patten <timvp@google.com> Reviewed-by: Cody Northrop <cnorthrop@google.com> Commit-Queue: Gert Wollny <gert.wollny@collabora.com>