Commit d6bf8e0df01cca165085ff8f1270a04918edab95
2023-10-02T18:10:36
Improve capture_replay_tests (win-trace) logging * Get rid of the CRASH expectation as it is always confusing. Replace items in expectations with SKIP_FOR_CAPTURE. This will no longer run the corresponding tests (which was making logs even more confusing) * Use CaptureFailed and ReplayFailed instead of Crashed status (the actual meaning is a non-zero return code from the test harness) * Introduce batch names by indexing them as they are created: batch_000, batch_001, ... This enables easy searching in logs. * When logging a batch (result) include its name and why it is logged * Failed batches are printed with full output instead of "abbreviated" output. One issue with b/303065176 was that an important part of the output (errors) was truncated by this "abbreviation" * Remove "regression log" and "Summary" as these are often just confusing: regression log just duplicates information printed earlier and summary may not be clear w.r.t test expectations. The part that prints what was unexpected is still there. It also now includes batch names so it's much more obvious what comes from where. Example: Unexpected 'CaptureFailed' (8): != Pass: batch_000 FramebufferTest_ES3.RenderAndInvalidateImmutableTextureWithBellowBaseLevelLOD/ES3_Vulkan_SwiftShader != Pass: batch_000 FramebufferTest_ES3.RenderAndInvalidateImmutableTextureWithBellowBaseLevelLOD/ES3_Vulkan_SwiftShader_AsyncCommandQueue != Pass: batch_000 FramebufferTest_ES3.RenderAndInvalidateImmutableTextureWithBellowBaseLevelLOD/ES3_Vulkan_SwiftShader_EnableParallelCompileAndLink != Pass: batch_000 FramebufferTest_ES3.RenderAndInvalidateImmutableTextureWithSubImageWithBeyondMaxLevel/ES3_Vulkan_SwiftShader != Pass: batch_000 FramebufferTest_ES3.RenderAndInvalidateImmutableTextureWithSubImageWithBeyondMaxLevel/ES3_Vulkan_SwiftShader_AsyncCommandQueue != Pass: batch_000 FramebufferTest_ES3.RenderAndInvalidateImmutableTextureWithSubImageWithBeyondMaxLevel/ES3_Vulkan_SwiftShader_EnableParallelCompileAndLink != Pass: batch_000 FramebufferTest_ES3.RenderImmutableTextureWithSubImageWithBeyondMaxLevel/ES3_Vulkan_SwiftShader != Pass: batch_000 FramebufferTest_ES3.RenderImmutableTextureWithSubImageWithBeyondMaxLevel/ES3_Vulkan_SwiftShader_AsyncCommandQueue Unexpected 'FailedToTrace' (1): != Pass: batch_001 FramebufferTest_ES3.RenderSampleDepthTextureWithExcludedLevel/ES3_Vulkan_SwiftShader Bug: angleproject:7519 Bug: b/303065176 Change-Id: Ie0c4734c7c4ac965aa6fd38b0c1783e0b92af7a5 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4908349 Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org> Commit-Queue: Roman Lavrov <romanl@google.com> Reviewed-by: Cody Northrop <cnorthrop@google.com>