Hash :
d6bf8e0d
Author :
Date :
2023-10-02T18:10:36
Improve capture_replay_tests (win-trace) logging
* Get rid of the CRASH expectation as it is always confusing. Replace
items in expectations with SKIP_FOR_CAPTURE. This will no longer run
the corresponding tests (which was making logs even more confusing)
* Use CaptureFailed and ReplayFailed instead of Crashed status (the
actual meaning is a non-zero return code from the test harness)
* Introduce batch names by indexing them as they are created:
batch_000, batch_001, ...
This enables easy searching in logs.
* When logging a batch (result) include its name and why it is logged
* Failed batches are printed with full output instead of "abbreviated"
output. One issue with b/303065176 was that an important part
of the output (errors) was truncated by this "abbreviation"
* Remove "regression log" and "Summary" as these are often just
confusing: regression log just duplicates information printed earlier
and summary may not be clear w.r.t test expectations. The part that
prints what was unexpected is still there. It also now includes
batch names so it's much more obvious what comes from where. Example:
Unexpected 'CaptureFailed' (8):
!= Pass: batch_000 FramebufferTest_ES3.RenderAndInvalidateImmutableTextureWithBellowBaseLevelLOD/ES3_Vulkan_SwiftShader
!= Pass: batch_000 FramebufferTest_ES3.RenderAndInvalidateImmutableTextureWithBellowBaseLevelLOD/ES3_Vulkan_SwiftShader_AsyncCommandQueue
!= Pass: batch_000 FramebufferTest_ES3.RenderAndInvalidateImmutableTextureWithBellowBaseLevelLOD/ES3_Vulkan_SwiftShader_EnableParallelCompileAndLink
!= Pass: batch_000 FramebufferTest_ES3.RenderAndInvalidateImmutableTextureWithSubImageWithBeyondMaxLevel/ES3_Vulkan_SwiftShader
!= Pass: batch_000 FramebufferTest_ES3.RenderAndInvalidateImmutableTextureWithSubImageWithBeyondMaxLevel/ES3_Vulkan_SwiftShader_AsyncCommandQueue
!= Pass: batch_000 FramebufferTest_ES3.RenderAndInvalidateImmutableTextureWithSubImageWithBeyondMaxLevel/ES3_Vulkan_SwiftShader_EnableParallelCompileAndLink
!= Pass: batch_000 FramebufferTest_ES3.RenderImmutableTextureWithSubImageWithBeyondMaxLevel/ES3_Vulkan_SwiftShader
!= Pass: batch_000 FramebufferTest_ES3.RenderImmutableTextureWithSubImageWithBeyondMaxLevel/ES3_Vulkan_SwiftShader_AsyncCommandQueue
Unexpected 'FailedToTrace' (1):
!= Pass: batch_001 FramebufferTest_ES3.RenderSampleDepthTextureWithExcludedLevel/ES3_Vulkan_SwiftShader
Bug: angleproject:7519
Bug: b/303065176
Change-Id: Ie0c4734c7c4ac965aa6fd38b0c1783e0b92af7a5
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4908349
Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org>
Commit-Queue: Roman Lavrov <romanl@google.com>
Reviewed-by: Cody Northrop <cnorthrop@google.com>