kc3-lang/angle/doc/TestingAndProcesses.md

Branch - branch -main

• Show log

  Commit

• Hash : 914fe61b
  Author :
  Date : 2024-03-15T13:20:49
  

  Vulkan: Rename RendererVk.* to vk_renderer.*
  
  Done in a separate CL from the move to namespace vk to avoid possible
  rebase-time confusion with the file name change.
  
  Bug: angleproject:8564
  Change-Id: Ibab79029834b88514d4466a7a4c076b1352bc450
  Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/5370107
  Reviewed-by: Amirali Abdolrashidi <abdolrashidi@google.com>
  Commit-Queue: Amirali Abdolrashidi <abdolrashidi@google.com>
  

Download

---

Source

• doc/TestingAndProcesses.md

• # Handling Test Failures
  
  ANGLE is tested by a number of test suites, including by the ANGLE CI and Try testers, and by the
  Chromium FYI testers.  See pointers and explanations in the
  [ANGLE Wrangling](https://chromium.googlesource.com/angle/angle/+/refs/heads/main/infra/ANGLEWrangling.md) documentation.
  
  We run a large number of tests for each ANGLE CL, both in ANGLE standalone and Chromium
  configurations, both pre- and post- commit.  Some tests will fail, crash, or timeout.  If these
  cannot be addressed in a timely manner, file a bug and update test expectations files.  Timeliness
  depends on the context.  For example, a Wrangler trying to unblock an AutoRoller will typically
  suppress failures immediately; where a developer will typically delay landing their CL in favor of
  fixing test failures.
  
  ## Handling a Vulkan Validation Layer error
  
  Many tests are run with an option that enables the Vulkan Validation Layers (sometimes referred to
  as VVL).  Validation errors will cause an otherwise-passing test to fail.
  
  The [vulkan-deps into ANGLE AutoRoller](https://autoroll.skia.org/r/vulkan-deps-angle-autoroll)
  updates ANGLE to the top-of-tree (ToT) upstream Vulkan tools and SDK.  Sometimes validation errors
  are the result of bugs in the Vulkan Validation Layers, sometimes because of bugs in ANGLE.
  Therefore, investigate the cause of the error and determine if it's an ANGLE bug or a Vulkan
  Validation Layer bug.  For Vulkan Validation Layer bugs, file an
  [upstream bug](https://github.com/KhronosGroup/Vulkan-ValidationLayers/issues/new), and
  suppress the error.  The ANGLE Wrangler will also suppress a validation error when the
  `vulkan-deps` AutoRoller introduces a new validation error.  The ANGLE Wrangler isn't expected to
  resolve the error or diagnose an upstream bug (but it is welcome as extra credit).
  
  Handle a validation error by doing the following:
  
  1. [File an ANGLE bug](http://anglebug.com/new). If this is an active Wrangler issue, set the Label
     `Hotlist-Wrangler` on the bug.
  2. Add the VVL error tag to the
     [kSkippedMessages](https://chromium.googlesource.com/angle/angle.git/+/8f8ca06dfb903fcc8517c69142c46c05e618f40d/src/libANGLE/renderer/vulkan/RendererVk.cpp#129)
     array in `vk_renderer.cpp` file.  Follow the pattern for adding a comment with the associated bug
     in the line above the VVL tag.
  
  
  ## dEQP test expectations
  
  There are a set of [dEQP](dEQP.md) expectations files in the
  [src/tests/deqp_support](../src/tests/deqp_support) directory.  Notice the format of a line and
  your choices for OS, driver, etc.  This is described in the directory's
  [README.md](../src/tests/deqp_support/README.md) file.  This includes:
  - `FLAKY` - For rare flakes
  - `FAIL` - For a test that flakes often or persistently fails
  - `SKIP` - For a test that crashes
  - `TIMEOUT` - For a test that is very slow and may timeout
  
  
  ## angle_end2end_tests expectations
  
  These expectations all live in the
  [angle_end2end_tests_expectations.txt](../src/tests/angle_end2end_tests_expectations.txt) file.  The file format
  is the same as for the dEQP expectations.  However, `FAIL` and `FLAKY` are not valid, and so the choices are:
  - `SKIP` - For a test that fails or crashes
  - `TIMEOUT` - For a test that is very slow and may timeout
  
  
  ## Rubber stamping expectations CLs
  
  CLs that only change dEQP or angle_end2end_tests expectations don't require a human reviewer and can be
  [Rubber Stamped](https://chromium.googlesource.com/infra/infra/+/refs/heads/main/go/src/infra/appengine/rubber-stamper/README.md).
  The procedure is:
  1. Add rubber-stamper@appspot.gserviceaccount.com as the reviewer on the expectations CL. Please also CC a human reviewer for TBR.
  2. After 1 minute, Rubber Stamper should set "Bot-Commit: +1" on the CL.
  3. Now you should be able to "Commit-Queue: +2" your CL.
  
  
  ## WebGL conformance test expectations
  
  The expectations files are hosted in the `chromium/src` repository under
  [content/test/gpu/gpu_tests/test_expectations](https://chromium.googlesource.com/chromium/src/+/refs/heads/main/content/test/gpu/gpu_tests/test_expectations/).
  Note that this is not included in local ANGLE-only source tree.
  
  The format of the file, including the different tags, is documented at the top of the file.  This
  includes the following results:
  
  - `RetryOnFailure` - For a test that rarely flakes
  - `Failure` - For a test that fails consistently or flakes often
  - `Skip` - For a test that causes catastrophic failures (e.g. ends an entire test run, causes a bot
     to BSoD); `Skip` should be used very sparingly
  
  You will need to contact an OWNER of the file to +1 your CL.
  
  You have two options for creating a CL to the expectations files:
  
  1. For trivial edits, you can edit the expectations files via
     [Chromium Code Search](https://source.chromium.org/chromium/chromium/src/+/main:content/test/gpu/gpu_tests/test_expectations/):
      - In the browser, press the `Edit code` button.  This will bring up a new browser window/tab,
        in an editor mode.
      - Edit the expecations and then press the `Create change` (or `Update change` button if you
        need to change your CL later), which will create a CL.
  2. Otherwise please [check out the code](https://chromium.googlesource.com/chromium/src/+/HEAD/docs/get_the_code.md)
     and [upload a CL](https://chromium.googlesource.com/chromium/src/+/refs/heads/main/docs/contributing.md#Creating-a-change)
  
  
  ## "internal swarming failure" on one of the test shards
  
  This might be caused by a bad bot, clicking on link "Bot assigned to task" will show recent tasks which might look bad
  (for example, lots of BOT_DIED in [crbug.com/1350556](https://crbug.com/1350556)). The best course of action in this case is to
  stop the bot by clicking the "stop the bot gracefully" button which will result in a task that looks like
  "Terminate build117-a9", then open a bug in the Infra>Labs component.
  Googlers, see also: [go/gpu-pixel-wrangler-how-to#manually-taking-a-machine-out-of-the-swarming-pool](http://go/gpu-pixel-wrangler-how-to#manually-taking-a-machine-out-of-the-swarming-pool)
  
  
  # Requesting access to the Commit Queue and issue tracker
  
  If you need to request access for a user (or yourself) please ask a project owner to help. Googlers, email
  angle-team@google.com. External users, find a couple owners in the [OWNERS](../OWNERS) file and send a
  request via email.
  
  If you are an ANGLE owner and you want to grant a user access:
  
   * For the Monorail issue tracker, add users via the ["People" administration console][MonorailPeople].
   * For the ANGLE Commit Queue (CQ), grant try-job access via the [Chrome Infra Auth administration console][ChromeAdmin].
   * The admin console also has a separate group for [CQ access][CQAccessAdmin].
  
  [MonorailPeople]: https://bugs.chromium.org/p/angleproject/people/list
  [ChromeAdmin]: https://chrome-infra-auth.appspot.com/auth/groups/project-angle-tryjob-access
  [CQAccessAdmin]: https://chrome-infra-auth.appspot.com/auth/groups/project-angle-committers
  

Download