|
0c77f3ad
|
2021-03-10T15:58:00
|
|
Vulkan: Implement shader buffers descriptor cache.
Implements a descriptor set cache for UBOs, SSBOs, and atomic counter
buffers. Storage Images and framebuffer fetch input attachments are
not yet included. Requires moving the buffer barrier handling into
ContextVk, similarly to how we handle the barriers for Textures.
The packed description key for the descriptors uses a "fast" vector
with a basic minimum size. For most cases of a few buffers this will
fit easily in stack memory, but for larger programs with many buffers
we fit this into heap memory. The key has a large upper bound due to
the high ES 3.2 requirements and the need to index several values such
as the offset and binding size.
We use dynamic offsets for uniform buffers when possible. This ensures
applications like Manhattan 3.1 that use sets of common buffers with
changing offsets hit the cache most of the time.
Because of resource limits we pick at compilation time whether to use
dynamic or static descriptor sets. Mostly this applies to tests that
use a large number of uniform buffers. A future implementation could
be smart and would recompile the program with heuristics to use a
minimal number of dynamic indices.
Reduces the number of descriptor set updates from ~300 -> ~30 per frame
in Manhattan 3.1 and in Asphalt 9 from 900+ to as low as 0 per frame.
Bug: angleproject:5736
Change-Id: I5c2a3881bec90d301dab15cc86c8a70e60674ad7
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2757515
Commit-Queue: Jamie Madill <jmadill@chromium.org>
Reviewed-by: Tim Van Patten <timvp@google.com>
Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org>
|
|
|
89d2a96a
|
2021-03-09T18:37:57
|
|
Vulkan: Add test for UBO descriptor allocations.
This performance counter test verifies that re-binding the same
two buffers repeatedly doesn't allocate new descriptor sets. Currently
the test fails because we don't cache descriptor sets for UBOs.
Covers equivalent code patterns in Asphalt 9.
Reorganizes the perf counters collected for the program objects. Now
they are per-frame reset instead of cumulative. This tracking is now
consistent for the different counter types. In the future we can add
cumulative tracking for all per-object and global perf counters.
Bug: angleproject:5736
Change-Id: I23d04b6453e38af1cf4af7274d24382d136efad3
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2746176
Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org>
Reviewed-by: Tim Van Patten <timvp@google.com>
Commit-Queue: Jamie Madill <jmadill@chromium.org>
|
|
bc82325e
|
2020-11-24T21:58:30
|
|
Fix RedGreenGradient to not output 0 for half the image
The shader directly used v_position.xy as output color, but for half of
the image x and y are negative and the respective channel became zero.
This change makes the shader use v_position.xy*0.5+0.5. Additionally,
the tests that use this shader are changed to mathematically derive the
pixel colors instead of using magic numbers.
Bug: angleproject:5395
Change-Id: Ic11c362c22ca725aa173faa5fd6033a02a4303fd
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2559265
Reviewed-by: Tim Van Patten <timvp@google.com>
Reviewed-by: Jamie Madill <jmadill@chromium.org>
Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org>
|
|
35df41da
|
2020-11-23T21:34:13
|
|
Skip more tests on Linux AMD RX 5500 XT
Remove EGLFeatureControlTest suppression, which crashed due to calling
IsVulkan() without an initialized context.
angle_perftests:
MultisampledRenderToTextureBenchmark.Run/vulkan_multipass_ds
angle_white_box_tests:
VulkanPerformanceCounterTest.RenderToTextureDepthStencilRenderbufferShouldNotLoad
Bug: chromium:1004356, angleproject:5380, chromium:1097750
Change-Id: I38819374242b71fb57c54a9d0fb47b1547dda895
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2556445
Reviewed-by: Yuly Novikov <ynovikov@chromium.org>
Commit-Queue: Yuly Novikov <ynovikov@chromium.org>
|
|
|
46107d3e
|
2020-11-18T22:03:22
|
|
Vulkan: Delay defining D/S content to endRenderPass
Take the following situation:
1. Start RP with D/S undefined: loadOp = DONT_CARE, storeOp = STORE
* At this point, onDepthStencilWrite calls image->onWrite, setting
depth/stencil contents defined.
2. At endRP, observe depth/stencil is not used: storeOp = DONT_CARE
3. Start another RP with D/S: loadOp = LOAD, storeOp = STORE
Because the call to image->onWrite was done at startRP, the contents of
the depth/stencil image is marked as defined, and the next render pass
is loading these data. This change moves image->onWrite to
endRenderPass, and only calls it if storeOp = STORE, taking advantage of
all the opportunistic optimizations that try to set storeOp to another
value.
Bug: angleproject:4836
Change-Id: I9858e5caa6b1f67f841a5c6356e66927356ef469
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2548319
Reviewed-by: Jamie Madill <jmadill@chromium.org>
Reviewed-by: Charlie Lao <cclao@google.com>
Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org>
|
|
89653b97
|
2020-11-11T09:23:18
|
|
Suppress VulkanPerformanceCounterTest::DepthStencilTextureClearAndLoad
The test
VulkanPerformanceCounterTest::DepthStencilTextureClearAndLoad() is flaky
on the Windows AMD bots, so suppressing for now.
Bug: angleproject:5329
Test: VulkanPerformanceCounterTest.DepthStencilTextureClearAndLoad
Change-Id: I546cd18e319c0b73554fb3ce1e70c75be2168015
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2532703
Commit-Queue: Tim Van Patten <timvp@google.com>
Commit-Queue: Jamie Madill <jmadill@chromium.org>
Reviewed-by: Jamie Madill <jmadill@chromium.org>
|
|
|
2d964a47
|
2020-10-30T16:46:30
|
|
Vulkan: Defer clears even if following command is scissored
Take the following scenario:
1. glClear
2. glScissor(half of framebuffer)
3. glDrawArrays
The clear in step 1 is deferred. When FramebufferVk::syncState is
called in step 3, the deferred clear was applied using
vkCmdClearColorImage because the draw call is scissored. This causes
loadOp=LOAD to be used after the clear because the render pass is
started too small (the same size as the scissor).
This change makes scissored operations also take advantage of
loadOp=LOAD with deferred clears. A number of changes are made to this
effect:
- FramebufferVk::syncState no longer limits collecting deferred clears
to no-scissor.
- FramebufferVk::startNewRenderPass automatically expands the render
area to full size if it's clearing any attachment.
- A number of bugs are fixed where FramebufferVk::flushDeferredClears is
called with the scissor area. Instead, flushDeferredClears now
unconditionally uses the complete render area. Note that these bugs
didn't have symptoms as "scissor" and "deferred clears" were mutually
exclusive.
Bug: angleproject:4988
Change-Id: I24fc3d88bf9c8998869b36c863692d0f0acce994
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2511371
Reviewed-by: Jamie Madill <jmadill@chromium.org>
Reviewed-by: Tim Van Patten <timvp@google.com>
Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org>
|
|
|
a0e91016
|
2020-10-30T10:01:36
|
|
Vulkan: Don't break the render pass on scissor change
Prior to this change, the render area was decided when the render pass
was started, and remained fixed. If a small scissor was initially used,
this created a render pass with a small area. If then the scissor
region was expanded, the render pass was broken.
This change instead expands the render area on scissor change to avoid
breaking the render pass. If glInvalidateSubFramebuffer previously
successfully resulted in storeOp=DONT_CARE, this optimization may need
to undo that. As a result, the invalidate area is stored in the render
pass and if the render area grows beyond that, invalidate is undone.
Bug: angleproject:4988
Change-Id: I4e8039dec53a95a193a97cb40db3f71e397568d6
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2508983
Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org>
Reviewed-by: Tim Van Patten <timvp@google.com>
Reviewed-by: Jamie Madill <jmadill@chromium.org>
|
|
|
8f36b846
|
2020-10-29T15:13:55
|
|
Vulkan: Optimize glFramebufferSubInvalidate
If glFramebufferSubInvalidate() is called with an area that covers the
whole framebuffer, behave as if glFramebufferInvalidate() is called.
This allows deferred clears to be removed for example, and attachment
contents to be marked undefined.
Bug: angleproject:4988
Change-Id: Iff3f291ea6c07abccc2740174d0451b432ac5da8
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2508977
Reviewed-by: Charlie Lao <cclao@google.com>
Reviewed-by: Jamie Madill <jmadill@chromium.org>
Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org>
|
|
|
16102e8b
|
2020-10-17T01:15:05
|
|
Reland "Vulkan: Fold deferred clears into current clears"
This reverts commit 37c400146e59f718b516eb58e16fa53c8a88bf21.
Reason for revert: Need to clear the package cache when bisecting.
Original change's description:
> Revert "Vulkan: Fold deferred clears into current clears"
>
> This reverts commit e416c92a81c9ef01d633ec5c05e81c2551b6c5d6.
>
> Reason for revert: Reverted parent:
> https://chromium-review.googlesource.com/c/angle/angle/+/2481612
>
> Original change's description:
> > Vulkan: Fold deferred clears into current clears
> >
> > If there are clears prior to a glClear() call, those clears were
> > flushed (starting a new render pass) and then the clear call's clears
> > would be applied (essentially modifying the loadOps of said render
> > pass).
> >
> > The main downside of the above is that the current glClear() clears
> > don't get a chance to be deferred. This was observed in Chrome which
> > clears an attachment with an emulated format, then switches
> > framebuffers.
> >
> > Additionally, if the render pass had already been started, the deferred
> > clears could have become inlined instead of breaking the render pass.
> > Although, it's unlikely for there to be deferred clears when the render
> > pass is already open.
> >
> > This change first identifies which clears need to go through the draw
> > path (scissored, masked or as workaround for driver bug). It merges the
> > rest of the clears (that don't need the draw path) with the deferred
> > clears. It then checks deferred clears and applies them by either:
> >
> > - vkCmdClearAttachments if mid RP
> > - Start a new render pass and use loadOps, if any draw-based clear needs
> > to follow.
> > - Modify current RP loadOps / defer the clear
> >
> > Afterwards, the draw-based clears are applied.
> >
> > Bug: angleproject:4836
> > Change-Id: Id4992c78983b199734508c9d4bb18ed3195c91ec
> > Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2455167
> > Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org>
> > Reviewed-by: Jamie Madill <jmadill@chromium.org>
> > Reviewed-by: Charlie Lao <cclao@google.com>
>
> TBR=syoussefi@chromium.org,jmadill@chromium.org,cclao@google.com
>
> Change-Id: I85733b3594409df9b96e3d5b34933522c97c42cf
> No-Presubmit: true
> No-Tree-Checks: true
> No-Try: true
> Bug: angleproject:4836
> Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2481613
> Reviewed-by: Tim Van Patten <timvp@google.com>
> Commit-Queue: Tim Van Patten <timvp@google.com>
TBR=timvp@google.com,syoussefi@chromium.org,jmadill@chromium.org,cclao@google.com
# Not skipping CQ checks because this is a reland.
Bug: angleproject:4836
Change-Id: I702cd510f39ee46feab27d4efbf61ae5da10d4e2
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2481856
Reviewed-by: Tim Van Patten <timvp@google.com>
Commit-Queue: Tim Van Patten <timvp@google.com>
|
|
|
37c40014
|
2020-10-16T22:19:20
|
|
Revert "Vulkan: Fold deferred clears into current clears"
This reverts commit e416c92a81c9ef01d633ec5c05e81c2551b6c5d6.
Reason for revert: Reverted parent:
https://chromium-review.googlesource.com/c/angle/angle/+/2481612
Original change's description:
> Vulkan: Fold deferred clears into current clears
>
> If there are clears prior to a glClear() call, those clears were
> flushed (starting a new render pass) and then the clear call's clears
> would be applied (essentially modifying the loadOps of said render
> pass).
>
> The main downside of the above is that the current glClear() clears
> don't get a chance to be deferred. This was observed in Chrome which
> clears an attachment with an emulated format, then switches
> framebuffers.
>
> Additionally, if the render pass had already been started, the deferred
> clears could have become inlined instead of breaking the render pass.
> Although, it's unlikely for there to be deferred clears when the render
> pass is already open.
>
> This change first identifies which clears need to go through the draw
> path (scissored, masked or as workaround for driver bug). It merges the
> rest of the clears (that don't need the draw path) with the deferred
> clears. It then checks deferred clears and applies them by either:
>
> - vkCmdClearAttachments if mid RP
> - Start a new render pass and use loadOps, if any draw-based clear needs
> to follow.
> - Modify current RP loadOps / defer the clear
>
> Afterwards, the draw-based clears are applied.
>
> Bug: angleproject:4836
> Change-Id: Id4992c78983b199734508c9d4bb18ed3195c91ec
> Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2455167
> Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org>
> Reviewed-by: Jamie Madill <jmadill@chromium.org>
> Reviewed-by: Charlie Lao <cclao@google.com>
TBR=syoussefi@chromium.org,jmadill@chromium.org,cclao@google.com
Change-Id: I85733b3594409df9b96e3d5b34933522c97c42cf
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: angleproject:4836
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2481613
Reviewed-by: Tim Van Patten <timvp@google.com>
Commit-Queue: Tim Van Patten <timvp@google.com>
|
|
|
e416c92a
|
2020-10-06T23:29:02
|
|
Vulkan: Fold deferred clears into current clears
If there are clears prior to a glClear() call, those clears were
flushed (starting a new render pass) and then the clear call's clears
would be applied (essentially modifying the loadOps of said render
pass).
The main downside of the above is that the current glClear() clears
don't get a chance to be deferred. This was observed in Chrome which
clears an attachment with an emulated format, then switches
framebuffers.
Additionally, if the render pass had already been started, the deferred
clears could have become inlined instead of breaking the render pass.
Although, it's unlikely for there to be deferred clears when the render
pass is already open.
This change first identifies which clears need to go through the draw
path (scissored, masked or as workaround for driver bug). It merges the
rest of the clears (that don't need the draw path) with the deferred
clears. It then checks deferred clears and applies them by either:
- vkCmdClearAttachments if mid RP
- Start a new render pass and use loadOps, if any draw-based clear needs
to follow.
- Modify current RP loadOps / defer the clear
Afterwards, the draw-based clears are applied.
Bug: angleproject:4836
Change-Id: Id4992c78983b199734508c9d4bb18ed3195c91ec
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2455167
Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org>
Reviewed-by: Jamie Madill <jmadill@chromium.org>
Reviewed-by: Charlie Lao <cclao@google.com>
|
|
|
2aaeb81d
|
2020-10-06T17:20:29
|
|
Vulkan: loadOp=DONT_CARE + unused D/S => invalidate D/S
If depth/stencil is not loaded, and it's not written to during the
render pass, then treat it as if it was invalidated so storeOp can be
set to DONT_CARE and its corresponding resolve attachment (if any)
removed.
This is especially useful for MSRTT as a resolve attachment is added at
the start of render pass, and this optimization will give it a chance to
undo that if depth/stencil was not actually used in the render pass.
This situation can arise for example if a render pass is created for the
sole purpose of clearing color.
This change includes a bug fix for missing depth/stencil on*Access in
the UtilsVk blit/resolve path.
Bug: angleproject:4836
Change-Id: Ifc8eea3e6ffb3eb4bba19f03d1358f151ec69c44
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2453468
Reviewed-by: Charlie Lao <cclao@google.com>
Reviewed-by: Jamie Madill <jmadill@chromium.org>
Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org>
|
|
|
df8f71d1
|
2020-10-09T15:27:28
|
|
Vulkan: Don't break the render pass on scissored clears
clearImmediatelyWithRenderPassOp is removed and the draw path is used
for the scissor. That path was added to avoid creating a large number
of graphics pipelines due to the scissor state. This is now done by
using dynamic state for scissor in the draw path for clear.
Running the following dEQP tests without and with dynamic state for
scissor:
dEQP-GLES3.functional.fragment_ops.depth_stencil.stencil_ops.*
the number of graphics pipelines is reduced from 95392 to 16.
Bug: angleproject:4617
Bug: angleproject:4836
Change-Id: Ib373d8cd23ca2b67e6fd26aa2a1103f281f7e473
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2463985
Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org>
Reviewed-by: Charlie Lao <cclao@google.com>
Reviewed-by: Jamie Madill <jmadill@chromium.org>
|
|
|
68bd685a
|
2020-10-10T22:58:41
|
|
Reland: "4 Vulkan content defined CLs."
Reland "Vulkan: Avoid content restore by detecting no-op stencil"
This relands commit 243d0f899e443cd931c78aba7489382dff79edbb.
Reland "Vulkan: Restore at the end of RP if write-after-invalidate"
This relands commit e5d52ac3b9a00656acdd912ee8cd62dd14784075.
Reland "Vulkan: Invalidate/restore depth/stencil separately."
This relands commit 61fa0878964a796f6d3b3c13bc3a3849403ecdbd.
Reland "Vulkan: Move content-defined tracking to ImageHelper"
This relands commit 2392e6b34c0ddfbfd7b4c3cb67323ba463e11a57.
Reason for revert: Caused crashes in Fuchsia x64 and on ARM.
Reland fixes content defined for external images.
Original CL message:
Content-defined tracking was done in render targets prior to this
change. This had multiple drawbacks:
- When a framebuffer attachment is changed (including the first time
it's set), it's unknown whether the contents of the attachment is
defined.
- Invalidate takes effect at the end of render pass, at which point the
render target objects may be gone. Attachment ImageHelpers are
however correctly tracked.
This change moves content-defined tracking to the ImageHelper itself,
and tracks it per subresource. ImageHelper::onWrite() now receives the
subresource that is being written, and marks it as having defined
content.
A future optimization can make use of this change to
ImageHelper::onWrite to track "dirty" subresources. This can lead to
the removal of unnecessary barriers when same-kind writes are done on
different subresources of the image. See http://anglebug.com/3347#c15
Bug: b/167275320
Bug: angleproject:4836
Bug: angleproject:5159
Change-Id: If5c1ae7152657fd7c94db7d55bea4fb9ddf835ba
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2464825
Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org>
Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org>
|
|
|
d6b1c17b
|
2020-10-10T14:29:15
|
|
Revert 4 Vulkan content defined CLs.
Revert "Vulkan: Avoid content restore by detecting no-op stencil"
This reverts commit 243d0f899e443cd931c78aba7489382dff79edbb.
Revert "Vulkan: Restore at the end of RP if write-after-invalidate"
This reverts commit e5d52ac3b9a00656acdd912ee8cd62dd14784075.
Revert "Vulkan: Invalidate/restore depth/stencil separately."
This reverts commit 61fa0878964a796f6d3b3c13bc3a3849403ecdbd.
Revert "Vulkan: Move content-defined tracking to ImageHelper"
This reverts commit 2392e6b34c0ddfbfd7b4c3cb67323ba463e11a57.
Causes crashes in Fuchsia x64 and on ARM.
Original CL message:
Content-defined tracking was done in render targets prior to this
change. This had multiple drawbacks:
- When a framebuffer attachment is changed (including the first time
it's set), it's unknown whether the contents of the attachment is
defined.
- Invalidate takes effect at the end of render pass, at which point the
render target objects may be gone. Attachment ImageHelpers are
however correctly tracked.
This change moves content-defined tracking to the ImageHelper itself,
and tracks it per subresource. ImageHelper::onWrite() now receives the
subresource that is being written, and marks it as having defined
content.
A future optimization can make use of this change to
ImageHelper::onWrite to track "dirty" subresources. This can lead to
the removal of unnecessary barriers when same-kind writes are done on
different subresources of the image. See http://anglebug.com/3347#c15
Bug: b/167275320
Bug: angleproject:4836
Bug: angleproject:5159
Change-Id: I93d9dfe973caa7ce70aefa46b5b7d04a8637efb3
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2464822
Reviewed-by: Jamie Madill <jmadill@chromium.org>
Commit-Queue: Jamie Madill <jmadill@chromium.org>
|
|
|
243d0f89
|
2020-10-08T21:54:45
|
|
Vulkan: Avoid content restore by detecting no-op stencil
Previously, as long as stencil was enabled, it was considered that it is
also being modified. This caused stencil invalidate to be undone in a
number of situations, such as:
- glEnable(GL_STENCIL_TEST); // with func/ops default
- glDrawArrays();
- glInvalidateFramebuffer([GL_STENCIL_ATTACHMENT]);
- glClear(GL_DEPTH_BUFFER_BIT);
- Close render pass
In the above scenario, invalidation of stencil was undone at the end of
render pass.
In this change, the following cases are considered read-only stencil:
- Func = GL_NEVER, stencilFail = GL_KEEP
- Func = GL_ALWAYS, stencilPassDepth* = GL_KEEP
- stencilFail = GL_KEEP, stencilPassDepth* = GL_KEEP
Note that while the above scenario is fixed for no-op stencil, a similar
issue persists if stencil was not no-op. The reason stencil invalidate
is undone in that case is due to the fact that it's assumed any command
after the invalidate call will be a draw call that outputs to stencil,
but that is not the case with the glClear call in this example.
Bug: angleproject:4836
Change-Id: Ie2ea2d52b7c8ee2394f5456773a7ef434e2b2b16
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2461465
Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org>
Reviewed-by: Jamie Madill <jmadill@chromium.org>
Reviewed-by: Ian Elliott <ianelliott@google.com>
|
|
|
6c1c3bd9
|
2020-10-09T11:46:04
|
|
Vulkan: Clear depth by shader if depthClamp not supported
This will avoid breaking the render pass when clearing depth through
clearWithDraw if the depthClamp Vulkan feature is not present.
Bug: angleproject:4836
Change-Id: I845fd5074dd95f6896da89f9e119ebc5000a5688
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2462719
Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org>
Reviewed-by: Jamie Madill <jmadill@chromium.org>
Reviewed-by: Charlie Lao <cclao@google.com>
|
|
|
e5d52ac3
|
2020-10-08T14:26:22
|
|
Vulkan: Restore at the end of RP if write-after-invalidate
If a depth/stencil attachment is invalidated, but subsequently drawn to
in the same render pass, undo the invalidate when the render pass is
closed. Adapted from
https://chromium-review.googlesource.com/c/angle/angle/+/2386478.
Bug: b/167275320
Bug: angleproject:4836
Change-Id: I17a35bfd692ddc403ceaa6ec44b5c4f16ff9eed6
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2461464
Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org>
Reviewed-by: Jamie Madill <jmadill@chromium.org>
Reviewed-by: Ian Elliott <ianelliott@google.com>
|
|
|
f8070feb
|
2020-10-09T11:03:29
|
|
Vulkan: Use depthClamp to clear depth where available
This will avoid breaking render pass if clearing depth in clearWithDraw.
Bug: angleproject:4836
Change-Id: I50242d1115efc91059923143f6ae5fd25fb3d36f
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2462717
Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org>
Reviewed-by: Charlie Lao <cclao@google.com>
Reviewed-by: Jamie Madill <jmadill@chromium.org>
|
|
|
61fa0878
|
2020-10-08T11:35:48
|
|
Vulkan: Invalidate/restore depth/stencil separately.
Depth/stencil content defined is already tracked separately in the
ImageHelper. This change exposes this tracking from RenderTargetVk.
Bug: b/167275320
Bug: angleproject:4836
Change-Id: Ie6520e7a4ab557eb233c60c6ab0d4a8f8f098bf6
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2462039
Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org>
Reviewed-by: Jamie Madill <jmadill@chromium.org>
Reviewed-by: Ian Elliott <ianelliott@google.com>
|
|
|
5081f89b
|
2020-10-02T01:15:37
|
|
Vulkan: Support invalidate of MSRTT attachments
Invalidate was previously affecting only the storeOp of the color and
depth/stencil attachments. With multisampled-render-to-texture
attachments, the storeOp of the resolve attachments were not being
affected.
This change implements the latter, attempting to remove the attachment
altogether if possible. With MSRTT depth/stencil buffers, this makes
possible the ability to never write depth/stencil data to memory.
Bug: angleproject:4836
Change-Id: I53599e2f4ed6c390dfd03bf226274f6f53f438bb
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2437506
Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org>
Reviewed-by: Tim Van Patten <timvp@google.com>
Reviewed-by: Jamie Madill <jmadill@chromium.org>
|
|
|
887b1346
|
2020-10-02T01:12:01
|
|
Vulkan: Add resolve/unresolve counters
With this change, render-pass-related counters are calculated at render
pass creation time and stored alongside the render pass handle (and
serial) in the render pass cache. On every use, the render pass'
counters are accumulated over the global counters.
Additionally, this change adds MSRTT resolve and unresolve counters to
render pass counters.
Bug: angleproject:4836
Change-Id: If15a789e5a7d66c7ea5a2315bc76fe045ce57491
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2444099
Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org>
Reviewed-by: Tim Van Patten <timvp@google.com>
Reviewed-by: Jamie Madill <jmadill@chromium.org>
|
|
c6dc9d73
|
2020-09-19T20:09:34
|
|
Vulkan: Add a test and fix the bug with draw/invalidate/clear
This adds a test that does draw with depth enabled, then disable depth
test but with depth mask still enabled. Then invalidate framebuffer and
followed by a clear. That clear will go down clearWithCommand path and
should still work and data stored.
Bug: b/169590459
Change-Id: I6dd30d6a1e12ad7820d98fe79445c336cfa3a643
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2422081
Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org>
Reviewed-by: Ian Elliott <ianelliott@google.com>
Commit-Queue: Charlie Lao <cclao@google.com>
|
|
|
a8872e6d
|
2020-10-01T15:54:25
|
|
Skip MSRTT tests on Nvidia/Windows7
These bots don't have VK_EXT_depth_stencil_resolve.
Bug: chromium:1134286
Change-Id: I4cc5a627996139bf03dc3cc334435d08876a0a8a
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2443584
Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org>
Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org>
|
|
|
43163491
|
2020-09-22T11:45:06
|
|
Vulkan: Unresolve depth/stencil MSRTT attachments
Using the same shader that unresolves color, this change allows
depth/stencil to be unresolved as well.
In turn, this allows the depth and stencil loadOp/storeOp of the
implicit multisampled image associated with a
multisampled-render-to-texture renderbuffer to be set to DONT_CARE.
Stencil unresolve depends on VK_EXT_shader_stencil_export. In the
absence of this extension, the stencil aspect is not unresolved and must
continue to use loadOp=LOAD and storeOp=STORE. This is not ideal, but
the expected use-case of depth/stencil MSRTT renderbuffers is that they
get invalidated, so that load and store wouldn't happen in practice.
Bug: angleproject:4836
Change-Id: I9939d1e15e10fa8ed285acdd6fe6edb42c59054f
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2427049
Reviewed-by: Jamie Madill <jmadill@chromium.org>
Reviewed-by: Tim Van Patten <timvp@google.com>
Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org>
|
|
|
d2d9e682
|
2020-09-15T16:06:42
|
|
Vulkan: Use DepthStencilReadOnly when it is read only.
We are tracking depth and stencil read or write during the renderpass.
We can use that to switch to DepthStencilReadOnly layout if both depth
and stencil are not writing. This allows drivers to optimize out the
storeOp for the renderpass.
Bug: b/168953278
Change-Id: Id82e06b4bae1ae8c83d880bb5e58accfa61f8191
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2411336
Commit-Queue: Charlie Lao <cclao@google.com>
Reviewed-by: Tim Van Patten <timvp@google.com>
Reviewed-by: Jamie Madill <jmadill@chromium.org>
|
|
|
f39e0f01
|
2020-09-07T23:07:37
|
|
Vulkan: Use subpass to unresolve render-to-texture attachments
GL_EXT_multisampled_render_to_texture allows singlesampled textures to
be used with multisampled framebuffers in such a way that the final
resolve operation is automatically done. In Vulkan terminology, the
render-to-texture GL attachment is used as a Vulkan subpass resolve
attachment with an implicit (ideally-)lazy-memory multisampled image as
the color attachment.
This extension expects that if the texture is drawn to after the
automatic resolve, the implicit multisampled image would take its
fragment colors from the singlesampled image. In other words, the
opposite of a resolve operation should be automatically performed at the
start of the render pass. This change refers to this operation as
"unresolve".
The goal of this extension is to allow tiling GPUs to always keep
multisampled data on tile memory and only ever load/store singlesampled
data. The latter is achieved by using a subpass resolve attachment and
setting storeOp of the multisampled color attachment to DONT_CARE. This
change achieves the former by using an initial subpass that uses the
resolve attachment as input attachment, draws into the multisampled
color attachment and sets loadOp of said attachment to DONT_CARE.
Bug: angleproject:4881
Change-Id: I99f410530365963567c77a7d62fc9db1500e5e3e
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2397206
Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org>
Reviewed-by: Tim Van Patten <timvp@google.com>
Reviewed-by: Jamie Madill <jmadill@chromium.org>
|
|
a7e03ed7
|
2020-09-21T14:56:00
|
|
Vulkan: Further refine invalidate for depth/stencil
Bug: angleproject:5079
Change-Id: Idc732b1e6729b2776d66c63c3ae2bd94e11bdbb5
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2422684
Reviewed-by: Jamie Madill <jmadill@chromium.org>
Commit-Queue: Ian Elliott <ianelliott@google.com>
|
|
|
edc0d2ee
|
2020-09-15T16:02:05
|
|
Vulkan: Disallow loadOp=LOAD for MSRTT depth/stencil textures
EXT_multisampled_render_buffer2 specifies that depth/stencil textures
are expected to be in a perpetual state of invalidated, except during
rendering. This change makes sure that they never use loadOp=LOAD.
Additionally fixes a bug where clears applied to MSRTT depth/stencil
textures didn't take effect because they were applied to the
multisampled image (since the resolved image was not given to the render
target).
Bug: angleproject:4836
Bug: angleproject:5063
Change-Id: I4506f4de415dca6c222111a1ae62017d2fb1e2b1
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2412848
Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org>
Reviewed-by: Jamie Madill <jmadill@chromium.org>
Reviewed-by: Tim Van Patten <timvp@google.com>
|
|
|
e8d01272
|
2020-09-18T10:34:06
|
|
Vulkan: Additional "invalidate" white-box tests
These are additional tests that were written when trying to refactor
mContentDefined and move it from RenderTargetVk to ImageHelper for
depth/stencil attachments. We still want to land the tests.
Bug: b/167275320
Change-Id: I12f6bf8cece6499e4474a8f114b177626c1aa4d5
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2419117
Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org>
Reviewed-by: Charlie Lao <cclao@google.com>
Commit-Queue: Ian Elliott <ianelliott@google.com>
|
|
|
c03c4490
|
2020-08-28T11:03:20
|
|
Vulkan: Defer glFlush issued in middle of renderpass to endRenderpass
Manhattan is calling glFlush in middle of a renderpass. This CL defers
the flush that issued in the middle of renderpass to the end of
renderpass.
Bug: b/166475273
Change-Id: I6baa3898d5efc456e2205c44e13c64f3d79d1464
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2381942
Commit-Queue: Charlie Lao <cclao@google.com>
Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org>
Reviewed-by: Jamie Madill <jmadill@chromium.org>
Reviewed-by: Tim Van Patten <timvp@google.com>
|
|
|
e4e2a016
|
2020-09-01T11:23:09
|
|
Vulkan: Use color mask to handle draw buffer disabled case
When draw buffers set to GL_NONE, instead of remove the attachment from
renderpass which breaks renderpass, we force vulkan's per buffer color
mask to false while keep the disabled draw buffer attached. This CL also
always create FrameBuffer with all color attachments regardless it is
enabled or not.
Bug: b/167301719
Change-Id: Ice9fca9aacf774a47d13b749f822b222cc050174
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2389007
Reviewed-by: Jamie Madill <jmadill@chromium.org>
Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org>
Reviewed-by: Tim Van Patten <timvp@google.com>
Commit-Queue: Charlie Lao <cclao@google.com>
|
|
|
4a41204d
|
2020-09-02T05:04:33
|
|
Vulkan: Improve invalidate for depth/stencil
Improve state tracking when the depth and/or stencil attachments are
invalidated. Since no draw-time tracking is done, we use the number
of command-buffer commands to determine when an attachment is drawn
to. That allows all cases to be handled for store ops. Still need to
handle mContentDefined at endRP time (we have the data, just not the
plumbing).
Test: angle_white_box_tests --gtest_filter=VulkanPerformanceCounterTest.*Invalidate*/*
Test: angle_deqp_gles3_tests --gtest_filter=dEQP.GLES3/functional_fbo_invalidate_* --use-angle=vulkan
Bug: b/167276207
Change-Id: Iae10857dbb4d43b934c51ad7e400b71ae0db4f55
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2378670
Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org>
Reviewed-by: Charlie Lao <cclao@google.com>
Commit-Queue: Ian Elliott <ianelliott@google.com>
|
|
|
b2944fcf
|
2020-09-02T11:10:00
|
|
Vulkan: Don't break the RP on masked/scissored clears
Bug: b/166809097
Change-Id: Iedd10a6528808e859c5693a2d30c98aca1a1159c
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2390862
Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org>
Reviewed-by: Charlie Lao <cclao@google.com>
Reviewed-by: Tim Van Patten <timvp@google.com>
|
|
|
44df3fa9
|
2020-08-27T14:26:49
|
|
Vulkan: Fix whitebox test to end render pass
Since the RAII-based GLRenderbuffer goes away at the end of the test
function, we should end the render pass before ending the test.
Bug: chromium:1122621
Change-Id: I5aaf2a0b4fd63a38fc5018830ee4e9d7d93f0c81
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2380248
Reviewed-by: Cody Northrop <cnorthrop@google.com>
Reviewed-by: Ian Elliott <ianelliott@google.com>
Reviewed-by: Courtney Goeltzenleuchter <courtneygo@google.com>
Commit-Queue: Ian Elliott <ianelliott@google.com>
|
|
|
c5b9c49c
|
2020-08-25T18:01:29
|
|
Vulkan: Fix optimizeRenderPassForPresent regression.
gfxbench clears the depth buffer right before the swap. Even though the
last draw call that presents the frame didn't ever read or write to
depth, the tracking we added thought this depth write meant we had to
keep the LOAD_OP as CLEAR. Instead we can refine our check to treat
clears specially when enabling the depth-stencil read-only mode instead
of changing how the tracking works for clears. This way the tracking
can not affect other apps that don't use depth-stencil read-only loops.
Also adds a regression test that counts the clears after a swap.
Bug: angleproject:4959
Bug: angleproject:4979
Change-Id: I12ece6474019f7519a467f827110ad817f7d4df7
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2370364
Commit-Queue: Jamie Madill <jmadill@chromium.org>
Reviewed-by: Charlie Lao <cclao@google.com>
Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org>
|
|
|
296d3bfd
|
2020-08-21T10:38:32
|
|
Vulkan: do not end render pass when invalidating
Initially, FramebufferVk::invalidateImpl() was very conservative and
always ended a render pass (if the framebuffer is part of the current
render pass). This adversely affects PUBG Mobile, which invalidates
the depth buffer every frame, causing the render pass to be split.
Test: PUBG MOBILE on Android
Test: angle_white_box_tests --gtest_filter=VulkanPerformanceCounterTest.InvalidatingAndUsingDepthDoesNotBreakRenderPass/*
Test: angle_deqp_gles3_tests --gtest_filter=dEQP.GLES3/functional_fbo_invalidate_* --use-angle=vulkan
Bug: b/163854287
Change-Id: I343dee1db3ebaf039ff92557f9ef25b24bcdcc93
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2352627
Reviewed-by: Jamie Madill <jmadill@chromium.org>
Reviewed-by: Tim Van Patten <timvp@google.com>
Reviewed-by: Charlie Lao <cclao@google.com>
Commit-Queue: Ian Elliott <ianelliott@google.com>
|
|
|
d3e800e9
|
2020-08-15T17:26:04
|
|
Vulkan: Restart RenderPasses in DS read-only mode.
We can combine an initial RenderPass with a read-only RP if the first
RP never writes to depth. We can check the depth-write tracking in
CommandBufferHelper and substitute in a new Framebuffer/RP Desc in this
case as well as issue new layout barriers. We need to disable barrier
merging in this special case.
This reduces the RenderPass count in the Manhattan trace from 15->13.
The performance on the Pixel 4 benchmark goes to ~82% of native for
the on-screen version and ~88% for off-screen. There's also a ~5% bump
in speed for the desktop trace.
Bug: angleproject:4959
Change-Id: I70241824f75eaa1e11b50370f826abc36e91686e
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2358772
Commit-Queue: Jamie Madill <jmadill@chromium.org>
Reviewed-by: Tim Van Patten <timvp@google.com>
Reviewed-by: Charlie Lao <cclao@google.com>
|
|
|
e689d316
|
2020-08-14T22:51:03
|
|
Vulkan: Don't end RenderPass on DS feedback loops.
Instead of always switching the Framebuffer to mask out depth/stencil
loops we now switch the RenderPass to a "read-only" depth/stencil mode.
Reduces the RenderPass count in Manhattan from 18->15. There are still
a couple extra RenderPasses inserted that we can get rid. We can merge
a few RenderPasses by retroactively changing a started RenderPass to
"read-only" when there are no prior recorded depth writes or clears.
Also adds a test to count the number of RenderPasses ANGLE generates in
DS feedback loop situaions.
Bug: angleproject:4959
Change-Id: I1855a45959655fc27ccd47a3469c1c672fc8fd9e
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2357973
Commit-Queue: Jamie Madill <jmadill@chromium.org>
Reviewed-by: Charlie Lao <cclao@google.com>
Reviewed-by: Tim Van Patten <timvp@google.com>
|
|
|
dff47d5f
|
2020-07-14T19:10:12
|
|
Vulkan: Optimize MSAA using subpass resolve attachments
If a user is performing a blit to resolve a multisample color buffer
into a single attachment, ANGLE can use subpass resolve attachments to
resolve directly into the destination buffer as part of the render pass.
This allows the data to remain in tiler memory and reduce the extra
bandwidth required to write the multisampled data back to perform the
copy.
This work also requires restoring/reopening a render pass if it has been
finished already, assuming the finished render pass was started and for
the framebuffer that is the source for the blit command. Other objects
that were created when the render pass was started need to be updated as
well, such as the source FramebufferVk's resolve attachment, the
CommandBufferHelper's vk::Framebuffer and vk::RenderPassDesc, etc.
While this is better than performing vkCmdResolveImage(), there is still
another major part of optimizing MSAA using resolve attachments not
implemented here: discarding the multisampled image rather than writing
it to GMEM, which requires the user to invalidate the read FBO after the
blit.
This CL was verified with AGI to make sure there are no explicit blits
to resolve the multiple sampled image.
Bug: b/159903491
Test: FramebufferTest_ES31.*Blit*
Test: VulkanPerformanceCounterTest_ES31.MultisampleResolveWithBlit
Change-Id: I320a26088d8f614a295e7feec275d71310391806
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2298663
Commit-Queue: Tim Van Patten <timvp@google.com>
Reviewed-by: Jamie Madill <jmadill@chromium.org>
Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org>
Reviewed-by: Charlie Lao <cclao@google.com>
|
|
|
d63bca58
|
2020-08-06T17:37:18
|
|
Vulkan: Add test for RGB texture not break renderpass
This adds a test that will use an empty RGB texture. It should not break
renderpass.
Bug: b/162603208
Change-Id: I90f5aa2fafdbd70c654c9ad0675d6def32b58313
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2338828
Commit-Queue: Charlie Lao <cclao@google.com>
Reviewed-by: Jamie Madill <jmadill@chromium.org>
Reviewed-by: Tim Van Patten <timvp@google.com>
|
|
|
d201ed8b
|
2020-08-02T16:29:35
|
|
Vulkan: Track used Images in RenderPass.
Adding a simple ImageSerial tracking map in our RenderPass allows us
to know when we do or do not need to close the RenderPass on a new
Image access. This simple tracking scheme improves Manhattan
performance by up to 25% on Android. The improved perf comes from
reducing our RenderPass count (23->18 RenderPasses in our capture
scene).
Adds a FastUnorderedSet class to manage the used RP Image serials.
Updates the Query helpers to explicitly flush the RP before inserting
queries.
Bug: angleproject:4911
Change-Id: I0c34fc8e307514ebdf3e81e08d8e5aedb70ebe8f
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2334346
Reviewed-by: Charlie Lao <cclao@google.com>
Reviewed-by: Tim Van Patten <timvp@google.com>
Commit-Queue: Jamie Madill <jmadill@chromium.org>
|
|
|
f9dd2c15
|
2020-08-02T18:03:16
|
|
Vulkan: Accumulate Buffer barriers.
Uses an unordered_map in the CommandBufferHelper to track buffer
reads and writes. Buffer barriers are tracked specially in the
CommandBufferHelper class as a barrier we execute immediately when
we execute the commands into the primary. So when we run into an
incompatible buffer access we must start a new command buffer.
The rules for an incompatible access are:
- when we are reading a buffer, any prior write in the same command
buffer is incompatible.
- when we are writing a buffer, any prior read or write in the same
command buffer is incopatible.
Also adds a regression test using a new performance counter.
Bug: angleproject:4429
Change-Id: I393a4ed87314f955eb998940b877ba76ea15a7b8
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2334091
Reviewed-by: Tim Van Patten <timvp@google.com>
Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org>
Reviewed-by: Charlie Lao <cclao@google.com>
Commit-Queue: Jamie Madill <jmadill@chromium.org>
|
|
|
18dd0c28
|
2020-08-02T17:50:48
|
|
Vulkan: Add command buffer performance counter.
Adds a counter for the secondary command buffers (non-RenderPass).
We'll use this in an upcoming test that validates that ANGLE only
issues a single barrier (CB) in some buffer read/write scenarios.
Also adds a PerfCounters struct.
Bug: angleproject:4429
Change-Id: Iaf75ca89da3d02753897cb4066e2c56db497417e
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2334090
Reviewed-by: Tim Van Patten <timvp@google.com>
Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org>
Commit-Queue: Jamie Madill <jmadill@chromium.org>
|
|
|
25b0de6b
|
2020-08-01T13:45:11
|
|
Vulkan: Squash Texture+ImageView Serial and improve caching.
Previously we regenerated TextureSerial on Texture state changes such
as base/max level changes. This caused ANGLE to update descriptor sets
even though it was using the same image view handles. This change
instead uses an ImageViewSubresourceSerial which includes both a
serial for the ImageView and a 32-bit packed subresource range. The CL
speeds up NBA2k because ANGLE no longer writes new descriptors
for Texture max level changes. Local testing showed up to a 40% speedup.
Also adds a regression test with a counter for the number of descriptor
set writes in a frame.
This change will also be useful in upcoming changes that track Image
serials in the RenderPass.
Bug: angleproject:4911
Change-Id: I66249634aa56288079acf2c0eb8aa3391103533c
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2333396
Commit-Queue: Jamie Madill <jmadill@chromium.org>
Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org>
Reviewed-by: Tim Van Patten <timvp@google.com>
|
|
|
322cc825
|
2020-08-02T17:27:23
|
|
Rename RenderPass test to performance counter test.
This is a better naming for new tests that will check other counters
such as the number of command buffers, barriers, and descriptor set
writes.
Bug: angleproject:4911
Change-Id: I8b2c12f6e043c2833e64fa9627f781e61f8f0f3d
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2334089
Reviewed-by: Jamie Madill <jmadill@chromium.org>
Commit-Queue: Jamie Madill <jmadill@chromium.org>
|