kc3-lang/angle/src/libANGLE/Program.cpp

src/libANGLE/Program.cpp

Log

Author Commit Date CI Message
Chris Dalton a05a0e15 2024-09-25T22:33:36 Validate PLS shaders against context state Add shader introspection for PLS uniforms and validate that they match context state during draw calls. Bug: angleproject:40096838 Change-Id: I76cdf8add03de8f8b0b3e772c15c0087c1d97e98 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/5893962 Commit-Queue: Kenneth Russell <kbr@chromium.org> Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org> Reviewed-by: Kenneth Russell <kbr@chromium.org>
Shahbaz Youssefi 08c1724f 2024-10-11T14:29:00 Vulkan: Support GL_ARM_shader_framebuffer_fetch_depth_stencil Bug: angleproject:352364582 Change-Id: I63fd78314fa7ebccbf366c252e309a9c0f09c8c1 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/5938150 Reviewed-by: Charlie Lao <cclao@google.com> Reviewed-by: Yuxin Hu <yuxinhu@google.com> Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org>
Amirali Abdolrashidi 95e66307 2024-07-08T15:39:58 Resolve compilation during program link resolve Currently, it seems that Program::resolveLink() itself does not resolve shader compilations (Shader::resolveCompile()). Therefore, in case of many shader compilations without them being destroyed, this can result in extra memory usage. * During resolveLinkImpl() of a program, the compilation of the attached shaders are resolved after successful linking in order to save memory. Bug: b/342012929 Change-Id: Iaa6f02a6d2fb3dbd6306660609faa29e73fe82a2 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/5684826 Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org> Reviewed-by: Charlie Lao <cclao@google.com> Commit-Queue: Amirali Abdolrashidi <abdolrashidi@google.com>
Shahbaz Youssefi d193d51b 2024-06-17T22:46:08 Replace issue ids post migration to new issue tracker This change replaces anglebug.com/NNNN links. Bug: None Change-Id: I8ac3aec8d2a8a844b3d7b99fc0a6b2be8da31761 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/5637912 Reviewed-by: Geoff Lang <geofflang@chromium.org> Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org>
Igor Nazarov c41cbb3d 2024-06-03T18:59:27 Fix binary is not cached after isBinaryReady() is true Backend may wait and clear `mPostLinkSubTasks`. The `isBinaryReady()` will check, that there are no tasks, and will skip the caching. This change renamed `cacheProgramBinaryIfNotAlready()` to more general `cacheProgramBinaryIfNecessary()`, because it skips caching if there are post link tasks. The `cacheProgramBinary()` renamed to `cacheProgramBinaryIfNotAlready()`, to better reflect it's behavior. Call `cacheProgramBinaryIfNotAlready()` from `isBinaryReady()` if there no post link tasks to ensure binary is cached. Removed check for empty `mPostLinkSubTasks` before calling executable's wait, because same check already performed internally. The `waitForPostLinkTasks()` is not on hot path and does not require avoiding no-op method call. Bug: angleproject:8297 Change-Id: I2bd78c15231805cda4b62a9889287d8cf780cd1e Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/5593928 Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org> Reviewed-by: Charlie Lao <cclao@google.com> Commit-Queue: Igor Nazarov <i.nazarov@samsung.com>
Shahbaz Youssefi c3a1cae4 2024-04-15T14:58:55 Use angle::SimpleMutex everywhere in libGLESv2 Only cases left that use std::mutex are: - Share group and the context ErrorSet mutexes as they need try_lock() - Anywhere mutexes are used in conjunction with std::condition_variables (as they explicitly require std::mutex) Bug: angleproject:8667 Change-Id: Ib6d68938b0886f9e7c43e023162557990ecfb300 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/5453294 Reviewed-by: Roman Lavrov <romanl@google.com> Reviewed-by: Charlie Lao <cclao@google.com> Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org>
Shahbaz Youssefi f4d3041a 2024-04-13T22:37:11 Remove double-serialization for glGetProgramBinary The applications get the binary length first, and then get the binary itself. Prior to this change, ANGLE was serializing the program binary twice. What's more, if the blob cache is enabled, ANGLE serialized the program binary yet another time for that. With this change, the program binary is serialized only once. If the application queries the program binary, serialization is done the first time needed, is cached and then discarded as soon as the binary itself is returned. Bug: angleproject:7393 Change-Id: If6e3011097ca4d4a1cdcd2dcc23496901196d999 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/5448090 Reviewed-by: Geoff Lang <geofflang@chromium.org> Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org>
Shahbaz Youssefi 24ba48eb 2024-04-13T23:07:11 Do not cache program binary in blob cache redundantly with app If the application has specified the GL_PROGRAM_BINARY_RETRIEVABLE_HINT hint, let the application itself cache the program binary and skip ANGLE's implicit caching. This saves time in serializing the binary, and memory and disk space storing these binaries. If the application sets the hint, but does not actually restore the binary, they will suffer instead. This change also adds a perf warning for applications that retrieve the binary without setting this hint. Bug: angleproject:7393 Change-Id: I05aa880a31fa2fbbd61447c257c990a57137e1e8 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/5448089 Reviewed-by: Geoff Lang <geofflang@chromium.org> Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org>
Shahbaz Youssefi e41286e1 2024-04-10T11:05:09 Vulkan: Fix internal caching missing Since the backend wait for post-link tasks was made possible [1], the front-end may never get a chance to internally cache its binary. This is because the decision to cache the binary was based on the existence of post-link tasks, and the backend clearing that list made the front-end confused. In this change, a specific bool tracks whether binary is cached, and that is checked at various times to make sure the binary cache is eventually done. [1]:https://chromium-review.googlesource.com/c/angle/angle/+/5427001 Bug: angleproject:8297 Change-Id: Iceefd0bca79d570d4910983be3660d6e725684a0 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/5445484 Reviewed-by: Charlie Lao <cclao@google.com> Reviewed-by: mohan maiya <m.maiya@samsung.com> Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org>
Shahbaz Youssefi d9943e44 2024-04-09T23:53:48 Remove Program::syncState The last bit of responsibility still left in Program::syncState was to wait for post-link tasks for the sake of EGLBlobCacheTest tests. A new extension, GL_ANGLE_program_binary_readiness_query is created so that the wait can be done in the test itself. This extension is ultimately useful for applications as well, so they can avoid blocking the CPU by calling glGetProgramBinary prematurely. Bug: angleproject:8297 Change-Id: Ied6b755cb9b060198f82c7948bfd03441435a578 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/5440302 Auto-Submit: Shahbaz Youssefi <syoussefi@chromium.org> Reviewed-by: mohan maiya <m.maiya@samsung.com> Reviewed-by: Charlie Lao <cclao@google.com> Commit-Queue: Charlie Lao <cclao@google.com>
Shahbaz Youssefi 126f8269 2024-04-05T22:00:02 Additional fix for link task worker pool race See [1] for context. While that change fixed the race condition, it was not great that the task held on to the worker pool for so long. This change avoids the race condition as well by releasing the worker pool as soon as the link task has no further use for it. [1]:https://chromium-review.googlesource.com/c/angle/angle/+/5428806 Bug: angleproject:8661 Change-Id: Ibcbad0afe518e649a63dfdf938bd566c8f80e368 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/5431154 Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org> Reviewed-by: Roman Lavrov <romanl@google.com> Commit-Queue: Roman Lavrov <romanl@google.com> Auto-Submit: Shahbaz Youssefi <syoussefi@chromium.org>
Mohan Maiya 924b40dc 2024-04-04T18:44:21 Selectively wait for post-link tasks in the frontend The frontend waits for post-link tasks only for a relink or in syncState when `disableProgramCaching` feature is not enabled. Bug: angleproject:8297 Change-Id: If7a3b8a10a2d01f82fd2bebac5c8f378be56e19e Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/5427001 Reviewed-by: Charlie Lao <cclao@google.com> Commit-Queue: mohan maiya <m.maiya@samsung.com> Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org>
Mohan Maiya ac654931 2024-04-04T15:42:47 ProgramExecutable manages post-link tasks Post-link tasks are now owned and managed by ProgramExecutable. Also, update `LinkTask::link` API so backends have a choice to populate either the linkSubTask or postLinkSubTask containers (but not both). Bug: angleproject:8297 Change-Id: I5dbed6a070efe851fc1376699e9eccd3a21bd03f Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/5406487 Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org> Commit-Queue: mohan maiya <m.maiya@samsung.com> Reviewed-by: Charlie Lao <cclao@google.com>
Mohan Maiya e85bc345 2024-03-29T13:43:46 Account for disableProgramCaching Blob cache tests rely on implicit caching of programs. Account for disableProgramCaching frontend feature when saving and retrieving from cache. Also skip EGLBlobCache tests if the feature is enabled. Bug: angleproject:8297 Change-Id: Ib101eb5712deb7c04dd19c7092f93a6c6b42cd40 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/5408103 Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org> Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org>
Mohan Maiya ee02014d 2024-03-22T11:52:35 Selectively wait for LinkSubTasks Backends have the option to request frontend to run their LinkSubTask post-link. Do not wait for such sub tasks during most glProgram APIs. Note that we do wait for these "post-link sub tasks" incase of a program relink or first use by a draw call. Bug: angleproject:8297 Change-Id: I936fb5a127570f9dfa567ee8cbc02e25c4b70a6b Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/5387064 Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org> Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org>
Shahbaz Youssefi ad013650 2024-03-22T03:42:44 Revert "Rename LinkSubTask -> PostLinkTask" This reverts commit 00eb6edba074a22389b09990ab856adfd417dd64. Reason for revert: Sub tasks are not actually post-link tasks for all the other backends (other than Vulkan), but they are a real part of the link job. Original change's description: > Rename LinkSubTask -> PostLinkTask > > This is a renaming change, no behavior changes are expected. > > Bug: angleproject:8297 > Change-Id: I734c7959f5ed6db2447853cc6f6256e3c8e86213 > Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/5382224 > Commit-Queue: mohan maiya <m.maiya@samsung.com> > Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org> > Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org> Bug: angleproject:8297 Change-Id: Iaebf9d165d810344bfc524042206ca427d270034 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/5386432 Reviewed-by: Shahbaz Youssefi <syoussefi@google.com> Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org> Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com> Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org>
Mohan Maiya 00eb6edb 2024-03-20T06:49:38 Rename LinkSubTask -> PostLinkTask This is a renaming change, no behavior changes are expected. Bug: angleproject:8297 Change-Id: I734c7959f5ed6db2447853cc6f6256e3c8e86213 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/5382224 Commit-Queue: mohan maiya <m.maiya@samsung.com> Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org> Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org>
Shahbaz Youssefi 8346addb 2024-02-06T15:40:31 Contain X11 includes and free usage of common terms This change undoes workarounds where some terms were avoided so there is no clash with X11 (such as Success, Bool and None). In particular, this helps us make sure we never include the X11 headers in such an unconstrained manner as to clash with our code. Bug: angleproject:8520 Change-Id: I53d9657c5a33164064d2c80a206b96fd52f607f1 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/5273491 Reviewed-by: Geoff Lang <geofflang@chromium.org> Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org> Reviewed-by: Liza Burakova <liza@chromium.org>
Shahbaz Youssefi ecc35205 2024-01-25T23:58:25 Move uniform block dirty bits to State When glUniformBlockBinding changes the mapping from a program uniform block to a buffer binding, all contexts in the share group need to reprocess the affected block index. Prior to this change, the dirty bits that indicated which blocks have their mapping redefined were placed in the program executable, and were reset by the first context that processed them. As a result, the other contexts in the share group where not aware of such modifications. Similarly, when a buffer changed in one context, the mapped program blocks were marked dirty, with similar cross-context issues. In this change, the dirty bits are moved to State, so every context would react to these changes. Bug: angleproject:8493 Change-Id: I5712002224cbc4a576bf2ac46e8e75f26ebc5b2a Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/5238991 Reviewed-by: Geoff Lang <geofflang@chromium.org> Reviewed-by: Charlie Lao <cclao@google.com> Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org>
Shahbaz Youssefi 0c4d6446 2024-01-24T10:38:45 Rework uniform block <-> uniform buffer mapping In GLES, the shader declares which buffer binding a block (uniform, storage or atomic counter) is bound to. For example: layout(binding = 1) uniform ubo0 { ... }; layout(binding = 2) uniform ubo1 { ... }; layout(binding = 1) uniform ubo2 { ... }; In the above, ubo0 and ubo2 use data from the buffer bound to index 2 (through glBindBufferRange), while ubo1 uses data from the buffer bound to index 1. For uniform blocks in particular, omitting the binding is allowed, in which case it is implicitly bound to buffer 0. GLES allows uniform blocks (and only uniform blocks) to remap their bindings through calls to glUniformBlockBinding. This means that the mapping of uniform blocks in the program (ubo0, ubo1, ubo2) to the buffer bindings is not constant. For storage blocks and atomic counter buffers, this binding _is_ constant and is determined at link time. At link time, the mapping of blocks to buffers is determined based on values specified in the shaders. This info is stored was stored in gl::InterfaceBlock::binding (for UBOs and SSBOs), and gl::AtomicCounterBuffer::binding. For clarity, this change renames these members to ...::inShaderBinding. When glUniformBlockBinding is called, the mapping is updated. Prior to this change, gl::InterfaceBlock::binding was directly updated, trumping the mapping determined at link time. A bug here was that after a call to glProgramBinary, GL expects the mappings to reset to their original link-time values, but instead ANGLE restored the mappings to what was configured at the time the binary was retrieved. This change tracks the uniform block -> buffer binding mapping separately from the link results so that the original values can be restored during glProgramBinary. In the process, the support data structures for tracking this mapping are moved to ProgramExecutable and the algorithms are simplified. Program Pipeline Objects maintain this mapping identically to Programs and no longer require a special and more costly path when a buffer state changes. This change prepares for but does not yet fix the more fundamental bug that the dirty bits are tracked in the program executable instead of the context state, which makes changes not propagate to all contexts correctly. Bug: angleproject:8493 Change-Id: Ib0999f49be24db06ebe9a4917d06b90af899611e Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/5235883 Reviewed-by: Geoff Lang <geofflang@chromium.org> Reviewed-by: Charlie Lao <cclao@google.com> Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org>
Shahbaz Youssefi b007c74d 2024-01-23T14:17:54 GL: Separate dirty bits leading to glUniformBlockBinding The GL backend is special in that it needs to make actual calls (native glUniformBlockBinding) in response to (application) glUniformBlockBinding calls. The other backends just remap the bindings based on that information when creating descriptor sets. Previously, an optimization to track which bindings have changed used the same dirty bits that were used to signify when the GL backend needs to make these native calls. That ended up as a source of bugs. In a previous change [1], the context DIRTY_BIT_UNIFORM_BUFFER_BINDINGS is set when these mappings change, which fixes some of these issues. That change obviates the need for an actual backend sync of programs, except for GL programs that need to make these native calls. This change splits the dirty bits maintained for the purposes of the GL backend, moves them to that backend and removes the program backend sync. [1]: https://chromium-review.googlesource.com/c/angle/angle/+/5228599 Bug: angleproject:8493 Bug: b/318806125 Change-Id: I73c6514e88a116f1cd701cb06da0d8c38f07f7f6 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/5230137 Reviewed-by: Geoff Lang <geofflang@chromium.org> Reviewed-by: Charlie Lao <cclao@google.com> Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org>
Shahbaz Youssefi c6fbf93d 2024-01-19T09:57:12 Vulkan: Fix input attachments leaking into uniform list To communicate the existence of input attachments added to the shader, the translator was adding `ShaderVariable`s for each to the list of uniforms exported from the shader. This was incorrect, as this list is visible to the application through `glGetActiveUniform`. Additionally, this was unnecessarily causing these uniforms to go through program link. Reserving SPIR-V ids for these uniforms, all that is needed from the translator is the mere existence of these input attachments. This change removes the addition of uniforms, and instead exports a bitset. Elsewhere, that bitset is consulted and reserved SPIR-V ids are used. Bug: b/320563594 Bug: angleproject:5792 Change-Id: Id93846cbc3996248f391fd2d5a65af1e48d6d46e Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/5215089 Reviewed-by: mohan maiya <m.maiya@samsung.com> Reviewed-by: Charlie Lao <cclao@google.com> Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org>
Shahbaz Youssefi 175514c6 2024-01-12T14:43:33 Translator: Bundle metadata flags coming out of the translator In preparation for a follow up change that adds more such metadata. Bundling them together makes it convenient to retrieve, save and load all those flags. Bug: b/320563594 Change-Id: I4f95b32acfb0842cc5d9e72c1788a827bee2c760 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/5209450 Reviewed-by: Amirali Abdolrashidi <abdolrashidi@google.com> Reviewed-by: Charlie Lao <cclao@google.com> Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org>
Shahbaz Youssefi 572323cc 2024-01-11T16:20:02 Fix program link after backend rejects program binary If ANGLE believes the program binary is fine, it populates the program executable. If the backend then rejects the program binary, the executable was not reset. After the rejection, ANGLE proceeds to redo the program link, in which case it fails in various ways (ASSERT failures, incorrect data etc) as it tries to accumulate info on top of the previous executable. Bug: angleproject:8471 Change-Id: Ia4d626f5f9643c39a81062da3d5d58aa4c6be762 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/5189152 Reviewed-by: Quyen Le <lehoangquyen@chromium.org> Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org>
Shahbaz Youssefi a1143857 2023-12-18T15:24:42 Fix UBO dirty bits vs PPOs This change fixes propagation of UBO dirty bits (such as through glUniformBlockBinding and glBindBufferRange) to program pipeline objects. Since PPOs concatenate the attached programs' UBOs in a list, a map of program UBO indices to PPO UBO indices is introduced to offset these dirty bits appropriately. Additionally, when the program's executable's buffer bindings change (through glUniformBlockBinding), a notification is send to the PPO to update its executable's buffer binding accordingly (which is otherwise only updated during PPO link). Bug: angleproject:8462 Change-Id: I4965ae23e6fc6cac0842e1643755e42e95d3d5cc Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/5131418 Reviewed-by: Charlie Lao <cclao@google.com> Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org> Reviewed-by: Cody Northrop <cnorthrop@google.com>
Shahbaz Youssefi 3680a5dc 2023-11-17T13:51:07 Vulkan: Let program warmup continue passed link The warmup task does not actually affect the link results, so there is no reason to wait for it when the application queries the link status. This change allows the warm up task to continue in parallel until the program is used at draw time. This allows the warm up to be more efficient when the link itself is not parallelized. For applications that create programs in the middle of every frame, it's still likely best to disable warm up (as the following immediate draw will already effectively do the warm up). Note that currently the warm up code in the Vulkan backend is not completely thread-safe, and so the program still blocks on that task before the first draw can happen (or the program is modified in any way). Bug: angleproject:8417 Change-Id: I0877fef39a0585c3279e32699ce817d4643d7cd6 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/5037538 Reviewed-by: Charlie Lao <cclao@google.com> Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org> Reviewed-by: Geoff Lang <geofflang@chromium.org>
Shahbaz Youssefi 6f4f5e39 2023-11-18T22:15:49 Fix checking result of link sub tasks Bug: angleproject:8297 Change-Id: I143a26be6e7340a4a2d2b917bbe4dd9dd474f7af Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/5042346 Reviewed-by: Geoff Lang <geofflang@chromium.org> Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org>
Shahbaz Youssefi aa3bebf9 2023-11-18T22:15:11 GL: Fix parallel link query of in-progress link Bug: angleproject:8297 Change-Id: Iab2f12467a2c567c945742435939057a40642951 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/5042345 Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org> Auto-Submit: Shahbaz Youssefi <syoussefi@chromium.org> Reviewed-by: Geoff Lang <geofflang@chromium.org>
Shahbaz Youssefi dd8432b5 2023-11-14T14:39:45 Remove angle::Result::Incomplete from shader/program paths angle::Result is not an error code, and having Incomplete made it very unclear what the purpose of this class is. A follow up will remove it entirely. Bug: angleproject:8414 Change-Id: Ica8271b9f7d8868671c7658161e50a53ef23c681 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/5028091 Reviewed-by: Geoff Lang <geofflang@chromium.org> Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org>
Shahbaz Youssefi ade3dacd 2023-11-06T21:56:41 Do compile/link unlocked if not threaded (but thread-safe) If GL_KHR_parallel_shader_compile is not supported, or it is not used to do threaded compilation and link, this change lets the compile and link jobs be done after releasing the share group lock. With multithreaded/multi-context applications, this allows the other context (typically the main context) to make progress in the meantime. A typical scenario where this optimization matters is games seamlessly loading a new area of the game and performing compilation and link in a separate context. Before this change, the game would stutter as the compile/link jobs prevent the main thread from drawing anything. With this change, the hitching is removed. Bug: angleproject:8297 Change-Id: I702d84324a7442561b49677bf42c16d650304313 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/5006640 Reviewed-by: Charlie Lao <cclao@google.com> Reviewed-by: Geoff Lang <geofflang@chromium.org> Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org>
Shahbaz Youssefi 93b97a59 2023-11-03T22:07:23 Make link job directly wait on compile job Previously, program link waited on the compile job on the calling thread before launching the link job. As a result, sequences of intermixed compile and link would get largely serialized as such: Main Thread Thread 1 Thread 2 Thread 3 Thread 4 Compile -------> Compile Compile -----------|----------> Compile Link | | Wait | | | | | |<--------------/--------------/ \------------------------------------------> Link Compile -------> Compile | Compile -----------|----------> Compile | Link | | | Wait | | | | | | | |<--------------/--------------/ | \---------------------------------------------|-----------> Link Compile -------> Compile | | Compile -----------|----------> Compile | | Link | | | | Wait | | | | | | | | | ... With this change, the main thread no longer waits for compilation to finish. It's the link job itself that does the waiting. This allows the main thread to go through Compile and Link commands without blocking, generating as many jobs as needed. The above scenario therefore becomes: Main T1 T2 T3 T4 T5 T6 T7 T8 T9 C ----> C C ------|----> C L ------|------|----> L C ------|------|-------W---> C C ------|------|-------|-----|----> C L ------|------|-------|-----|------|----> L C ------|------|-------|-----|------|-------W---> C C ------|------|-------|-----|------|-------|-----|----> C L ------|------|-------|-----|------|-------|-----|------|----> L . \-----\------>/ | | | | | W . | \-----\------>/ | | | . | | \-----\------>/ . | | | . | | | This greatly improves the amount of parallelism compile and link jobs get. The careful observer may note that the link job being blocked on the compile job is now wasting a thread from the thread pool. While this change is strictly an improvement, parallelism can be further improved if the link job is just not assigned to a thread until the corresponding compile jobs are finished. This is currently not possible, but may be if: - Instead of a thread pool, the operating system's FIFO scheduler is used. Then the operating system would automatically put blocking tasks to sleep and pick up another task. This has the downside of requiring threads to be created for each task. - The thread pool work scheduler is enhanced to be made aware of relationship between tasks and avoid scheduling jobs whose dependencies are not yet met. Alternatively, the number of threads in the pool can be increased by 30% and hope for the best. Bug: angleproject:8297 Change-Id: If4e6540ade47558a10cfab55e2286f073b904928 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/5006874 Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org> Reviewed-by: Geoff Lang <geofflang@chromium.org> Reviewed-by: Charlie Lao <cclao@google.com>
Shahbaz Youssefi c114fc26 2023-08-28T15:40:26 Reland: Verify shader link correctness after looking up in program cache Since the program cache takes shader sources into account, if the cache is hit the shaders should have been correct for link. Bug: angleproject:8297 Change-Id: Iccadc63a976689d7fa06d0a5ccfe81a594c424e8 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/5003236 Reviewed-by: Geoff Lang <geofflang@chromium.org> Reviewed-by: Charlie Lao <cclao@google.com> Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org>
Cody Northrop 1ea49a22 2023-10-13T11:28:41 Move uniform dirty bits to ProgramExecutable Rather than try to funnel them through Program and ProgramPipeline to the executable in the backend, just move them to ProgramExecutable in the front end. This fixes Dota Underlords at the same time due to not needing to set the Program dirty to propagate bits. Test: Dota Underlords Test: ProgramPipelineTest31.ProgramPipelineBindBufferRange Bug: b/299532942 Change-Id: Ic73c45608e22f89ca400ebf684f8cd287ed2f43a Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4922969 Reviewed-by: Yiwei Zhang <zzyiwei@chromium.org> Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org> Commit-Queue: Cody Northrop <cnorthrop@google.com>
Roman Lavrov 34c8778b 2023-09-26T13:45:06 Use atomic counters early in perf warning macros Before this CL, snprintf was called repeatedly to format the warning message which was then discarded after 4 logs. snprintf showed up in profiling at ~2% and this CL appears to yield an ~8% power improvement in one of the traces (egypt_1500). A mutex was previously used to avoid the race condition on the static sRepeatCount variable. This CL avoids the need for that by using static atomics instead. Also updated the Debug macro to use the VK macro vararg approach so that formatting only happens when the message is actually logged. Bug: b/302112423 Change-Id: Ia8a18361cfb5a9f2aa19ff939499754ba861efb7 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4886388 Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org> Reviewed-by: Charlie Lao <cclao@google.com> Commit-Queue: Roman Lavrov <romanl@google.com>
Shahbaz Youssefi 97a48912 2023-09-26T10:31:58 Turn is-link-thread-safe feature into a positive condition Chrome's --disable-gpu-driver-bug-workarounds currently sets every feature to false, which breaks the GL backend. Bug: angleproject:8297 Change-Id: I284d0699e356d7c1a362eb992cdc0d052f9ea7c2 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4887598 Reviewed-by: Geoff Lang <geofflang@chromium.org> Reviewed-by: Roman Lavrov <romanl@google.com> Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org>
Charlie Lao 8fcd4a50 2023-09-19T14:08:14 Cleanup POD struct usage to make them more consistent In recent months, I have made many CLs to wrap various data structures into a trivially copy-able struct. Some of them uses different terminology. This CL replaces all of these to use a consistent name "pod" in the struct, and "mPod" in a class. This CL also turns ActiveVariable methods into macros to remove the code duplication. This CL also moves ProgramInput/ProgramOutput struct implementations from Program.cpp to ProgramExecutable.cpp Bug: b/275102061 Change-Id: Ia2a4210a9ea633f3d323bebe674ee74f8b90b363 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4877335 Commit-Queue: Charlie Lao <cclao@google.com> Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org> Reviewed-by: Roman Lavrov <romanl@google.com>
Charlie Lao 9ca025d2 2023-09-18T15:50:48 Flatten BufferVariable/ShaderVariableBuffer/InterfaceBlock struct InterfaceBlock inherits from ShaderVariableBuffer, ShaderVariableBuffer is not a trivially copyable struct, this made InterfaceBlock not trivially copyable. InterfaceBlock is being used by some app traces for uniform blocks. BufferVariable inherits from sh::ShaderVariable which is very complicated and not trivially copyable. This CL flattens all of these three structs to simple structs without inheritance, and wraps all trivially copyable data into one POD struct, thus load/save are cheaper. Bug: b/275102061 Change-Id: I96f89176ce3d3131cb1d3ea3280c3c36c257560f Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4874610 Commit-Queue: Charlie Lao <cclao@google.com> Reviewed-by: Roman Lavrov <romanl@google.com> Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org>
Shahbaz Youssefi eb0d5997 2023-09-15T16:41:13 Move set/get uniform machinery to ProgramExecutable This is done because some uniforms are internally added by the compiler (draw ID, base vertex, and base instance) and are automatically set **on the installed executable**. This change fixes scenarios where a draw is done after a program has failed a relink, and therefore is unable to correctly set the uniforms (as it does not have access to the executable that is installed). It also fixes draws that use those uniforms in a PPO. Bug: angleproject:8297 Change-Id: Id74b4984b88aa09b5b81be1c91412d6c91711136 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4864693 Reviewed-by: Charlie Lao <cclao@google.com> Reviewed-by: Yuxin Hu <yuxinhu@google.com> Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org>
Shahbaz Youssefi 936694e3 2023-09-15T16:03:25 Cache isSeparable in the executable For convenience, particularly for the next CL. Bug: angleproject:8297 Change-Id: I55690aecf3936a51a2a2163d7c354a710b81d7f8 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4864069 Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org> Reviewed-by: Yuxin Hu <yuxinhu@google.com> Reviewed-by: Charlie Lao <cclao@google.com> Auto-Submit: Shahbaz Youssefi <syoussefi@chromium.org>
Charlie Lao 65220256 2023-09-18T12:29:59 Add ProgramOutput struct for ProgramExecutable::mOutputVariables Right now ProgramExecutable::mOutputVariables is a vector of sh::ShaderVariable. ShaderVariable. itself is not a POD struct and can't memcpy. And most of variables are not needed for mOutputVariables. This CL adds a custom struct for mOutputVariables so that we only store what we actually needed and data can be memcpy. Bug: b/275102061 Change-Id: I045d0618b6dab5f8d58afe40e55147d12987cf61 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4862977 Commit-Queue: Charlie Lao <cclao@google.com> Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org>
Shahbaz Youssefi 29561184 2023-09-06T22:08:33 Remove forward-to-executable helpers from program This helps avoid accidental usage of the executable that is in the program instead of the installed executable in the GL context. The program's executable is still accessed in specific cases of: - During link - GL program queries Bug: angleproject:8297 Change-Id: I40a956e740944f2ecfbf6e4a3060aac08c21f7f3 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4864448 Commit-Queue: Yuxin Hu <yuxinhu@google.com> Reviewed-by: Charlie Lao <cclao@google.com> Reviewed-by: Yuxin Hu <yuxinhu@google.com> Auto-Submit: Shahbaz Youssefi <syoussefi@chromium.org>
Shahbaz Youssefi 68bfa1ed 2023-08-22T22:02:15 Support for link to be entirely parallelized The link job is split as such: - Front-end link - Back-end link - Independent back-end link subtasks (typically native driver compile jobs) - Post-link finalization Each step depends on the previous. These steps are executed as such: 1. Program::link calls into ProgramImpl::link - ProgramImpl::link runs whatever needs the Context, such as releasing resources - ProgramImpl::link returns a LinkTask 2. Program::link implements a closure that calls the front-end link and passes the results to the backend's LinkTask. 3. The LinkTask potentially returns a set of LinkSubTasks to be scheduled by the worker pool 4. Once the link is resolved, the post-link finalization is run In the above, steps 1 and 4 are done under the share group lock. Steps 2 and 3 can be done in threads or without holding the share group lock if the backend supports it. Step 2 is not yet made independent of the Context on some backends, and a frontend feature is used to make that step either run on the main thread or as a worker thread. Bug: angleproject:8297 Change-Id: I12f1e6bbaf365543dfcac969e166e0b5aa622104 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4808191 Reviewed-by: Charlie Lao <cclao@google.com> Reviewed-by: Geoff Lang <geofflang@chromium.org> Reviewed-by: Shahbaz Youssefi <syoussefi@google.com> Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org>
Charlie Lao c5d19d76 2023-09-07T09:58:24 Reland "Load mUniformLocations using loadVector" This is a reland of commit 6f526b8f52d17bf89082a901f06df6edb75a7dbd Original change's description: > Load mUniformLocations using loadVector > > Bug: b/275102061 > Change-Id: I1da5bdd6bf0ec40cd877c2274a8fe1ee0b11267a > Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4849551 > Reviewed-by: Roman Lavrov <romanl@google.com> > Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org> > Commit-Queue: Charlie Lao <cclao@google.com> Bug: b/275102061 Change-Id: If501db97a37b00104a9d16fc40200f772ffcd2fc Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4856749 Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org> Commit-Queue: Charlie Lao <cclao@google.com>
Charlie Lao 57761d85 2023-09-08T10:02:48 Attempt to fix the MSan failure on chromium bots struct BlockMemberInfo was not tightly packed. This CL makes it packed. Bug: b/296433003 Change-Id: I38da3e463cf304028236e651b6b4e334ce9220e2 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4851063 Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org> Commit-Queue: Charlie Lao <cclao@google.com>
Shahbaz Youssefi fc7a2853 2023-09-07T17:10:56 Revert "Verify shader link correctness after looking up in program cache" This reverts commit 60b56591dee59bc0bc770577f43d90be4b18863c. Reason for revert: Flakiness if the attached shaders are not compiled Original change's description: > Verify shader link correctness after looking up in program cache > > Since the program cache takes shader sources into account, if the cache > is hit the shaders should have been correct for link. > > Bug: angleproject:8297 > Change-Id: I115c7eb807ed620d30b9e30669c99fffb5c7912d > Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4817830 > Reviewed-by: Geoff Lang <geofflang@chromium.org> > Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org> Bug: angleproject:8297 Change-Id: Icf4058e4bea471f097caed950b18f690d8d0456d No-Presubmit: true No-Tree-Checks: true No-Try: true Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4846389 Commit-Queue: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com> Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com> Auto-Submit: Shahbaz Youssefi <syoussefi@chromium.org>
Shahbaz Youssefi 60b56591 2023-08-28T15:40:26 Verify shader link correctness after looking up in program cache Since the program cache takes shader sources into account, if the cache is hit the shaders should have been correct for link. Bug: angleproject:8297 Change-Id: I115c7eb807ed620d30b9e30669c99fffb5c7912d Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4817830 Reviewed-by: Geoff Lang <geofflang@chromium.org> Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org>
Shahbaz Youssefi 7b0bb0f6 2023-09-01T13:52:28 Properly "install" program executables According to GL: - The program has an executable - The executable is overwritten during link. - After a failed link, queries of the executable may return half-linked information - On glUseProgram, the executable is installed in the context - On glUseProgramStages, the executable is installed in the program pipeline - After a successful link, the executable is updated wherever the previous executable of the program was installed. This change implements exactly the above: - The program's and the program pipeline's executables are now shared_ptr. References to an executable in the context and PPO are also through a shared_ptr. Installing an executable thus translates to sharing the executable. - The context and PPOs are made to not reference the program directly, but work solely through the executable. As a result, the program is free to create a new executable for link. With this change, the link job will be free to modify the executable as necessary because that will not be accessed until the link is done. Note that previous changes made the backend executable accessed through the frontend one, and moved all link results to the frontend and backend executables as appropriate. Bug: angleproject:6358 Bug: angleproject:8297 Change-Id: Ie636b23ff7420ad284d18b525ec4f5fb559dd9d1 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4823089 Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org> Reviewed-by: Geoff Lang <geofflang@chromium.org> Reviewed-by: Charlie Lao <cclao@google.com>
Shahbaz Youssefi ebf1e716 2023-08-31T17:19:44 Cache transform feedback varying names in the executable Currently, ANGLE actually does a full link of the programs inside PPOs. This was never the intention of the spec (hence why an explicit link doesn't exist). During this link operation, the transform feedback varying names are used, and they are retrieved from the program itself. This is not correct, because the transform feedback varyings may have changed, the program may have failed to relink, and the program pipeline is expected to continue functioning using the "installed" executable. Bug: angleproject:5486 Bug: angleproject:8297 Change-Id: I583dbd2abcc51e8536b4c460b92211bdddebda16 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4834055 Reviewed-by: Charlie Lao <cclao@google.com> Reviewed-by: Yuxin Hu <yuxinhu@google.com> Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org>
Shahbaz Youssefi 33be2590 2023-08-30T21:58:54 Move transform feedback buffer mode to program state This is link input and should persist between relinks. A copy of it is still left in ProgramExecutable as it is used later after the executable is installed (and the program might get relinked; a future change reworks things so that the installed executable and the originating program are independent). Bug: angleproject:8297 Change-Id: Ifba6ac18ea09fd804b89d8fdb14930ef3d839cf4 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4827764 Reviewed-by: Charlie Lao <cclao@google.com> Reviewed-by: Yuxin Hu <yuxinhu@google.com> Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org>
Shahbaz Youssefi 6ff209bc 2023-08-30T00:02:30 Clean up InfoLog usage during link The info log doesn't really belong to ProgramExecutable; it belongs to ProgramState. However, it is placed there for convenience since many functions access it. This change cleans up usage of InfoLog so the one in ProgramExecutable is consistently used, but also that is turned into a reference to ProgramState's InfoLog. This is necessary for a follow up change that restores the previous executable on link failure (and would thus otherwise lose the info log of the failing link). Bug: angleproject:8297 Change-Id: I088408e3fce9ebb35b1ec4ad3dc599bdb90bf5c5 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4825624 Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org> Reviewed-by: Charlie Lao <cclao@google.com> Reviewed-by: Yuxin Hu <yuxinhu@google.com>
Shahbaz Youssefi b9e9c58b 2023-08-28T15:25:59 Move pre-link program state to ProgramState Bug: angleproject:8297 Change-Id: I2522bbe6250a0527112fc8161f62c94f2457555b Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4817828 Reviewed-by: Charlie Lao <cclao@google.com> Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org>
Shahbaz Youssefi d9672ada 2023-08-25T11:37:09 Move left over link state to ProgramExecutable Previously, only things that needed sharing with ProgramPipeline was moved to ProgramExecutable, and in particular only state that the Vulkan backend needed to access. In truth, everything that's a result of link needs moving. Bug: angleproject:8297 Change-Id: I1ca01c5dedbfc62ddcfb4ef945336ceb8ad0f1c4 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4812044 Reviewed-by: Charlie Lao <cclao@google.com> Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org> Reviewed-by: Yuxin Hu <yuxinhu@google.com>
Shahbaz Youssefi 3822ea3a 2023-08-24T22:42:43 D3D: Move program state to ProgramExecutableD3D Bug: angleproject:8297 Change-Id: Ieead87d2f0ebe5937f262f598443a00504ea1492 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4812139 Reviewed-by: Geoff Lang <geofflang@chromium.org> Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org>
Shahbaz Youssefi bb135f0e 2023-08-24T15:29:11 Make ProgramExecutableImpl managed by ProgramExecutable This change allows both parts of the program executable to be safely backed up and swapped on link. Bug: angleproject:8297 Change-Id: I17e4b6c05e4e481a66a227d6047dbf943d2c2603 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4812138 Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org> Reviewed-by: Charlie Lao <cclao@google.com> Reviewed-by: Geoff Lang <geofflang@chromium.org>
Shahbaz Youssefi c8ec8739 2023-08-22T11:30:08 Frontend: Remove link job dependence to context The part of link that needs the context is moved up. Usage of gl::Context is eliminated from the rest of the link (whatever is done in the front-end). Bug: angleproject:8297 Change-Id: Ifa71d2b2c0c0bc7c0c0b7ee89e1cbb203c3018cc Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4803109 Reviewed-by: Yuxin Hu <yuxinhu@google.com> Reviewed-by: Charlie Lao <cclao@google.com> Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org>
Shahbaz Youssefi 571b4cdb 2023-08-14T16:55:28 Vulkan: Move pipeline/desc-set layout creation to link job The pipeline and desc-set layout caches are consequently made thread-safe. The reference counter on the layouts are also made atomic. With this change, practically all of the link in the Vulkan backend is moved to the link job. Bug: angleproject:8297 Change-Id: Iba694ece5fc5510d34cce2c34441ae08ca5bb646 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4774787 Reviewed-by: Charlie Lao <cclao@google.com> Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org>
Charlie Lao 03f9dff6 2023-08-21T16:57:12 Separate out std::vector<GLuint> boundTextureUnits from SamplerBinding SamplerBinding struct is trivial copy-able except the std::vector<GLuint> boundTextureUnits. This CL moves boundTextureUnits out of the SamplerBinding struct and becomes a vector in ProgramExecutable. This means all of sampler binding's texture units will be stored in one vector and SamplerBinding will remember where its own texture units is stored in that vector by keeping textureUnitsStartIndex and textureUnitsCount. In other word, SamplerBinding::boundTextureUnits (before this CL) is equivalent to mSamplerBoundTextureUnits[SamplerBinding.textureUnitsStartIndex], ..., mSamplerBoundTextureUnits[SamplerBinding.textureUnitsStartIndex + SamplerBinding.textureUnitsCount - 1] after this CL. With this, ProgramExecutable::mSamplerBindings are load/stored with one readBytes/writeBytes call since it is trivially copyable. Bug: b/275102061 Change-Id: I0974cf940875ecbcf655b4469b3bbc910717f1ec Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4798195 Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org> Reviewed-by: Roman Lavrov <romanl@google.com> Commit-Queue: Charlie Lao <cclao@google.com>
Shahbaz Youssefi 68f2c10c 2023-08-15T22:47:32 Remove program executable backup during link This was added in https://chromium-review.googlesource.com/c/angle/angle/+/2181450/30 to support the case where a program fails to relink, but needs to still be usable. However, this does not seem to be an issue anymore. New tests are specifically added for this, and they, along with every other test, pass. If this needs to be reintroduced, it needs to be rethought. It does not play well with parallel link as it changes the executable pointer while link is in progress (and it was done on the assumption that everything needing the executable is linked serially). A better solution would likely be an `mLastSuccessfullyLinkedExecutable` that normally points to `mExecutable`, but not during link. On `resolveLink`, it would either make `mExecutable` point back to `mLastSuccessfullyLinkedExecutable`, or the other way around based on whether the link was successful or not. Bug: angleproject:8297 Change-Id: Ic9d55bccb75fff0253fe299a244bf1e4bbc416a6 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4781632 Reviewed-by: Yuxin Hu <yuxinhu@google.com> Auto-Submit: Shahbaz Youssefi <syoussefi@chromium.org> Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org> Reviewed-by: Charlie Lao <cclao@google.com>
Shahbaz Youssefi cfd9ccd0 2023-08-09T17:21:20 Reland: Vulkan: Move SPIR-V set up to link job This is a reland of 10f54902e816fa7e4cf314384e00590e2b9bfa1d Bug: angleproject:8297 Change-Id: I701b750a13ac5b17df67dee5b6c37c13c60f5b10 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4793219 Reviewed-by: Yuxin Hu <yuxinhu@google.com> Reviewed-by: Charlie Lao <cclao@google.com> Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org>
Shahbaz Youssefi 8f64b51d 2023-08-09T16:51:13 Reland: Vulkan: Move default uniform init to link job This is a reland of d8cd4dcdc9c55c88f030f7fca41357e99e600ed2 Bug: angleproject:8297 Change-Id: Ib4f8e9dd258da71d44983bbb619b6b4abda0b109 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4793218 Reviewed-by: Charlie Lao <cclao@google.com> Reviewed-by: Yuxin Hu <yuxinhu@google.com> Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org>
Shahbaz Youssefi f3c1de36 2023-08-17T15:29:24 Make shader recompile while parallel linking safe Prior to this change, Program* objects held references to Shader* objects. This poses a problem where a shader recompile can race with a program link, if the program link is done in parallel. As a result, a good chunk of the link job is done serially and under the share group lock. After this change, that is no longer a problem, and most of the link can be made lockless/parallelized. This change separates out the "compiled state" from the rest of the shader state. This was already done for the front-end state (for the sake of caching), but is also now done for the backends that need it. The compiled state in turn is placed in a shared_ptr, and is shared with the program. When a shader is compiled, its own shared_ptr is replaced with a new object, leaving all programs currently compiling unaffected and using the previous compilation results. Once a program is linked, its references to compiled shader states is updated. Bug: angleproject:8297 Change-Id: Iff7094a37088fbad99c6241f1c48b0bd4c820eb2 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4791065 Reviewed-by: Geoff Lang <geofflang@chromium.org> Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org> Reviewed-by: Charlie Lao <cclao@google.com>
Charlie Lao 3479f4a1 2023-08-16T14:43:35 Move more ProgramExecutable basic type vars to mBasicDataTypeStruct They can be just load/saved along with mBasicDataTypeStruct, no need to do them separately: mLinkedShaderVersions was previously only saved if it is separate program, now it is always saved along with mBasicDataTypeStruct, since the extra cost of a few more bytes of memcpy is ignorable, but will make it more consistent. mActiveUniformBlockBindings was previously updated instead of load, now it is load (and verified with ASSERT) mCanDrawWith was previously updated, now chnaged to load. Bug: b/275102061 Change-Id: I4e15b955606f34966d56b1c5e3d9d2f22ddeb14d Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4786308 Reviewed-by: Roman Lavrov <romanl@google.com> Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org> Commit-Queue: Charlie Lao <cclao@google.com>
Charlie Lao c34f83d9 2023-08-16T11:53:27 Group all ProgramExecutable basic data type members into a struct So that we can load/save with a simple memcpy. Bug: b/275102061 Change-Id: I178404fd72b615174a7a0412ea5482ae2bea2f80 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4785567 Reviewed-by: Roman Lavrov <romanl@google.com> Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org> Commit-Queue: Charlie Lao <cclao@google.com>
Charlie Lao 1e1c9d9d 2023-08-11T15:16:26 Pack and reduce Program::mProgramInput size Right now mProgramInput is std::vector<sh::ShaderVariable>. It really only need a subset of ShaderVariable struct. This CL adds a ProgramInput struct so that we can add data members that actually required. This CL also makes bools into bitfield and some variables to uint16_t to further compact the size. This CL also groups the data memebers other than string to basicDataTypeStruct which only contains basic data types and the entire struct is memcpied during program binary load and save. This not just reduces number of memcpy calls, but also improves reliability so that when someone adds a new member into the struct, it will automatically load/save correctly. Bug: b/275102061 Change-Id: Ic055c986453ed46e56057a0122c9926245fef4d1 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4776267 Reviewed-by: Roman Lavrov <romanl@google.com> Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org> Commit-Queue: Charlie Lao <cclao@google.com>
Shahbaz Youssefi 4ba4af61 2023-08-17T11:28:18 Revert recompile blocking on link Causing timeouts on some platforms. An alternative implementation will follow. This change also reverts two changes that depend on it: Vulkan: Move SPIR-V set up to link job 10f54902e816fa7e4cf314384e00590e2b9bfa1d. Vulkan: Move default uniform init to link job d8cd4dcdc9c55c88f030f7fca41357e99e600ed2. Bug: angleproject:8297 Change-Id: I9a258460e7bcaeac214be5e63c16c20681e0bcde Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4789843 Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org> Reviewed-by: Yuly Novikov <ynovikov@chromium.org>
Vivian Pao 9f48f931 2023-08-16T06:38:15 Revert "Tightly pack LinkedUniform by using int16_t" This reverts commit 152cf62b38874238095a91307e4ea9bcdedf8f46. Reason for revert: Suspect cause of failure in for several Linux MSan Tests, e.g. https://ci.chromium.org/ui/p/chromium/builders/ci/Linux%20MSan%20Tests/42403/overview https://ci.chromium.org/ui/p/chromium/builders/ci/WebKit%20Linux%20MSAN/22174/overview https://ci.chromium.org/ui/p/chromium/builders/ci/Linux%20MSan%20Tests/42403/overview Original change's description: > Tightly pack LinkedUniform by using int16_t > > There is a check of vector size when we link uniforms and the maximum > vector size is 4096 due to we clamp the maxUniformBlockSize to 64KB. In > reality, if we exceeds this number, program link will take really long > time and then hit failure. So there is no real need to keep all the > variables in 32 bit integer. This CL changes to 16 bit integer. Further, > sh::BlockMemberInfo and ActiveVariable data members are embeded into > LinkedUniform struct as well so that the unused variables can be removed > and data can be tightly packed. This also makes LinkedUniform easier to > maintain as a simple struct with basic data types. With this change, > LinkedUniform size is reduced from 108 bytes down to 60 bytes, 48 bytes > reduction. Given some apps has 200-ish uniforms, this CL reduces 48 > bytes x 200 = ~9K memory just for uniforms per program (which goes > through hash compute and decompression and file reads). > > Bug: b/275102061 > Change-Id: I7fae20f5b75f3239305e2094a992e3040b8c8e4c > Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4754133 > Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org> > Commit-Queue: Charlie Lao <cclao@google.com> Bug: b/275102061 Change-Id: Id344e306307553731097f06edafc40bfeb73ff80 No-Presubmit: true No-Tree-Checks: true No-Try: true Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4780494 Reviewed-by: Geoff Lang <geofflang@chromium.org> Commit-Queue: Geoff Lang <geofflang@chromium.org>
Charlie Lao 152cf62b 2023-08-15T13:43:12 Tightly pack LinkedUniform by using int16_t There is a check of vector size when we link uniforms and the maximum vector size is 4096 due to we clamp the maxUniformBlockSize to 64KB. In reality, if we exceeds this number, program link will take really long time and then hit failure. So there is no real need to keep all the variables in 32 bit integer. This CL changes to 16 bit integer. Further, sh::BlockMemberInfo and ActiveVariable data members are embeded into LinkedUniform struct as well so that the unused variables can be removed and data can be tightly packed. This also makes LinkedUniform easier to maintain as a simple struct with basic data types. With this change, LinkedUniform size is reduced from 108 bytes down to 60 bytes, 48 bytes reduction. Given some apps has 200-ish uniforms, this CL reduces 48 bytes x 200 = ~9K memory just for uniforms per program (which goes through hash compute and decompression and file reads). Bug: b/275102061 Change-Id: I7fae20f5b75f3239305e2094a992e3040b8c8e4c Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4754133 Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org> Commit-Queue: Charlie Lao <cclao@google.com>
Shahbaz Youssefi 10f54902 2023-08-09T17:21:20 Vulkan: Move SPIR-V set up to link job Bug: angleproject:8297 Change-Id: I2c7eb0281d181560c8fa3ace007b1e547b6cf18e Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4764619 Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org> Reviewed-by: Charlie Lao <cclao@google.com> Reviewed-by: Yuxin Hu <yuxinhu@google.com>
Shahbaz Youssefi d8cd4dcd 2023-08-09T16:51:13 Vulkan: Move default uniform init to link job Bug: angleproject:8297 Change-Id: I5bab916f452439d92afa65b9172574000ee0b587 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4762838 Reviewed-by: Charlie Lao <cclao@google.com> Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org> Reviewed-by: Yuxin Hu <yuxinhu@google.com>
Shahbaz Youssefi 424f43e4 2023-08-11T17:07:16 Prevent shaders from recompiling while a link job is in progress This will prevent a guarantee that link jobs can take further advantage of. In particular, a good chunk of the link job is done serially and under the share group lock due to this recompile-while-link issue. After this change, that is no longer a problem, and most of the link can be made lockless/parallelized. Bug: angleproject:8297 Change-Id: Ic41ac62fb8c40131a69cd90fa9430584964677fa Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4776338 Reviewed-by: Charlie Lao <cclao@google.com> Reviewed-by: Geoff Lang <geofflang@chromium.org> Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org>
Charlie Lao 61a5707f 2023-08-11T12:51:21 Load/Save ShaderVariableBuffer::memberIndexes in one memcpy ShaderVariableBuffer::memberIndexes is std::vector<unsigned int>, and can be quite huge (164 on some programs in blade_and_soul_revolution). Right now it is calling readInt for individual index and then push_back. This causes vector to be copied as it grows size. This CL makes one memcpy for entire vector and increase size only once. Bug: b/275102061 Change-Id: I983ad289b126dbfd430cf94a3772c2bfd93cb1c6 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4775835 Reviewed-by: Roman Lavrov <romanl@google.com> Commit-Queue: Charlie Lao <cclao@google.com> Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org>
Charlie Lao eff12c1e 2023-08-03T12:10:05 Move name and mappedName out of LinkedUniform struct These are std::string objects. This CL moves these two data out of LinkedUniform and into ProgramExecutable class, side by side with mUniforms. With these two data moved out, LinkedUniform is now a simple struct with basic data types, and I can memcpy entire vector of uniforms. Bug: b/275102061 Change-Id: I9ae13e7daca85f8e5f3662a4718f190bebb5f5d2 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4750442 Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org> Reviewed-by: Yuxin Hu <yuxinhu@google.com> Commit-Queue: Charlie Lao <cclao@google.com>
Geoff Lang f4e901b4 2023-08-07T10:52:09 Validate program binaries are the same CPU bit-ness. ANGLE's program binary serialize/deserialize logic uses size_t and other non-fixed sized integer types. This can cause crashes if the CPU architecture changes between saving and loading of binaries. Bug: chromium:1470074 Bug: angleproject:8223 Change-Id: Ib2529e0e6e66e28a184aa1ec94075e343e1f1d5e Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4752265 Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org> Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org>
Charlie Lao 22438ad8 2023-08-02T14:01:45 Embed ActiveVariable into BufferVariable and ShaderVariableBuffer This CL embeds ActiveVariable into BufferVariable and ShaderVariableBuffer struct instead of subclass. This allows us to remove the virtual function of ~ActiveVariable(), which means ActiveVariable is a simple struct with basic types and memcpy can be used for load/save. Thus, in this CL, I also moved activeVariables to LinkedUniform::mFixedSizeData structure and let memcpy handle the load/save. Bug: b/275102061 Change-Id: I8d21080cfdd72d4d22cef927d136ca428d9b12e4 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4722265 Reviewed-by: Roman Lavrov <romanl@google.com> Commit-Queue: Charlie Lao <cclao@google.com> Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org>
Charlie Lao ee019f58 2023-08-01T17:03:24 Group fixed sized data of LinkedUniform into a struct Most of gl::LinkedUniform struct is a basic types and can be simply initialized with memcpy. This CL groups these together and encapsulate them with get APIs. The load/save is now a single memcpy for the entire fixed size data structure of basic types. Bug: b/275102061 Change-Id: I49120c06ec941c783790ac0ecb0ee314a4234b26 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4740298 Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org> Reviewed-by: Roman Lavrov <romanl@google.com> Commit-Queue: Charlie Lao <cclao@google.com>
Charlie Lao 79973750 2023-07-27T17:53:24 Let UniformLinker uses its own struct instead of LinkedUniform UniformLinker itself needs more detailed information than the final linked uniforms. Right now they are using the same data structure. This CL adds a new structure for linker to use. Since the linker data structure is temporary, we never needs to load/save from cache, we have more freedom here to use complex data types. The benefit of separate out intermediate structure from LinkedUniform is that it allows us to further simplify LinkedUniform structure without change much of ProgramLinkedResource implementation. As result of this, this CL removes field and outerArraySizes (both are std::vector type) from LinkedUniform. Bug: b/275102061 Change-Id: Ic1291b51b8906ac586a2f25eb1dbbe74eae8fdc5 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4727742 Reviewed-by: Roman Lavrov <romanl@google.com> Commit-Queue: Charlie Lao <cclao@google.com> Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org>
Charlie Lao f24ea41e 2023-07-31T11:12:44 Embed sh::ShaderVariable data member into gl::LinkedUniform ProgramExecutable::load and save function is load/save each individual data members. Uniform can be quite large as shown in some app traces and this causes performance problem even when program memory has a cache hit. This CL flattens gl::LinkedUniform structure by embedding sh::ShaderVariable data members so that we can remove unrelated members and also prepare for further optimization in future CLs. Bug: b/275102061 Change-Id: I7c18cb73e359fba7cd6375b4b1635e9040ef140d Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4735152 Reviewed-by: Roman Lavrov <romanl@google.com> Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org> Commit-Queue: Charlie Lao <cclao@google.com>
Charlie Lao 5446e87e 2023-07-24T16:46:56 Improve ProgramExecutable::load performance We are calling push_back for each vector inside ProgramExecutable. This causes c++ run time to constantly re-allocate storage and copy the vector over to new storage, and impacts performance negatively. Since the vector size is know when we load program from cache, this CL calls resize to the correct size first and then update each element as we walk over, thus reduces the vector storage reallocation. This CL reduces blade_and_soul_revolution frame time from 4.48 ms to 4.35 ms on pixel 7 pro. This CL also changes ProgramBinaryTest tests to use slightly more complicated program instead of the simplest program. Bug: b/275102061 Change-Id: I8d92117b07a9ad2d0851850e473ea1b86f9868f8 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4713685 Reviewed-by: Roman Lavrov <romanl@google.com> Commit-Queue: Charlie Lao <cclao@google.com> Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org>
Geoff Lang 972f8107 2023-07-14T12:05:40 Reject program binaries when the renderer string changes If the underlying driver changes, reject program binaries from the old versions. The driver is supposed to do this for us (on OpenGL, at least) but this adds some extra protection. Bug: angleproject:4981 Change-Id: Id9486d8e6f9136970c0d7c37d59dea5d43b0a50e Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4685317 Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org> Commit-Queue: Geoff Lang <geofflang@chromium.org>
Shahbaz Youssefi c1ba8e6f 2023-06-20T16:03:20 Vulkan: Flatten shader interface variable maps This change removes duplicate entries added in the shader shader interface variable maps. One level of arrayness (indexed by shader type) is removed from these maps as now there is only a single entry per linked resource/etc. Bug: angleproject:7220 Change-Id: Ibf2d06a0e1f68e68797c2066f36e14cb9e667f77 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4628677 Reviewed-by: Yuxin Hu <yuxinhu@google.com> Reviewed-by: Charlie Lao <cclao@google.com> Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org>
Shahbaz Youssefi 24f4007b 2023-06-08T00:41:55 Vulkan: Use SPIR-V ids instead of names in the transformer This change removes the SPIR-V transformer's reliance on type and variable names. As a result: - String hashing is removed from the info map data structure and the SPIR-V transformer - The ID discovery class is entirely removed - Internal variable names have become a detail of the compiler alone (and are no longer exposed as part of the compiler interface) - Some front-end name tracking is removed ("parentStructMappedName", etc) This change also properly cleans up xfb emulation types that were previously left over. This change allows the SPIR-V compiler to emit user strings as-is instead of prefixing them with `u_` leading to more readable debug shaders. Additionally, it will make it possible not to emit debug info at all. Both of these changes will be done in follow ups. Bug: angleproject:7220 Change-Id: Iaa127496209a27aaae2e0d14c41b22fffb0b72a2 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4600610 Reviewed-by: Roman Lavrov <romanl@google.com> Reviewed-by: Yuxin Hu <yuxinhu@google.com> Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org>
Roman Lavrov 21f16cb1 2023-06-09T17:30:38 Disable clang-format on ANGLE features autogen outputs Updates the script to produce reasonably formatted code without clang-format. Autogen files moved to autogen/ sub-directories because clang-format does not support per-file settings ;( This allows to run this codegen very quickly (~50ms on my machine) Bug: angleproject:8193 Change-Id: Ie84282090d574ebb4debe3edcfd82f983f27a5ff Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4604578 Commit-Queue: Roman Lavrov <romanl@google.com> Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org>
Geoff Lang 6e40ce00 2023-06-05T11:47:29 Do not call postResolveLink in Program::deserialize. postResolveLink assumes the program is currently linked, which is not the case in Program::deserialize. It makes calls to set uniforms which is not always expected by the backend. Call postResolveLink after the backend has linked when loading program binaries. Bug: angleproject:6073, angleproject:8183 Change-Id: Idacb81040ea79a7df51917aaa27c77b25df7d5cd Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4588410 Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org> Commit-Queue: Geoff Lang <geofflang@chromium.org> Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org>
Charlie Lao e21ecd1b 2023-05-26T14:06:46 Vulkan: Add dirty bit processing for uniform buffer change When app calls glBufferData for the uniform buffer, we may end up reallocate the storage. This will set DIRTY_BIT_UNIFORM_BUFFER_BINDINGS on the context, but the exact uniform block index gets lost along the way. This CL sets mDirtyBits on the program for the corresponding block index and then changed vulkan backend to utilize the program's mDirtyBits and only update the buffer if it is dirty, instead of always update all uniform buffers even if only one of the buffer is dirty. In order to make this work, this CL also adds the reverse tracking from buffer binding to uniform blocks. Previously we already have the tracking of which buffer binding index is used for which buffer block index. This CL adds mUniformBlockBindingMasks which is an array of BitSets. Each array element tracks all the uniform block index that is using this buffer binding index (you can have the same buffer bound to multiple uniform block index). Then when a buffer binding index is dirty, that BitSet gets added into program's uniform block dirty bits. This CL and previous CL improves GfxBench gl_driver2_off score 1.8% (from average 6797 to average 6919) on pixel 7 pro. Bug: b/282194402 Change-Id: Ic5002643a5297907276fc9b20ca7d21af9bdc4fe Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4553136 Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org> Reviewed-by: Roman Lavrov <romanl@google.com> Commit-Queue: Charlie Lao <cclao@google.com>
Scott Violet 65f4d2a4 2023-04-17T10:08:18 Adds features to enable loading/saving metallibs to blobcache CompileMetalShaders results in the appropriate command line tool being run to generate metallib blobs from shader source. DisableProgramCaching results in not saving programs to BlobCache LoadMetalShadersFromBlobCache results in trying to load metallibs from BlobCache. Bug: chromium:1423136 Change-Id: I01a4d7a5d60ed5ac978fb99db01b741e0f19e76b Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4434293 Reviewed-by: Geoff Lang <geofflang@chromium.org> Commit-Queue: Scott Violet <sky@chromium.org>
Geoff Lang be5e767e 2023-02-01T11:03:26 dump_shader_source: Also dump program links. When the dump_shader_source is enabled, dump a small file for each program link containing which shaders were attached to the program. This allows us to build perf tests knowing which shaders can link together. Bug: angleproject:7760, chromium:1385510 Change-Id: I0713e9f04cc157c3a3eeeec6d597db27b08575d1 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4214011 Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org> Reviewed-by: Cody Northrop <cnorthrop@google.com> Commit-Queue: Geoff Lang <geofflang@chromium.org>
Yuxin Hu e1bd0415 2023-02-03T16:58:21 Revert "Re-land fixing of commit ID for Android builds." This reverts commit e6662832af3c6394d0a0bbecd1e7a52581e88b12. After the commit https://github.com/google/angle/commit/61728827d2e5ecce685578bc54bb2c744b65fc9a, we no longer depends on git hash to determine if the program binary cache is valid. The reverted commit was added to handle unknown git hash. Android build infra does not guarantee access to git, and we needed GetANGLEHasBinaryLoading() check because we can't tell if ANGLE program changed if git hash remains "unknown" across different builds. Now that we used a different hash ANGLE_PROGRAM_VERSION as the cache key, which generates from a list of code files that affect program binary content and it won't be unknown on Android build, we can revert the commit and rely on ANGLE_PROGRAM_VERSION to reject invalid program binary cache. This fixes the dEQP test dEQP-GLES3.functional.shader_api.program_binary. simple.uniform_reset_on_binary_load, which failed because the GetANGLEHasBinaryLoading() is stopping Program::loadBinary() due to "unknown" git hash on Android build. This CL reverts most of the original commit, cleans up unused macros and variables, and reserves the change in program::loadBinary() that returns incomplete if binary format is not GL_PROGRAM_BINARY_ANGLE. This CL also reserves the changes in https://github.com/google/angle/commit/dd8021d98cd2909dfb85012f9b25010bd3ce2536 in case Sumsung parter still needs them. Bug: b/258445879 Change-Id: Ia4380de9362f7b8bed6de6a54943bec6600cb76b Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4219368 Commit-Queue: Yuxin Hu <yuxinhu@google.com> Reviewed-by: Roman Lavrov <romanl@google.com>
Kyle Piddington b23bf47c 2023-01-25T18:50:40 Reland "Metal: rewrite default uniforms and uniform blocks" Instead of rewriting uniforms in shaders to match std140 layout, re-pack incoming uniform blocks' std140 packed variables to match Metal's layout. This change intorduces a new BlockLayoutEncoder for Metal types The block encoder handles packing typically larger GL types (bools) into smaller types, and adding support for more compressed matrix types. Since we no longer need to do shader-time packing and unpacking of data from std140 padded structs, complicated shader transformations have been removed. This patch greatly reduces register pressure, especially when working with shaders with arrays of previously expanded types. (Vec3's) Reland: Fix an issue where the default uniform block's final size was not aligned to the default uniform block's alignment requirements, causing crashes with the debug layer enabled. Bug: angleproject:7137 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3733524 Commit-Queue: Kyle Piddington <kpiddington@apple.com> Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org> Reviewed-by: Kenneth Russell <kbr@chromium.org> Change-Id: I89d3b817675486fde73b91b0be0f4c25986d4ba5 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4209867
Kenneth Russell fb948319 2023-01-31T18:16:41 Revert "Metal: rewrite default uniforms and uniform blocks" This reverts commit b8bfe6f6c04a6cf4fac81f0363ad23a0b92b22f2. Reason for revert: potential cause of crashes/timeouts in crbug.com/1411755 Original change's description: > Metal: rewrite default uniforms and uniform blocks > > Instead of rewriting uniforms in shaders to match std140 layout, > re-pack incoming uniform blocks' std140 packed variables to match > Metal's layout. > > This change intorduces a new BlockLayoutEncoder for Metal types > The block encoder handles packing typically larger GL types (bools) > into smaller types, and adding support for more compressed matrix types. > > Since we no longer need to do shader-time packing and unpacking of data > from std140 padded structs, complicated shader transformations have been > removed. This patch greatly reduces register pressure, especially when > working with shaders with arrays of previously expanded types. (Vec3's) > > Bug: angleproject:7137 > Change-Id: Icd1da8c7a383f3354313a58618fc1bf6656726b4 > Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3733524 > Commit-Queue: Kyle Piddington <kpiddington@apple.com> > Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org> > Reviewed-by: Kenneth Russell <kbr@chromium.org> Bug: angleproject:7137 Bug: chromium:1411755 Change-Id: I1b6a62d7eb542c3e7ef4b84858ab2683abb66f8b No-Presubmit: true No-Tree-Checks: true No-Try: true Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4210448 Commit-Queue: Kenneth Russell <kbr@chromium.org> Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Kyle Piddington b8bfe6f6 2023-01-25T18:50:40 Metal: rewrite default uniforms and uniform blocks Instead of rewriting uniforms in shaders to match std140 layout, re-pack incoming uniform blocks' std140 packed variables to match Metal's layout. This change intorduces a new BlockLayoutEncoder for Metal types The block encoder handles packing typically larger GL types (bools) into smaller types, and adding support for more compressed matrix types. Since we no longer need to do shader-time packing and unpacking of data from std140 padded structs, complicated shader transformations have been removed. This patch greatly reduces register pressure, especially when working with shaders with arrays of previously expanded types. (Vec3's) Bug: angleproject:7137 Change-Id: Icd1da8c7a383f3354313a58618fc1bf6656726b4 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3733524 Commit-Queue: Kyle Piddington <kpiddington@apple.com> Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org> Reviewed-by: Kenneth Russell <kbr@chromium.org>
Yuxin Hu 82826be0 2023-01-06T11:20:14 Reland "Make ANGLE program version only dependent on data that matters" This is a reland of commit 61728827d2e5ecce685578bc54bb2c744b65fc9a Changes made on top of previous commit 1. In src/program_serialize_data_version.py, fix the script error on Windows: anglebug.com/7918. 2. In BUILD.gn, replace the 'write_file' with 'response_file_content'. See https://gn.googlesource.com/gn/+/main/docs/reference.md#var_response_file_contents 3. In scripts/generate_android.bp.py, add code to handle the special GN build flag {{response_file_name}}. The code writes the list defined in srcs (identified by $(in) in Android blueprint) into a temp file named 'gn_response_file', and replaces the {{response_file_name}} with 'gn_response_file'. Original change's description > Changes made on top of original commit > 1. Enable execution permission on python script > program_serialize_data_version.py > 2. Remove unused list in libGLESv2.gni > 3. In angle/BUILD.gn, change file path from > "relative to angle_root", to "relative to root_build_dir", > so that inside the script program_serialize_data_version.py, > we don't have to find the absolute path of the code files for > hashing. > Original change's description > > This change introduces a new variable ANGLE_PROGRAM_VERSION > > to track the version of ANGLE source files that affect shader > > program serialization/deserialization. This change include more > > source files than necessary, to serve the purpose of a conservative > > jumping off point. We will narrow down the list of files for > > ANGLE_PROGRAM_VERSION hash generation in the future. > > Add a new script program_serialize_data_version.py that will > > be triggered during the build when the related source files changed. > > The script will generate a hash and the hash size from the related > > source files. In program serialization/deserialization and cache > > key generation, we will use this hash value instead of the entire > > ANGLE git hash. When the hash value changed, we know that the > > related source files changed, and we should invalidate the program > > cache and re-generate the blob cache / program binary. > > Bug: angleproject:4981 > > Change-Id: I2fb609416738d459d3289190c232c2d797ba58e3 > > Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4072215 > > Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org> > > Reviewed-by: Cody Northrop <cnorthrop@google.com> > > Reviewed-by: Jamie Madill <jmadill@chromium.org> > > Commit-Queue: Yuxin Hu <yuxinhu@google.com> > Bug: angleproject:4981 > Change-Id: Iaa9eb0ab33439197bc30d03064fc245ea7ef1ea8 > Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4113445 > Reviewed-by: Amirali Abdolrashidi <abdolrashidi@google.com> > Reviewed-by: Cody Northrop <cnorthrop@google.com> > Commit-Queue: Yuxin Hu <yuxinhu@google.com> Bug: angleproject:4981 Change-Id: Ib5bba199be6d08a1e19807026df0e7b747dbc8a9 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4144078 Reviewed-by: Roman Lavrov <romanl@google.com> Commit-Queue: Yuxin Hu <yuxinhu@google.com> Reviewed-by: Cody Northrop <cnorthrop@google.com>
Yuxin Hu c4b95929 2023-01-05T19:59:07 Revert "Reland "Make ANGLE program version only dependent on data that matters"" This reverts commit 61728827d2e5ecce685578bc54bb2c744b65fc9a. Reason for revert: it breaks aosp build: https://android-review.git.corp.google.com/c/platform/external/angle/+/2374971. Build error: `FileNotFoundError: [Errno 2] No such file or directory: 'angle_code_affecting_program_serialize'` Original change's description: > Reland "Make ANGLE program version only dependent on data that matters" > > This is a reland of commit c303758fbc8d23266be578fae246e5687656b4c6 > > Changes made on top of original commit > 1. Enable execution permission on python script > program_serialize_data_version.py > 2. Remove unused list in libGLESv2.gni > 3. In angle/BUILD.gn, change file path from > "relative to angle_root", to "relative to root_build_dir", > so that inside the script program_serialize_data_version.py, > we don't have to find the absolute path of the code files for > hashing. > > Original change's description > > > This change introduces a new variable ANGLE_PROGRAM_VERSION > > to track the version of ANGLE source files that affect shader > > program serialization/deserialization. This change include more > > source files than necessary, to serve the purpose of a conservative > > jumping off point. We will narrow down the list of files for > > ANGLE_PROGRAM_VERSION hash generation in the future. > > > Add a new script program_serialize_data_version.py that will > > be triggered during the build when the related source files changed. > > The script will generate a hash and the hash size from the related > > source files. In program serialization/deserialization and cache > > key generation, we will use this hash value instead of the entire > > ANGLE git hash. When the hash value changed, we know that the > > related source files changed, and we should invalidate the program > > cache and re-generate the blob cache / program binary. > > > Bug: angleproject:4981 > > Change-Id: I2fb609416738d459d3289190c232c2d797ba58e3 > > Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4072215 > > Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org> > > Reviewed-by: Cody Northrop <cnorthrop@google.com> > > Reviewed-by: Jamie Madill <jmadill@chromium.org> > > Commit-Queue: Yuxin Hu <yuxinhu@google.com> > > Bug: angleproject:4981 > Change-Id: Iaa9eb0ab33439197bc30d03064fc245ea7ef1ea8 > Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4113445 > Reviewed-by: Amirali Abdolrashidi <abdolrashidi@google.com> > Reviewed-by: Cody Northrop <cnorthrop@google.com> > Commit-Queue: Yuxin Hu <yuxinhu@google.com> Bug: angleproject:4981 Change-Id: If7d3779c8f40dafa9a9c057f669ab53d0494526e Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4140138 Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com> Reviewed-by: Amirali Abdolrashidi <abdolrashidi@google.com> Commit-Queue: Yuxin Hu <yuxinhu@google.com>
Yuxin Hu 61728827 2022-12-14T15:29:15 Reland "Make ANGLE program version only dependent on data that matters" This is a reland of commit c303758fbc8d23266be578fae246e5687656b4c6 Changes made on top of original commit 1. Enable execution permission on python script program_serialize_data_version.py 2. Remove unused list in libGLESv2.gni 3. In angle/BUILD.gn, change file path from "relative to angle_root", to "relative to root_build_dir", so that inside the script program_serialize_data_version.py, we don't have to find the absolute path of the code files for hashing. Original change's description > This change introduces a new variable ANGLE_PROGRAM_VERSION > to track the version of ANGLE source files that affect shader > program serialization/deserialization. This change include more > source files than necessary, to serve the purpose of a conservative > jumping off point. We will narrow down the list of files for > ANGLE_PROGRAM_VERSION hash generation in the future. > Add a new script program_serialize_data_version.py that will > be triggered during the build when the related source files changed. > The script will generate a hash and the hash size from the related > source files. In program serialization/deserialization and cache > key generation, we will use this hash value instead of the entire > ANGLE git hash. When the hash value changed, we know that the > related source files changed, and we should invalidate the program > cache and re-generate the blob cache / program binary. > Bug: angleproject:4981 > Change-Id: I2fb609416738d459d3289190c232c2d797ba58e3 > Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4072215 > Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org> > Reviewed-by: Cody Northrop <cnorthrop@google.com> > Reviewed-by: Jamie Madill <jmadill@chromium.org> > Commit-Queue: Yuxin Hu <yuxinhu@google.com> Bug: angleproject:4981 Change-Id: Iaa9eb0ab33439197bc30d03064fc245ea7ef1ea8 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4113445 Reviewed-by: Amirali Abdolrashidi <abdolrashidi@google.com> Reviewed-by: Cody Northrop <cnorthrop@google.com> Commit-Queue: Yuxin Hu <yuxinhu@google.com>
Mohan Maiya cc34aa73 2022-12-19T14:03:27 Move BinaryStream to common and expose ShaderState to compiler This is a refactor change in preparation for adding support for glShaderBinary. Move BinaryStream to common so that it is accessible by both libANGLE and the Compiler. Extract members that hold the result of compilation from ShaderState and move into new CompiledShaderState struct. Move helper functions & classes relevant to ShaderVar serialization to the CompiledShaderState header. Tests: EGLBlobCacheTest* Bug: angleproject:7833 Change-Id: I7ec575247eccb3afbc6ab6bfa24d36e69d4576f5 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4080998 Reviewed-by: Charlie Lao <cclao@google.com> Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org> Commit-Queue: mohan maiya <m.maiya@samsung.com>
Alexey Knyazev 28e7adca 2022-12-09T00:00:00 GL: Implement clip distance state emulation Pass the current set of enabled clip distances to vertex shaders via an internal uniform and dynamically set disabled elements to zero. Bug: angleproject:7880 Change-Id: I709d31dc7ca0606decf49adf674460a941837683 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4094314 Commit-Queue: Alexey Knyazev <lexa.knyazev@gmail.com> Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org>
Yuxin Hu 25ce2e23 2022-12-16T18:16:32 Revert "Make ANGLE program version only dependent on data that matters" This reverts commit c303758fbc8d23266be578fae246e5687656b4c6. Reason for revert: This is causing the Android roller fail: https://crrev.com/c/4072215/comments/6918d7f6_8100d66b?tab=comments Original change's description: > Make ANGLE program version only dependent on data that matters > > This change introduces a new variable ANGLE_PROGRAM_VERSION > to track the version of ANGLE source files that affect shader > program serialization/deserialization. This change include more > source files than necessary, to serve the purpose of a conservative > jumping off point. We will narrow down the list of files for > ANGLE_PROGRAM_VERSION hash generation in the future. > > Add a new script program_serialize_data_version.py that will > be triggered during the build when the related source files changed. > The script will generate a hash and the hash size from the related > source files. In program serialization/deserialization and cache > key generation, we will use this hash value instead of the entire > ANGLE git hash. When the hash value changed, we know that the > related source files changed, and we should invalidate the program > cache and re-generate the blob cache / program binary. > > Bug: angleproject:4981 > Change-Id: I2fb609416738d459d3289190c232c2d797ba58e3 > Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4072215 > Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org> > Reviewed-by: Cody Northrop <cnorthrop@google.com> > Reviewed-by: Jamie Madill <jmadill@chromium.org> > Commit-Queue: Yuxin Hu <yuxinhu@google.com> Bug: angleproject:4981 Change-Id: Ib235d85c79ed54110696ac5413571934890fdc98 No-Presubmit: true No-Tree-Checks: true No-Try: true Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4114274 Reviewed-by: Amirali Abdolrashidi <abdolrashidi@google.com> Commit-Queue: Yuxin Hu <yuxinhu@google.com>
Yuxin Hu c303758f 2022-12-14T15:29:15 Make ANGLE program version only dependent on data that matters This change introduces a new variable ANGLE_PROGRAM_VERSION to track the version of ANGLE source files that affect shader program serialization/deserialization. This change include more source files than necessary, to serve the purpose of a conservative jumping off point. We will narrow down the list of files for ANGLE_PROGRAM_VERSION hash generation in the future. Add a new script program_serialize_data_version.py that will be triggered during the build when the related source files changed. The script will generate a hash and the hash size from the related source files. In program serialization/deserialization and cache key generation, we will use this hash value instead of the entire ANGLE git hash. When the hash value changed, we know that the related source files changed, and we should invalidate the program cache and re-generate the blob cache / program binary. Bug: angleproject:4981 Change-Id: I2fb609416738d459d3289190c232c2d797ba58e3 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4072215 Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org> Reviewed-by: Cody Northrop <cnorthrop@google.com> Reviewed-by: Jamie Madill <jmadill@chromium.org> Commit-Queue: Yuxin Hu <yuxinhu@google.com>
Chris Dalton 9d41585e 2022-08-12T14:20:34 Make PLS coherent on D3D 11.3 Adds a new internal memory qualifier to the compiler called "rasterOrdered", which we set in RewritePixelLocalStorage.cpp when D3D 11.3 Rasterizer Order Views are supported. The HLSL translator then generates RasterizerOrderedTexture2D<> instead of RWTexture2D<> when this qualifier is set. Bug: angleproject:7279 Change-Id: I39b8c3279b7bff93b7e57272e8fb84d9c0312616 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3830288 Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org> Reviewed-by: Geoff Lang <geofflang@chromium.org> Commit-Queue: Chris Dalton <chris@rive.app>
Eddie Hatfield 955adb77 2022-08-12T10:14:48 Cache compiled shader By storing the compiled shader in the blob cache, the time to recompile the same shader is reduced. Based on work by <hckim.kim@samsung.com> Bug: angleproject:7036 Change-Id: I884ae40e715c49a9ccd12903012e8327811e3557 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3808235 Commit-Queue: Cody Northrop <cnorthrop@google.com> Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org> Reviewed-by: Cody Northrop <cnorthrop@google.com>
Eddie Hatfield 2ebd5100 2022-08-11T10:43:32 Add gl::Context as a parameter to Shader::resolveCompile This prepares us to access the Context's shader cache in resolveCompile in the next commit. Bug: angleproject:7036 Change-Id: I7995c54b290a5a48f0c8985cb56ea0048598ab2f Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3827642 Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org> Reviewed-by: Cody Northrop <cnorthrop@google.com> Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org>
kmx.io     mail     discord kmxgit v0.4.0