|
93b97a59
|
2023-11-03T22:07:23
|
|
Make link job directly wait on compile job
Previously, program link waited on the compile job on the calling thread
before launching the link job. As a result, sequences of intermixed
compile and link would get largely serialized as such:
Main Thread Thread 1 Thread 2 Thread 3 Thread 4
Compile -------> Compile
Compile -----------|----------> Compile
Link | |
Wait | |
| | |
|<--------------/--------------/
\------------------------------------------> Link
Compile -------> Compile |
Compile -----------|----------> Compile |
Link | | |
Wait | | |
| | | |
|<--------------/--------------/ |
\---------------------------------------------|-----------> Link
Compile -------> Compile | |
Compile -----------|----------> Compile | |
Link | | | |
Wait | | | |
| | | | |
...
With this change, the main thread no longer waits for compilation to
finish. It's the link job itself that does the waiting. This allows
the main thread to go through Compile and Link commands without
blocking, generating as many jobs as needed. The above scenario
therefore becomes:
Main T1 T2 T3 T4 T5 T6 T7 T8 T9
C ----> C
C ------|----> C
L ------|------|----> L
C ------|------|-------W---> C
C ------|------|-------|-----|----> C
L ------|------|-------|-----|------|----> L
C ------|------|-------|-----|------|-------W---> C
C ------|------|-------|-----|------|-------|-----|----> C
L ------|------|-------|-----|------|-------|-----|------|----> L
. \-----\------>/ | | | | | W
. | \-----\------>/ | | |
. | | \-----\------>/
. | | |
. | | |
This greatly improves the amount of parallelism compile and link jobs
get.
The careful observer may note that the link job being blocked on the
compile job is now wasting a thread from the thread pool. While this
change is strictly an improvement, parallelism can be further improved
if the link job is just not assigned to a thread until the corresponding
compile jobs are finished. This is currently not possible, but may be
if:
- Instead of a thread pool, the operating system's FIFO scheduler is
used. Then the operating system would automatically put blocking
tasks to sleep and pick up another task. This has the downside of
requiring threads to be created for each task.
- The thread pool work scheduler is enhanced to be made aware of
relationship between tasks and avoid scheduling jobs whose
dependencies are not yet met.
Alternatively, the number of threads in the pool can be increased by 30%
and hope for the best.
Bug: angleproject:8297
Change-Id: If4e6540ade47558a10cfab55e2286f073b904928
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/5006874
Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org>
Reviewed-by: Geoff Lang <geofflang@chromium.org>
Reviewed-by: Charlie Lao <cclao@google.com>
|
|
|
68bfa1ed
|
2023-08-22T22:02:15
|
|
Support for link to be entirely parallelized
The link job is split as such:
- Front-end link
- Back-end link
- Independent back-end link subtasks (typically native driver compile
jobs)
- Post-link finalization
Each step depends on the previous. These steps are executed as such:
1. Program::link calls into ProgramImpl::link
- ProgramImpl::link runs whatever needs the Context, such as releasing
resources
- ProgramImpl::link returns a LinkTask
2. Program::link implements a closure that calls the front-end link and
passes the results to the backend's LinkTask.
3. The LinkTask potentially returns a set of LinkSubTasks to be
scheduled by the worker pool
4. Once the link is resolved, the post-link finalization is run
In the above, steps 1 and 4 are done under the share group lock. Steps
2 and 3 can be done in threads or without holding the share group lock
if the backend supports it. Step 2 is not yet made independent of the
Context on some backends, and a frontend feature is used to make that
step either run on the main thread or as a worker thread.
Bug: angleproject:8297
Change-Id: I12f1e6bbaf365543dfcac969e166e0b5aa622104
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4808191
Reviewed-by: Charlie Lao <cclao@google.com>
Reviewed-by: Geoff Lang <geofflang@chromium.org>
Reviewed-by: Shahbaz Youssefi <syoussefi@google.com>
Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org>
|
|
962fdf7b
|
2023-08-31T17:44:51
|
|
Add templated BinaryOutputStream::writeVector and writeStruct
To avoid repeated code pattern, this CL adds templated
BinaryInputStream::readVector and readStruct and
BinaryOutputStream::writeVector and writeStruct, that does the static
assertion to ensure they are trivially copyable.
readIntVector/writeIntVector is removed.
This CL also padding warning for ProgramExecutable::PODStruct to avoid
potential Msan complain.
Bug: b/296433003
Change-Id: I8e718b41cde898960b9e86a2d7d1bc5a837fe561
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4834700
Reviewed-by: Roman Lavrov <romanl@google.com>
Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org>
Commit-Queue: Charlie Lao <cclao@google.com>
|
|
|
f3c1de36
|
2023-08-17T15:29:24
|
|
Make shader recompile while parallel linking safe
Prior to this change, Program* objects held references to Shader*
objects. This poses a problem where a shader recompile can race with a
program link, if the program link is done in parallel.
As a result, a good chunk of the link job is done serially and under the
share group lock. After this change, that is no longer a problem, and
most of the link can be made lockless/parallelized.
This change separates out the "compiled state" from the rest of the
shader state. This was already done for the front-end state (for the
sake of caching), but is also now done for the backends that need it.
The compiled state in turn is placed in a shared_ptr, and is shared with
the program. When a shader is compiled, its own shared_ptr is replaced
with a new object, leaving all programs currently compiling unaffected
and using the previous compilation results.
Once a program is linked, its references to compiled shader states is
updated.
Bug: angleproject:8297
Change-Id: Iff7094a37088fbad99c6241f1c48b0bd4c820eb2
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4791065
Reviewed-by: Geoff Lang <geofflang@chromium.org>
Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org>
Reviewed-by: Charlie Lao <cclao@google.com>
|
|
|
10380f4b
|
2023-06-06T11:52:08
|
|
Vulkan: Output SPIR-V ids from compiler
In this change, the shader interface variables are given SPIR-V ids by
the compiler before SPIR-V generation. Those ids are made available
through the ShaderVariable interface.
The transformer does not yet rely on this information. A follow up
change will rework the backend's name->info map and the transformer to
directly use ids instead of names.
Bug: angleproject:7220
Change-Id: Ic0a62681d4bcf3ed171c39c3ecd83e438ea068c8
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4600609
Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org>
Reviewed-by: Roman Lavrov <romanl@google.com>
Reviewed-by: Yuxin Hu <yuxinhu@google.com>
|
|
cc34aa73
|
2022-12-19T14:03:27
|
|
Move BinaryStream to common and expose ShaderState to compiler
This is a refactor change in preparation for adding support for
glShaderBinary.
Move BinaryStream to common so that it is accessible by both libANGLE
and the Compiler.
Extract members that hold the result of compilation from ShaderState
and move into new CompiledShaderState struct.
Move helper functions & classes relevant to ShaderVar serialization to
the CompiledShaderState header.
Tests: EGLBlobCacheTest*
Bug: angleproject:7833
Change-Id: I7ec575247eccb3afbc6ab6bfa24d36e69d4576f5
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4080998
Reviewed-by: Charlie Lao <cclao@google.com>
Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org>
Commit-Queue: mohan maiya <m.maiya@samsung.com>
|