src/compiler/translator/gen_builtin_symbols.py


Log

Author Commit Date CI Message
Jamie Madill a905cbcd 2019-09-19T23:05:39 Compact built-in symbol table. Should reduce the binary size bloat from the perfect hashing. Local testing on Windows shows a significant size reduction. Bug: chromium:998535 Change-Id: I411cc5a917036d2239d15353d760f44e40faa26c Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/1814725 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Tim Van Patten <timvp@google.com>
Jamie Madill c37d2370 2019-09-19T18:54:40 Reland "Refactor built-in symbol table." This is a reland of 59d982c0dcad580ccbf4e29f32ffa3ea9a280122 Original change's description: > Refactor built-in symbol table. > > In preparation for reducing binary size. Shortend a bunch of strings > and symbol prefixes so the autogen file will be more compact. > > Bug: chromium:998535 > Change-Id: If1bedaaacc06dce9678d432055ffa0c9215a8dbd > Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/1814724 > Reviewed-by: Tim Van Patten <timvp@google.com> > Reviewed-by: Jonah Ryan-Davis <jonahr@google.com> > Commit-Queue: Jamie Madill <jmadill@chromium.org> Bug: chromium:998535 Change-Id: Iebc5b1b5e1a084257fba3c50026cc62df5a879b3 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/1817499 Reviewed-by: Zhenyao Mo <zmo@chromium.org> Commit-Queue: Zhenyao Mo <zmo@chromium.org>
Zhenyao Mo f585c2fb 2019-09-21T02:12:37 Revert "Refactor built-in symbol table." This reverts commit 59d982c0dcad580ccbf4e29f32ffa3ea9a280122. Reason for revert: crbug.com/1006501 Original change's description: > Refactor built-in symbol table. > > In preparation for reducing binary size. Shortend a bunch of strings > and symbol prefixes so the autogen file will be more compact. > > Bug: chromium:998535 > Change-Id: If1bedaaacc06dce9678d432055ffa0c9215a8dbd > Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/1814724 > Reviewed-by: Tim Van Patten <timvp@google.com> > Reviewed-by: Jonah Ryan-Davis <jonahr@google.com> > Commit-Queue: Jamie Madill <jmadill@chromium.org> TBR=timvp@google.com,jonahr@google.com,jmadill@chromium.org Change-Id: Ie540da8ea4429e075a3284183e5a004cbcf1307b No-Presubmit: true No-Tree-Checks: true No-Try: true Bug: chromium:998535,chromium:1006501 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/1817497 Reviewed-by: Zhenyao Mo <zmo@chromium.org> Commit-Queue: Zhenyao Mo <zmo@chromium.org>
Jamie Madill 59d982c0 2019-09-19T18:54:40 Refactor built-in symbol table. In preparation for reducing binary size. Shortend a bunch of strings and symbol prefixes so the autogen file will be more compact. Bug: chromium:998535 Change-Id: If1bedaaacc06dce9678d432055ffa0c9215a8dbd Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/1814724 Reviewed-by: Tim Van Patten <timvp@google.com> Reviewed-by: Jonah Ryan-Davis <jonahr@google.com> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Clemen Deng d7d42540 2019-08-21T15:22:49 Don't build symbol table for GLSL built-ins if on Android The GLSL + ESSL autogenerated symbol table is too large for android, and android also doesn't need desktop GL functionality If on android, compile the ESSL only symbol table Bug: chromium:996286 Change-Id: I14dfc7748dae389e78c35f82a390c67962665356 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/1757372 Commit-Queue: Clemen Deng <clemendeng@google.com> Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org> Reviewed-by: Geoff Lang <geofflang@chromium.org>
Clemen Deng 56db3789 2019-08-01T17:09:53 Use flat arrays instead of switches for function lookups Current implementation of built in function lookup uses autogenerated switch statements. Instead, use the perfect hash mapping to have the lookup use arrays instead. This will improve runtime performance. Bug: angleproject:3805 Change-Id: I6d0ba62d79abd53a7fe818fe675282800781f256 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/1756883 Commit-Queue: Clemen Deng <clemendeng@google.com> Reviewed-by: Geoff Lang <geofflang@chromium.org>
Jonah Ryan-Davis 396527c9 2019-08-20T13:48:38 Autogenerated kFunction* variables detected as mutable constants The android binary size is increased incorrectly because the autogen kFunction* variables are detected as mutable constants. Rename them to function* to workaround this. Bug: angleproject:3823 Change-Id: Ia335d1ae97ce5276a0ff8b9e432b53181cc36680 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/1762494 Reviewed-by: Jonah Ryan-Davis <jonahr@google.com> Reviewed-by: Jamie Madill <jmadill@chromium.org> Commit-Queue: Jonah Ryan-Davis <jonahr@google.com>
Clemen Deng bcae3828 2019-08-19T09:37:18 Auto-generated variables detected as mutable constants Need to rename kVar* variables to just var Bug: angleproject:3823 Change-Id: I34b9f4eec4ee7829d47dcac2f06f1acc6b35f867 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/1758601 Reviewed-by: Jamie Madill <jmadill@chromium.org> Commit-Queue: Clemen Deng <clemendeng@google.com>
Clemen Deng 486f381f 2019-07-18T15:47:31 Add desktop GL built-in functions to symbol table Added GL built-in functions to symbol table autogeneration Built-in lookup now checks against spec and returns GL symbols when appropriate. Bug: angleproject:3719 Change-Id: I74786150f6063b40e13aebcb2eb9ea35e4ecaa53 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/1731550 Reviewed-by: Geoff Lang <geofflang@chromium.org> Commit-Queue: Clemen Deng <clemendeng@google.com>
Clemen Deng e7970c3e 2019-08-07T14:43:17 BasicMangledName class Need a class for basic mangled names since with the addition of GLSL types the number of basic types is > 52 (a-z, A-Z), so we need more than one character to represent a type Bug: angleproject:3719 Change-Id: I98beee9d42a016cb0c017f56ab82538c89212e33 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/1742221 Commit-Queue: Clemen Deng <clemendeng@google.com> Reviewed-by: Geoff Lang <geofflang@chromium.org>
Clemen Deng 44f518b5 2019-08-06T13:31:01 Have run_code_generation only call vpython when needed gen_builtin_symbols.py uses a vpython module to generate a perfect hash function Also seeded the perfect hash function to make it deterministic Bug: angleproject:3747 Change-Id: I660fe71bd6b2213be9d4ccc2f68641637a49a047 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/1738747 Commit-Queue: Clemen Deng <clemendeng@google.com> Reviewed-by: Geoff Lang <geofflang@chromium.org>
Clemen Deng 9031bdd9 2019-07-26T14:51:23 Use perfect-hash module in gen_builtin_symbols.py The script currently takes ~4 minutes to run Using this module instead of manually hashing will improve runtime significantly Bug: angleproject:3747 Change-Id: I7e2d2ef5bbfd136b0299d571e0acc11f334c80b5 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/1724667 Commit-Queue: Clemen Deng <clemendeng@google.com> Reviewed-by: Jamie Madill <jmadill@chromium.org>
Geoff Lang d7d42395 2019-05-06T13:15:35 Format all of ANGLE's python code. BUG=angleproject:3421 Change-Id: I1d7282ac513c046de5d8ed87f7789290780d30a6 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/1595440 Reviewed-by: Jamie Madill <jmadill@chromium.org> Commit-Queue: Geoff Lang <geofflang@chromium.org>
Jamie Madill 0f34f3f2 2019-03-11T10:18:57 Use auto_script in run_code_generation. Cleans up the generator scripts to prepare for listing outputs in the generated hashes file. Also reorganizes the scripts somewhat to make them a bit more maintainable. Bug: angleproject:3227 Change-Id: If40946c2004941d3f6cd51d5521c7db8cc0b52df Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/1512052 Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org> Reviewed-by: Geoff Lang <geofflang@chromium.org> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Jamie Madill c09ae15c 2019-02-01T14:16:32 Enable -Wextra-semi and -Wextra-semi-stmt. This will prevent users from accidentally making semicolon errors in the future. Bug: chromium:926235 Change-Id: I79a6fa376fb1ad8f0fcf1b65b1f572a035d1f4e9 Reviewed-on: https://chromium-review.googlesource.com/c/1446493 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Nico Weber <thakis@chromium.org> Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org>
Olli Etuaho 8ca60805 2018-08-23T14:10:02 Add 2D MS array sampler support to compiler This also places textureSize(gsampler2DMS) correctly in the ESSL 3.10 builtins instead of ESSL 3.00 builtins. BUG=angleproject:2775 TEST=angle_unittests Change-Id: Ieb0f7a7424a5558a5569af6d4fcbcc9b12ec9840 Reviewed-on: https://chromium-review.googlesource.com/1186466 Commit-Queue: Olli Etuaho <oetuaho@nvidia.com> Reviewed-by: Jamie Madill <jmadill@chromium.org>
Jamie Madill f5557acc 2018-06-15T09:46:58 translator: Store symbol type in TField. This allows us to keep a separate symbol type for each field in a struct. This can allow us to assign internal names to struct types. It could also allow us to add internal fields to user defined stucts. Bug: angleproject:2665 Change-Id: I6a129107d9db66c54b98b07684c3ead5801712ba Reviewed-on: https://chromium-review.googlesource.com/1101565 Reviewed-by: Olli Etuaho <oetuaho@nvidia.com>
Olli Etuaho 5fec7ab2 2018-04-04T11:58:33 Identify functions by unique id in BuiltInFunctionEmulator Now that unique ids of all builtins are compile-time constants, we can use them to look up functions in BuiltInFunctionEmulator. This is simpler than using a custom struct with the name and parameters for identifying functions. This requires that we store a reference to a TFunction in those TIntermUnary nodes that were created based on a function. This decreases shader_translator binary size by about 6 KB on Windows. BUG=angleproject:2267 BUG=chromium:823856 TEST=angle_unittests Change-Id: Idd5a00c772c6f26dd36fdbbfbe161d22ab27c2fe Reviewed-on: https://chromium-review.googlesource.com/995372 Reviewed-by: Jamie Madill <jmadill@chromium.org> Commit-Queue: Olli Etuaho <oetuaho@nvidia.com>
Olli Etuaho 89398b65 2018-03-21T17:30:50 Avoid mangled name comparisons of 3-parameter functions The hash values used for looking up built-ins now encode whether the mangled name contains arrays, structs or interface blocks in its parameters list. This is written in the most significant bit of the hash value. We check this bit at the start of built-in lookup and if the bit is set we exit early. After that we know that the lookup name doesn't contain array, struct or interface block parameters. When we find a hash that matches a hash of a built-in function, we now know 3 things: 1) the length of the mangled name matches 2) the open parentheses in the mangled name matches 3) the lookup doesn't contain array, struct or block parameters. Additionally, we have an if statement checking whether the function name matches. Collisions are only possible with functions that 1) have the same name 2) have the same number of parameters With these preconditions we can check beforehand whether collisions are possible for 3-parameter functions. If there are no collisions, we don't need to compare the full mangled name. This is similar to what was already being done with functions that had 0 to 2 parameters. This reduces shader_translator binary size by around 4 KB on Windows. Besides increased complexity, the tradeoff is that an exhaustive search of hash values for possible 3-parameter combinations is costly, so the gen_builtin_functions.py code generation script now takes around one minute to run on a high-end workstation. Due to this, the script now exits early if it detects it has already been run with the same inputs based on a hash value stored in builtin_symbols_hash_autogen.txt. BUG=angleproject:2267 BUG=chromium:823856 TEST=angle_unittests Change-Id: I3ff8c6eb85b90d3c4971ac8d73ee171a07a7e55f Reviewed-on: https://chromium-review.googlesource.com/973372 Reviewed-by: Corentin Wallez <cwallez@chromium.org> Commit-Queue: Olli Etuaho <oetuaho@nvidia.com>
Olli Etuaho 3b678745 2018-03-21T12:59:19 Use a specialized hash function for mangled names The hash values used for looking up built-ins now encode the string length and the location of parentheses as six-bit values, so that we don't need to check for these if the hash matches. This decreases shader_translator binary size on Windows by around 10 KB. BUG=angleproject:2267 BUG=chromium:823856 TEST=angle_unittests Change-Id: If8c28e1c8851750633509ec6273f556e06e91cd1 Reviewed-on: https://chromium-review.googlesource.com/973243 Reviewed-by: Jamie Madill <jmadill@chromium.org> Reviewed-by: Corentin Wallez <cwallez@chromium.org> Commit-Queue: Olli Etuaho <oetuaho@nvidia.com>
Olli Etuaho e79d0f86 2018-03-19T11:55:14 Rely on hash to check for some mangled name matches When we are looking up a function with only a few parameters, we can optimize the lookup by relying on the information encoded in the hash value. There's often only one list of parameters with the same function name and mangled name length that results in a matching hash, so we don't actually need to compare the full mangled name. We can just compare 1) the hash value of the mangled name 2) the mangled name length 3) the function name to make sure that the mangled name matches the mangled name of the function. This decreases the binary size since we don't need store as many mangled names of built-in functions. Effect on symbol lookup speed is marginal. BUG=angleproject:2267 BUG=chromium:823856 TEST=angle_unittests Change-Id: I3ef41d943209509d4e8e6ece14ebad7e2677abc6 Reviewed-on: https://chromium-review.googlesource.com/973242 Reviewed-by: Corentin Wallez <cwallez@chromium.org> Reviewed-by: Jamie Madill <jmadill@chromium.org> Commit-Queue: Olli Etuaho <oetuaho@nvidia.com>
Olli Etuaho 4ba4a82b 2018-03-19T11:28:48 Only refer to built-in function mangled names in lookup TFunction entries for built-ins don't need to store mangled names. BUG=angleproject:2267 TEST=angle_unittests Change-Id: Ia88e8bfa5357719c98d725dc3d00885dd59c9f59 Reviewed-on: https://chromium-review.googlesource.com/973241 Reviewed-by: Corentin Wallez <cwallez@chromium.org> Reviewed-by: Jamie Madill <jmadill@chromium.org> Commit-Queue: Olli Etuaho <oetuaho@nvidia.com>
Olli Etuaho c26214de 2018-03-16T10:43:11 Move AST utilities to a subdirectory Move AST related utilities to compiler/translator/tree_util. BUG=angleproject:2409 TEST=angle_unittests Change-Id: I7567c2f6f2710292029263257c7ac26e2a144ac8 Reviewed-on: https://chromium-review.googlesource.com/966032 Reviewed-by: Jamie Madill <jmadill@chromium.org> Reviewed-by: Corentin Wallez <cwallez@chromium.org> Commit-Queue: Olli Etuaho <oetuaho@nvidia.com>
Olli Etuaho 8574357f 2018-03-19T14:45:53 Share parameter arrays more between built-in functions The parameters of some built-ins are a subarray of the parameters of another built-in. In this kind of case they can point to the same parameters array, and just use a different parameter count. This shaves a few kilobytes from the binary size. BUG=angleproject:2267 TEST=angle_unittests Change-Id: Id166340b4bf5be966bf8c62ce6cd88e5409f647a Reviewed-on: https://chromium-review.googlesource.com/968601 Reviewed-by: Jamie Madill <jmadill@chromium.org> Reviewed-by: Corentin Wallez <cwallez@chromium.org> Commit-Queue: Olli Etuaho <oetuaho@nvidia.com>
Olli Etuaho 115b2c4a 2018-03-15T17:46:29 Remove desktop GLSL builtins from symbol lookups Desktop GLSL builtins can be accessed through functions in BuiltIn_autogen.h. They don't need to be included in symbol table lookups. BUG=angleproject:2267 TEST=angle_unittests Change-Id: I8ba188a0d2584353e34159c2732c9e2bd420c168 Reviewed-on: https://chromium-review.googlesource.com/964447 Reviewed-by: Corentin Wallez <cwallez@chromium.org> Reviewed-by: Jamie Madill <jmadill@chromium.org> Commit-Queue: Olli Etuaho <oetuaho@nvidia.com>
Olli Etuaho b391ec40 2018-03-12T17:04:59 Generate code for looking up built-ins Instead of storing built-ins in a std::unordered_map, we now generate a series of switch statements using the hash value of the look-up string. This works similarly to earlier implementation of looking up unmangled built-ins. Those built-ins that need to be initialized at run-time are stored as member variables of TSymbolTable. This increases compiler init performance significantly, as well as increasing compiler perf test scores around 1-2%. Binary size is larger than before though. BUG=angleproject:2267 TEST=angle_unittests Change-Id: If1dcd36f0d2b30c2ed315cdcf6e831ae9fe70c94 Reviewed-on: https://chromium-review.googlesource.com/960031 Reviewed-by: Jamie Madill <jmadill@chromium.org> Commit-Queue: Olli Etuaho <oetuaho@nvidia.com>
Olli Etuaho d4bd963f 2018-03-08T16:32:44 Don't use TIntermSymbol nodes for function parameters Parameter nodes are not needed - it's simpler to just create a TVariable object for each parameter when the TFunction is initialized. With this change we also store only one object per each parameter type used in built-in functions, instead of one array of TConstParameter entries for each unique parameter sequence. This simplifies code and reduces binary size and compiler memory use. Compiler perf does not seem to be significantly affected. BUG=angleproject:2267 TEST=angle_unittests Change-Id: I2b82400dd594731074309f92a705e75135a4c82c Reviewed-on: https://chromium-review.googlesource.com/955589 Commit-Queue: Olli Etuaho <oetuaho@nvidia.com> Reviewed-by: Jamie Madill <jmadill@chromium.org>
Olli Etuaho 2bfe9f6b 2018-03-02T16:53:29 Use function id to group functions in ParseContext This way we can do numeric comparisons instead of string comparisons. The effect on compiler perf test scores is fairly marginal, but this reduces binary size by a few kilobytes, and there may be a larger effect on shaders calling a lot of texture functions. BUG=angleproject:2267 TEST=angle_unittests Change-Id: I077db97b16b16b70b7e18ee037e06d7450d08dc9 Reviewed-on: https://chromium-review.googlesource.com/947952 Commit-Queue: Olli Etuaho <oetuaho@nvidia.com> Reviewed-by: Jamie Madill <jmadill@chromium.org>
Olli Etuaho e600c0aa 2018-03-02T11:23:29 Use non-human-readable mangled names for types The new mangled name format is as follows: The first character is a hex digit from 0 to F that encodes vector or matrix size. For scalars, structs etc. the character is 0. Then, if it's a struct, the mangled name continues with "{s", followed by mangled names of fields, and ends with "}". If it's an interface block, the mangled name continues with "{i", followed by mangled names of fields, and ends with "}". If it's anything else, the second alphabetic character encodes the basic type. Characters are assigned to basic types in the enumeration order. If it's an array, the mangled name has a suffix [array_size]. This saves a few kilobytes from the binary size. The effect on symbol lookup speed seems mostly marginal. BUG=angleproject:2267 TEST=angle_unittests Change-Id: I26e65dcb48c3478df9a719ffff9c15f2fd12e293 Reviewed-on: https://chromium-review.googlesource.com/945910 Commit-Queue: Olli Etuaho <oetuaho@nvidia.com> Reviewed-by: Corentin Wallez <cwallez@chromium.org>
Olli Etuaho 391bda23 2018-02-23T11:43:14 Generate code for initializing built-in variables gen_builtin_symbols.py now generates code for initializing built-in variable symbols as well. Some of the variable symbols are static, but some of them also get initialized dynamically based on values in ShBuiltInResources. The static symbols have get functions in a header file so they can be referenced from AST traversers as well without doing a lookup. BUG=angleproject:2267 TEST=angle_unittests, angle_end2end_tests Change-Id: Ida7f3aeb06d2bce0f737f1483b1bd5833aeddd2e Reviewed-on: https://chromium-review.googlesource.com/911768 Reviewed-by: Corentin Wallez <cwallez@chromium.org> Commit-Queue: Olli Etuaho <oetuaho@nvidia.com>
Olli Etuaho 065aa863 2018-02-22T15:30:27 Generate code for unmangled name lookup Instead of using an std::map at each symbol table level, use the gen_builtin_symbols.py script to build a function to query unmangled names. BUG=angleproject:2267 TEST=angle_unittests Change-Id: I4f1cf1df1f50fe9d909f3249150ee002ee6efb61 Reviewed-on: https://chromium-review.googlesource.com/931885 Commit-Queue: Olli Etuaho <oetuaho@nvidia.com> Reviewed-by: Jamie Madill <jmadill@chromium.org>
Olli Etuaho 140152e7 2018-02-08T14:46:44 Statically allocate built-in function symbols A script gen_builtin_symbols.py now generates code for initializing built-in function symbols. The TFunction objects are initialized at C++ compile time. The source file used for the functions is in a format that's similar to how functions are given out in the GLSL spec, so it is easy to maintain. The function symbols are still inserted to the symbol table levels same as before. Getting rid of inserting the symbols at runtime is intended to be done as follow-up. This speeds up angle_unittests on Linux in release mode by a bit less than half, and in debug mode by more than half. BUG=angleproject:2267 TEST=angle_unittests Change-Id: I11c9de98c74d28e7e8cdf024516e2f6ee30ca33e Reviewed-on: https://chromium-review.googlesource.com/924155 Commit-Queue: Olli Etuaho <oetuaho@nvidia.com> Reviewed-by: Jamie Madill <jmadill@chromium.org>