scripts/run_code_generation.py


Log

Author Commit Date CI Message
Tim Van Patten 11c487a8 2019-10-25T16:50:01 Manually copy dEQP data files to output directory dEQP has data files that live in several directories within dEQP's source tree. For example, GLES3 has data files that live within: data/gles3/data/ external/graphicsfuzz/data/gles3/ However, we can only tell dEQP about a single data directory during initialization of dEQP. To get around this, we are manually copying all of the necessary data files to the output generated files directory and pointing dEQP to this single directory. This also helps us solve a second problem related to the paths that dEQP uses when accessing graphicsfuzz data files. For the graphicsfuzz tests, dEQP will attempt to open the necessary shaders by accessing them with the path: data/gles3/graphicsfuzz/ However, those files would normally live at the path that matches their location within the source tree: external/graphicsfuzz/data/gles3/graphicsfuzz/ As part of the manual copy of these data files, we are also able to strip the extra 'external/graphicsfuzz/' portion of the path. Bug: angleproject:2322 Test: dEQP Change-Id: Ibc96442c221485e2f246890fa8fe51f090c5e222 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/1881759 Commit-Queue: Tim Van Patten <timvp@google.com> Reviewed-by: Tim Van Patten <timvp@google.com>
Le Quyen 25301999 2019-10-11T14:49:00 Metal implementation pt 1: autogen resources Autogen format table and internal shaders for Metal. Bug: angleproject:2634 Change-Id: I4d81fcd17a0e9959ba9c38a250acc7abb168f54c Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/1855067 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Jamie Madill <jmadill@chromium.org>
Shahbaz Youssefi 050b124d 2019-06-30T03:26:18 Reland "Vulkan: Debug overlay" This is a reland of e54d0f90d1a165404236fd7abd1b05ddd041a686 This was reverted due to a build failure as a result of a missing virtual destructor in the widget base class. Original change's description: > Vulkan: Debug overlay > > A debug overlay system for the Vulkan backend designed with efficiency > and runtime configurability in mind. Overlay widgets are of two > fundamental types: > > - Text widgets: A single line of text with small, medium or large font. > - Graph widgets: A bar graph of data. > > Built on these, various overlay widget types are defined that gather > statistics. Five such types are defined with one widget per type as > example: > > - Count: A widget that counts something. VulkanValidationMessageCount > is an overlay widget of this type that shows the number of validation > messages received from the validation layers. > - Text: A generic text. VulkanLastValidationMessage is an overlay > widget of this type that shows the last validation message. > - PerSecond: A value that gets reset every second automatically. FPS is > an overlay widget of this type that simply gets incremented on every > swap(). > - RunningGraph: A graph of last N values. VulkanCommandGraphSize is an > overlay of this type. On every vkQueueSubmit, the number of nodes in > the command graph is accumulated. On every present(), the value is > taken as the number of nodes for the whole duration of the frame. > - RunningHistogram: A histogram of last N values. Input values are in > the [0, 1] range and they are ranked to N buckets for histogram > calculation. VulkanSecondaryCommandBufferPoolWaste is an overlay > widget of this type. On vkQueueSubmit, the memory waste from command > buffer pool allocations is recorded in the histogram. > > Overlay font is placed in libANGLE/overlay/ which gen_overlay_fonts.py > processes to create an array of bits, which is processed at runtime to > create the actual font image (an image with 3 layers). > > The overlay widget layout is defined in overlay_widgets.json which > gen_overlay_widgets.py processes to generate an array of widgetss, each > of its respective type, and sets their properties, such as color and > bounding box. The json file allows widgets to align against other > widgets as well as against the framebuffer edges. > > Two compute shaders are implemented to efficiently render the UI: > > - OverlayCull: This shader creates a bitset of Text and Graph widgets > whose bounding boxes intersect a corresponding subgroup processed by > OverlayDraw. This is done only when the enabled overlay widgets are > changed (a feature that is not yet implemented) or the surface is > resized. > - OverlayDraw: Using the bitsets generated by OverlayCull, values that > are uniform for each workgroup (set to be equal to hardware subgroup > size), this shader loops over enabled widgets that can possibly > intersect the pixel being processed and renders and blends in texts > and graphs. This is done once per frame on present(). > > Currently, to enable overlay widgets an environment variable is used. > For example: > > $ export ANGLE_OVERLAY=FPS:VulkanSecondaryCommandBufferPoolWaste > $ ./hello_triangle --use-angle=vulkan > > Possible future work: > > - On Android, add settings in developer options and enable widgets based > on those. > - Spawn a small server in ANGLE and write an application that sends > enable/disable commands remotely. > - Implement overlay for other backends. > > Bug: angleproject:3757 > Change-Id: If9c6974d1935c18f460ec569e79b41188bd7afcc > Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/1729440 > Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org> > Reviewed-by: Jamie Madill <jmadill@chromium.org> Bug: angleproject:3757 Change-Id: I47915d88b37b6f882c686c2de13fca309a10b572 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/1780897 Reviewed-by: Jamie Madill <jmadill@chromium.org> Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org>
Corentin Wallez fc58af47 2019-09-02T07:46:44 Revert "Vulkan: Debug overlay" This reverts commit e54d0f90d1a165404236fd7abd1b05ddd041a686. Reason for revert: causes compile failure on Linux CFI bot. Sample build: https://ci.chromium.org/p/chromium/builders/ci/Linux%20CFI/14810 Sample log: https://logs.chromium.org/logs/chromium/buildbucket/cr-buildbucket.appspot.com/8903575125463586160/+/steps/compile/0/stdout?format=raw Original change's description: > Vulkan: Debug overlay > > A debug overlay system for the Vulkan backend designed with efficiency > and runtime configurability in mind. Overlay widgets are of two > fundamental types: > > - Text widgets: A single line of text with small, medium or large font. > - Graph widgets: A bar graph of data. > > Built on these, various overlay widget types are defined that gather > statistics. Five such types are defined with one widget per type as > example: > > - Count: A widget that counts something. VulkanValidationMessageCount > is an overlay widget of this type that shows the number of validation > messages received from the validation layers. > - Text: A generic text. VulkanLastValidationMessage is an overlay > widget of this type that shows the last validation message. > - PerSecond: A value that gets reset every second automatically. FPS is > an overlay widget of this type that simply gets incremented on every > swap(). > - RunningGraph: A graph of last N values. VulkanCommandGraphSize is an > overlay of this type. On every vkQueueSubmit, the number of nodes in > the command graph is accumulated. On every present(), the value is > taken as the number of nodes for the whole duration of the frame. > - RunningHistogram: A histogram of last N values. Input values are in > the [0, 1] range and they are ranked to N buckets for histogram > calculation. VulkanSecondaryCommandBufferPoolWaste is an overlay > widget of this type. On vkQueueSubmit, the memory waste from command > buffer pool allocations is recorded in the histogram. > > Overlay font is placed in libANGLE/overlay/ which gen_overlay_fonts.py > processes to create an array of bits, which is processed at runtime to > create the actual font image (an image with 3 layers). > > The overlay widget layout is defined in overlay_widgets.json which > gen_overlay_widgets.py processes to generate an array of widgetss, each > of its respective type, and sets their properties, such as color and > bounding box. The json file allows widgets to align against other > widgets as well as against the framebuffer edges. > > Two compute shaders are implemented to efficiently render the UI: > > - OverlayCull: This shader creates a bitset of Text and Graph widgets > whose bounding boxes intersect a corresponding subgroup processed by > OverlayDraw. This is done only when the enabled overlay widgets are > changed (a feature that is not yet implemented) or the surface is > resized. > - OverlayDraw: Using the bitsets generated by OverlayCull, values that > are uniform for each workgroup (set to be equal to hardware subgroup > size), this shader loops over enabled widgets that can possibly > intersect the pixel being processed and renders and blends in texts > and graphs. This is done once per frame on present(). > > Currently, to enable overlay widgets an environment variable is used. > For example: > > $ export ANGLE_OVERLAY=FPS:VulkanSecondaryCommandBufferPoolWaste > $ ./hello_triangle --use-angle=vulkan > > Possible future work: > > - On Android, add settings in developer options and enable widgets based > on those. > - Spawn a small server in ANGLE and write an application that sends > enable/disable commands remotely. > - Implement overlay for other backends. > > Bug: angleproject:3757 > Change-Id: If9c6974d1935c18f460ec569e79b41188bd7afcc > Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/1729440 > Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org> > Reviewed-by: Jamie Madill <jmadill@chromium.org> TBR=geofflang@chromium.org,syoussefi@chromium.org,jmadill@chromium.org Bug: angleproject:3757 Change-Id: Ib08e2e7b1a9449ca097673acb11655df5d2bbf31 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/1778862 Commit-Queue: Corentin Wallez <cwallez@chromium.org> Reviewed-by: Corentin Wallez <cwallez@chromium.org>
Shahbaz Youssefi e54d0f90 2019-06-30T03:26:18 Vulkan: Debug overlay A debug overlay system for the Vulkan backend designed with efficiency and runtime configurability in mind. Overlay widgets are of two fundamental types: - Text widgets: A single line of text with small, medium or large font. - Graph widgets: A bar graph of data. Built on these, various overlay widget types are defined that gather statistics. Five such types are defined with one widget per type as example: - Count: A widget that counts something. VulkanValidationMessageCount is an overlay widget of this type that shows the number of validation messages received from the validation layers. - Text: A generic text. VulkanLastValidationMessage is an overlay widget of this type that shows the last validation message. - PerSecond: A value that gets reset every second automatically. FPS is an overlay widget of this type that simply gets incremented on every swap(). - RunningGraph: A graph of last N values. VulkanCommandGraphSize is an overlay of this type. On every vkQueueSubmit, the number of nodes in the command graph is accumulated. On every present(), the value is taken as the number of nodes for the whole duration of the frame. - RunningHistogram: A histogram of last N values. Input values are in the [0, 1] range and they are ranked to N buckets for histogram calculation. VulkanSecondaryCommandBufferPoolWaste is an overlay widget of this type. On vkQueueSubmit, the memory waste from command buffer pool allocations is recorded in the histogram. Overlay font is placed in libANGLE/overlay/ which gen_overlay_fonts.py processes to create an array of bits, which is processed at runtime to create the actual font image (an image with 3 layers). The overlay widget layout is defined in overlay_widgets.json which gen_overlay_widgets.py processes to generate an array of widgetss, each of its respective type, and sets their properties, such as color and bounding box. The json file allows widgets to align against other widgets as well as against the framebuffer edges. Two compute shaders are implemented to efficiently render the UI: - OverlayCull: This shader creates a bitset of Text and Graph widgets whose bounding boxes intersect a corresponding subgroup processed by OverlayDraw. This is done only when the enabled overlay widgets are changed (a feature that is not yet implemented) or the surface is resized. - OverlayDraw: Using the bitsets generated by OverlayCull, values that are uniform for each workgroup (set to be equal to hardware subgroup size), this shader loops over enabled widgets that can possibly intersect the pixel being processed and renders and blends in texts and graphs. This is done once per frame on present(). Currently, to enable overlay widgets an environment variable is used. For example: $ export ANGLE_OVERLAY=FPS:VulkanSecondaryCommandBufferPoolWaste $ ./hello_triangle --use-angle=vulkan Possible future work: - On Android, add settings in developer options and enable widgets based on those. - Spawn a small server in ANGLE and write an application that sends enable/disable commands remotely. - Implement overlay for other backends. Bug: angleproject:3757 Change-Id: If9c6974d1935c18f460ec569e79b41188bd7afcc Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/1729440 Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org> Reviewed-by: Jamie Madill <jmadill@chromium.org>
Clemen Deng d7d42540 2019-08-21T15:22:49 Don't build symbol table for GLSL built-ins if on Android The GLSL + ESSL autogenerated symbol table is too large for android, and android also doesn't need desktop GL functionality If on android, compile the ESSL only symbol table Bug: chromium:996286 Change-Id: I14dfc7748dae389e78c35f82a390c67962665356 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/1757372 Commit-Queue: Clemen Deng <clemendeng@google.com> Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org> Reviewed-by: Geoff Lang <geofflang@chromium.org>
Clemen Deng 44f518b5 2019-08-06T13:31:01 Have run_code_generation only call vpython when needed gen_builtin_symbols.py uses a vpython module to generate a perfect hash function Also seeded the perfect hash function to make it deterministic Bug: angleproject:3747 Change-Id: I660fe71bd6b2213be9d4ccc2f68641637a49a047 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/1738747 Commit-Queue: Clemen Deng <clemendeng@google.com> Reviewed-by: Geoff Lang <geofflang@chromium.org>
Jiacheng Lu f3dbf0a7 2019-07-29T11:31:20 FrameCapture dump GLenum to enum instead of value Add functionality to capture GLenum and GLbitfield parameters as enum names instead of values. Bug: angleproject:3611 Change-Id: I7ddc21093d44d1480b205563f30d89f82779b1f1 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/1724460 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Jamie Madill <jmadill@chromium.org>
Clemen Deng 9031bdd9 2019-07-26T14:51:23 Use perfect-hash module in gen_builtin_symbols.py The script currently takes ~4 minutes to run Using this module instead of manually hashing will improve runtime significantly Bug: angleproject:3747 Change-Id: I7e2d2ef5bbfd136b0299d571e0acc11f334c80b5 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/1724667 Commit-Queue: Clemen Deng <clemendeng@google.com> Reviewed-by: Jamie Madill <jmadill@chromium.org>
Jamie Madill 53aff41e 2019-07-11T14:06:23 Use separate json files for each code generator. This should reduce the number of merge conflicts when multiple people touch different generators. There still will be merge conflicts when two people touch the same code generator. Bug: angleproject:3691 Change-Id: I0b179368a1352331c091972568c00213ab106e46 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/1698650 Reviewed-by: Yuly Novikov <ynovikov@chromium.org> Reviewed-by: Ian Elliott <ianelliott@google.com> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Clemen Deng 7558e836 2019-06-18T13:02:03 Windows Desktop GL Implementation Bug: angleproject:3620 Change-Id: I4ef4ab3ee145e5ce9b1ebf0c2d61d0777db72c43 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/1678405 Reviewed-by: Jamie Madill <jmadill@chromium.org> Reviewed-by: Geoff Lang <geofflang@chromium.org> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Geoff Lang d7d42395 2019-05-06T13:15:35 Format all of ANGLE's python code. BUG=angleproject:3421 Change-Id: I1d7282ac513c046de5d8ed87f7789290780d30a6 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/1595440 Reviewed-by: Jamie Madill <jmadill@chromium.org> Commit-Queue: Geoff Lang <geofflang@chromium.org>
Jamie Madill 13264033 2019-04-03T15:18:16 Minor cleanups to run_code_generation. Prints more info when a particular file is not found. Bug: angleproject:3333 Change-Id: I11ceb0d319023cd1fd23fd25297e5367030342a0 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/1550899 Reviewed-by: Yuly Novikov <ynovikov@chromium.org> Reviewed-by: Jamie Madill <jmadill@chromium.org> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Jamie Madill 3c1f5a6b 2019-03-12T11:18:15 Include script outputs in run_code_generation.py. This will prevent incorrect modifications to outputs from being checked into the repository. Requires a few changes to some generators. Bug: angleproject:3227 Change-Id: I5285cb78a9d85df155a5272edf8b6b8cd27fc04c Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/1515212 Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org> Reviewed-by: Geoff Lang <geofflang@chromium.org> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Jamie Madill 0f34f3f2 2019-03-11T10:18:57 Use auto_script in run_code_generation. Cleans up the generator scripts to prepare for listing outputs in the generated hashes file. Also reorganizes the scripts somewhat to make them a bit more maintainable. Bug: angleproject:3227 Change-Id: If40946c2004941d3f6cd51d5521c7db8cc0b52df Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/1512052 Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org> Reviewed-by: Geoff Lang <geofflang@chromium.org> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Jamie Madill 14126505 2019-03-06T21:13:20 Revert "Use a constexpr array for es3 copy conversion table." This reverts commit f30808db86e31b8b301eb9ec07cfe785d60b6a16. Reason for revert: build/android/gyp/assert_static_initializers.py thinks this adds a static initializer. See https://ci.chromium.org/p/chromium/builders/try/android-marshmallow-arm64-rel/208664 Need to revert since this is blocking the roll. Original change's description: > Use a constexpr array for es3 copy conversion table. > > With the relaxed C++14 constexpr rules allowed in Chromium, we can > use a constexpr sorted array to store our table data. This can lead > to very fast lookups while being more maintanable than using auto- > generator scripts for every lookup table. > > Note that to be sure this syntax is permitted, we should land this > through the bots and let it sit for a little while. > > Bug: angleproject:1389 > Change-Id: I9395c40276470108ce3e5786d8f1b8d85462c517 > Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/777544 > Commit-Queue: Jamie Madill <jmadill@google.com> > Reviewed-by: Yuly Novikov <ynovikov@chromium.org> TBR=ynovikov@chromium.org,jmadill@google.com,syoussefi@chromium.org,jmadill@chromium.org # Not skipping CQ checks because original CL landed > 1 day ago. Bug: angleproject:1389 Change-Id: I482729b6f16975896b0e5c29999f9a081056e800 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/1506238 Reviewed-by: Jamie Madill <jmadill@chromium.org> Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Jamie Madill f30808db 2019-03-05T10:55:39 Use a constexpr array for es3 copy conversion table. With the relaxed C++14 constexpr rules allowed in Chromium, we can use a constexpr sorted array to store our table data. This can lead to very fast lookups while being more maintanable than using auto- generator scripts for every lookup table. Note that to be sure this syntax is permitted, we should land this through the bots and let it sit for a little while. Bug: angleproject:1389 Change-Id: I9395c40276470108ce3e5786d8f1b8d85462c517 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/777544 Commit-Queue: Jamie Madill <jmadill@google.com> Reviewed-by: Yuly Novikov <ynovikov@chromium.org>
Jamie Madill 2f1c1d7e 2019-03-04T17:02:52 Minor improvements to run_code_generation. Isolate variables better. And add a check for out-of-date hases. Currently if a generator is removed the hashes will stick around. Bug: angleproject:3227 Change-Id: I9b9da245b9b29093b1f3f320a81810cd7da82395 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/1500575 Commit-Queue: Jamie Madill <jmadill@google.com> Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org>
Shahbaz Youssefi 98a3550b 2019-01-08T22:09:39 Add a presubmit step to make sure autogenerated code is up-to-date While here, added a presubmit test for the Bug: tag too. Bug: angleproject:3046 Change-Id: I7030685230b4ce4bfc435c14ef85324e16b76274 Reviewed-on: https://chromium-review.googlesource.com/c/1402061 Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org> Reviewed-by: Jamie Madill <jmadill@chromium.org>
Jamie Madill 4638dc9d 2018-12-17T13:13:49 Re-land "Load correct libGLESv2 on Linux and Mac." Re-land fixes build to ensure commit_id is built before libEGL. libEGL was implicitly loading libGLESv2 on startup. This is bad because on platforms like Linux and Mac we could sometimes use the incorrect rpath. This in turn meant we needed workarounds like using "_angle" extensions to our shared objects to get the correct loading behaviour. Fix this by loading libGLESv2 dynamically in libEGL. We build the loader automatically from egl.xml. The loader itself is lazily initialized on every EGL entry point call. This is necessary because on Linux, etc, there is no equivalent to Windows' DLLMain. We also use an EGL.h with different generation options so we have the proper function pointer types. A README is included for instructions on how to regenerate EGL.h. The entry point generation script is refactored into a helper class that is used in the loader generator. Also adds the libGLESv2 versions of the EGL entry points in the DEF file on Windows. This allows them to be imported properly in 32-bit configurations. Also fixes up some errors in ANGLE's entry point definitions. Also includes a clang-format disable rule for the Khronos headers. This CL will help us to run ANGLE tests against native drivers. Bug: angleproject:2871 Bug: chromium:915731 Change-Id: I4192a938d1f4117cea1bf1399c98bda7ac25ddab Reviewed-on: https://chromium-review.googlesource.com/c/1380511 Reviewed-by: Yuly Novikov <ynovikov@chromium.org> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Yuly Novikov 175d918a 2018-12-16T19:53:23 Revert "Load correct libGLESv2 on Linux and Mac." This reverts commit dd815b623e60a1e1550f328104ffcd7caf20fde1. Reason for revert: Broke https://luci-milo.appspot.com/p/chromium/builders/luci.chromium.ci/win-rel/8006 Original change's description: > Load correct libGLESv2 on Linux and Mac. > > libEGL was implicitly loading libGLESv2 on startup. This is bad > because on platforms like Linux and Mac we could sometimes use the > incorrect rpath. This in turn meant we needed workarounds like using > "_angle" extensions to our shared objects to get the correct loading > behaviour. > > Fix this by loading libGLESv2 dynamically in libEGL. We build the > loader automatically from egl.xml. The loader itself is lazily > initialized on every EGL entry point call. This is necessary because > on Linux, etc, there is no equivalent to Windows' DLLMain. > > We also use an EGL.h with different generation options so we have the > proper function pointer types. A README is included for instructions > on how to regenerate EGL.h. > > The entry point generation script is refactored into a helper class > that is used in the loader generator. Also adds the libGLESv2 versions > of the EGL entry points in the DEF file on Windows. This allows them to > be imported properly in 32-bit configurations. > > Also fixes up some errors in ANGLE's entry point definitions. Also > includes a clang-format disable rule for the Khronos headers. > > This CL will help us to run ANGLE tests against native drivers. > > Bug: angleproject:2871 > Change-Id: Id6ecf969308f17b1be4083538428c9c1a1836572 > Reviewed-on: https://chromium-review.googlesource.com/c/1370725 > Commit-Queue: Jamie Madill <jmadill@chromium.org> > Reviewed-by: Geoff Lang <geofflang@chromium.org> TBR=ynovikov@chromium.org,geofflang@chromium.org,jmadill@chromium.org Change-Id: I921b3c45435ab4f05cbc2d1c1172b4185d6257b0 No-Presubmit: true No-Tree-Checks: true No-Try: true Bug: angleproject:2871 Reviewed-on: https://chromium-review.googlesource.com/c/1378887 Reviewed-by: Yuly Novikov <ynovikov@chromium.org> Commit-Queue: Yuly Novikov <ynovikov@chromium.org>
Jamie Madill dd815b62 2018-12-15T10:39:00 Load correct libGLESv2 on Linux and Mac. libEGL was implicitly loading libGLESv2 on startup. This is bad because on platforms like Linux and Mac we could sometimes use the incorrect rpath. This in turn meant we needed workarounds like using "_angle" extensions to our shared objects to get the correct loading behaviour. Fix this by loading libGLESv2 dynamically in libEGL. We build the loader automatically from egl.xml. The loader itself is lazily initialized on every EGL entry point call. This is necessary because on Linux, etc, there is no equivalent to Windows' DLLMain. We also use an EGL.h with different generation options so we have the proper function pointer types. A README is included for instructions on how to regenerate EGL.h. The entry point generation script is refactored into a helper class that is used in the loader generator. Also adds the libGLESv2 versions of the EGL entry points in the DEF file on Windows. This allows them to be imported properly in 32-bit configurations. Also fixes up some errors in ANGLE's entry point definitions. Also includes a clang-format disable rule for the Khronos headers. This CL will help us to run ANGLE tests against native drivers. Bug: angleproject:2871 Change-Id: Id6ecf969308f17b1be4083538428c9c1a1836572 Reviewed-on: https://chromium-review.googlesource.com/c/1370725 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Geoff Lang <geofflang@chromium.org>
Jamie Madill 8dc27f99 2018-11-29T11:45:44 Use packed enum for DrawElementsType. The packing and unpacking take a few extra instructions. But it completely obviates the need for any switches in the validation code. Speed is slightly faster or the similar depending on the back-end. Also add gl_angle_ext.xml to GL entry point generator inputs. This was missing and would cause the code generation to miss certain changes. Bug: angleproject:2985 Change-Id: I1ea41a71db71135000166ead8305ec42d22ff7b3 Reviewed-on: https://chromium-review.googlesource.com/c/1351729 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org>
Brandon Jones b77884b7 2018-10-10T09:07:12 Autogenerate Repeated Portions of Blit11 Introduces a script to generate Blit11's BlitShaderType enum and shader mapping functions. Bug: angleproject:2870 Change-Id: I5b704f55bdde1706614a519ecb0ee3756f7799e0 Reviewed-on: https://chromium-review.googlesource.com/c/1273725 Reviewed-by: Jamie Madill <jmadill@chromium.org> Reviewed-by: Geoff Lang <geofflang@chromium.org> Commit-Queue: Geoff Lang <geofflang@chromium.org>
jchen10 547edfe2 2018-09-25T21:17:56 Fix angle::Format::ID in gen_dxgi_format_table.py The dxgi_format_map_autogen.cpp was updated, but the generating script was missed to submit in https://crrev.com/c/1142299. Bug: angleproject:2729 Change-Id: I8d726854581d27881a737f12457482ac955e6312 Reviewed-on: https://chromium-review.googlesource.com/1242853 Commit-Queue: Jie A Chen <jie.a.chen@intel.com> Reviewed-by: Geoff Lang <geofflang@chromium.org>
Frank Henigman 569b9cb9 2018-07-07T21:44:35 Fix run_code_generation.py hash calculation. Open files with mode 'r' instead of 'rb' so the hash calculation produces the same results on Linux and Windows. Recalculate the hashes and proc_table_autogen.cpp which was out of date. BUG=angleproject:2711 Change-Id: I31562c96ce36e6df009c44c565fe9a3f1b5ba6c4 Reviewed-on: https://chromium-review.googlesource.com/1128549 Reviewed-by: Jamie Madill <jmadill@chromium.org> Commit-Queue: Frank Henigman <fjhenigman@chromium.org>
Geoff Lang b1cc789c 2018-07-05T15:05:21 Finish populating the new_hashes map after finding a dirty input. any_input_dirty would return early when finding a dirty input and not finish filling the new_hashes map. This would require multiple runs of the code generation script to fully generate all the outputs. BUG=angleproject:2695 Change-Id: Ie62190efe2765df432b0a535fb8d33ed2ffa66a7 Reviewed-on: https://chromium-review.googlesource.com/1127439 Commit-Queue: Geoff Lang <geofflang@chromium.org> Reviewed-by: Jamie Madill <jmadill@chromium.org>
Geoff Lang 9a257801 2018-07-05T14:31:08 Write relative paths with forward slashes to the code generation hashes. This should avoid generating diffs when running the code generation with different source directories or on different platforms. BUG=angleproject:2695 Change-Id: I67776883bdbeb867a49bea00f16998c04f7857b4 Reviewed-on: https://chromium-review.googlesource.com/1127355 Commit-Queue: Geoff Lang <geofflang@chromium.org> Reviewed-by: Jamie Madill <jmadill@chromium.org>
Geoff Lang 5695a5b2 2018-07-05T14:29:30 Allow the run_code_generation.py script to be run from any directory. BUG=angleproject:2695 Change-Id: I9a083bf67b17ca0dc1ee213b75caa4ecece62f00 Reviewed-on: https://chromium-review.googlesource.com/1127354 Commit-Queue: Geoff Lang <geofflang@chromium.org> Reviewed-by: Jamie Madill <jmadill@chromium.org>
Jamie Madill a72f400c 2018-06-29T17:05:01 run_code_generation: Compare hashes instead of mtime. Using mtime is fundamentally flawed when working with git. Replace these flaky checks with hash comparisons. The hashes are stored in an autogenerated json file that will be stored in the repository. This makes the run_code_generation script robust against any and all input changes. It also removes the need to track script outputs as dependencies. Bug: angleproject:2697 Change-Id: I60f2a87a8680b1f775ad678b05112f5b16c7dde7 Reviewed-on: https://chromium-review.googlesource.com/1120159 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Yuly Novikov <ynovikov@chromium.org>
Frank Henigman 388f991c 2018-06-15T17:18:09 Vulkan: fix third_party path. run_code_generation.py stopped working, likely as a result of the big Vulkan repo reorganization. I fixed a path, regenerated everything (by temporarily hacking the script so it thought everything was dirty), and nothing changed except one comment. BUG=angleproject:2558 Change-Id: I5ac4c040ac1ec207098172303cc2f2507cccdecb Reviewed-on: https://chromium-review.googlesource.com/1103281 Reviewed-by: Yuly Novikov <ynovikov@chromium.org> Reviewed-by: Jamie Madill <jmadill@chromium.org> Reviewed-by: Tobin Ehlis <tobine@google.com>
Jamie Madill d4703d50 2018-05-24T17:31:43 Move packed enum code to common/ This makes it accessible in the utilities files. Bug: angleproject:2574 Bug: angleproject:2169 Change-Id: I0fdd34b4233e72b7534cb2b09f451539c1a394cd Reviewed-on: https://chromium-review.googlesource.com/1067110 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Yuly Novikov <ynovikov@chromium.org>
Jamie Madill 1c597eea 2018-05-24T14:19:31 Fix run_code_generation for huge diffs. This uses the --full argument to work around max path limitations on Windows. Also includes a couple minor changes for generators to generate already-formatted code. Bug: angleproject:2578 Change-Id: I1e400b02e828bfdca21cacb73c649f41226bef55 Reviewed-on: https://chromium-review.googlesource.com/1072161 Reviewed-by: Lingfeng Yang <lfy@google.com> Reviewed-by: Luc Ferron <lucferron@chromium.org> Reviewed-by: Brandon1 Jones <brandon1.jones@intel.com> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Jamie Madill d47044ad 2018-04-27T11:45:03 Vulkan: Add framework for internal shaders. Vulkan intenal shaders are stored in a ShaderLibrary, and this is owned by the RendererVk. This way the shaders are reused between all the different Contexts. They are initialized lazily to keep init time low. They also have an associated Serial (called a ProgramSerial) so they can be identified in a PipelineDesc (used by the Pipeline cache). We use a python script to build and invoke the glslang validator, that also produces SPIR-V binary code snippets. These snippets are gathered into an auto-generated file that is exposed via an auto-generated header file. The InternalShaderID enum class gives access to the internal shaders that are shared through the Vulkan back-end. This also adds simple clear shaders to be used in masked color clears. The patch doesn't add any functionality but it is split off from the color clear functionality to keep the code size down. Bug: angleproject:2339 Bug: angleproject:2455 Change-Id: Ie83043eda217c9f013817b198c92a3b7ba0878b4 Reviewed-on: https://chromium-review.googlesource.com/1031372 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Geoff Lang <geofflang@chromium.org>
Jamie Madill 0ade8e88 2018-04-27T11:45:02 Use time tolerance in run_code_generation check. The mtime values I was seeing on my file system were sometimes floating point numbers that were very very close but not exactly the same. This might be due to numerical errors. Increasing the tolerance to something still very small, precision is less than a second. It will lead to fewer rebuilds of non-dirty files. Bug: angleproject:2455 Change-Id: I95dc3214ee91af7a70a20cc625405e0e2bc18698 Reviewed-on: https://chromium-review.googlesource.com/1032855 Reviewed-by: Luc Ferron <lucferron@chromium.org> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Geoff Lang 8ceea819 2018-04-10T03:07:13 Refactor packed enum generation to support EGL enums. Convert the very simple EGL texture type enum. BUG=angleproject:1618 Change-Id: Ieea382a282a8f2544f2982627e8445e6e5cea826 Reviewed-on: https://chromium-review.googlesource.com/1019386 Commit-Queue: Geoff Lang <geofflang@chromium.org> Reviewed-by: Geoff Lang <geofflang@chromium.org>
Olli Etuaho 5fec7ab2 2018-04-04T11:58:33 Identify functions by unique id in BuiltInFunctionEmulator Now that unique ids of all builtins are compile-time constants, we can use them to look up functions in BuiltInFunctionEmulator. This is simpler than using a custom struct with the name and parameters for identifying functions. This requires that we store a reference to a TFunction in those TIntermUnary nodes that were created based on a function. This decreases shader_translator binary size by about 6 KB on Windows. BUG=angleproject:2267 BUG=chromium:823856 TEST=angle_unittests Change-Id: Idd5a00c772c6f26dd36fdbbfbe161d22ab27c2fe Reviewed-on: https://chromium-review.googlesource.com/995372 Reviewed-by: Jamie Madill <jmadill@chromium.org> Commit-Queue: Olli Etuaho <oetuaho@nvidia.com>
Olli Etuaho 89398b65 2018-03-21T17:30:50 Avoid mangled name comparisons of 3-parameter functions The hash values used for looking up built-ins now encode whether the mangled name contains arrays, structs or interface blocks in its parameters list. This is written in the most significant bit of the hash value. We check this bit at the start of built-in lookup and if the bit is set we exit early. After that we know that the lookup name doesn't contain array, struct or interface block parameters. When we find a hash that matches a hash of a built-in function, we now know 3 things: 1) the length of the mangled name matches 2) the open parentheses in the mangled name matches 3) the lookup doesn't contain array, struct or block parameters. Additionally, we have an if statement checking whether the function name matches. Collisions are only possible with functions that 1) have the same name 2) have the same number of parameters With these preconditions we can check beforehand whether collisions are possible for 3-parameter functions. If there are no collisions, we don't need to compare the full mangled name. This is similar to what was already being done with functions that had 0 to 2 parameters. This reduces shader_translator binary size by around 4 KB on Windows. Besides increased complexity, the tradeoff is that an exhaustive search of hash values for possible 3-parameter combinations is costly, so the gen_builtin_functions.py code generation script now takes around one minute to run on a high-end workstation. Due to this, the script now exits early if it detects it has already been run with the same inputs based on a hash value stored in builtin_symbols_hash_autogen.txt. BUG=angleproject:2267 BUG=chromium:823856 TEST=angle_unittests Change-Id: I3ff8c6eb85b90d3c4971ac8d73ee171a07a7e55f Reviewed-on: https://chromium-review.googlesource.com/973372 Reviewed-by: Corentin Wallez <cwallez@chromium.org> Commit-Queue: Olli Etuaho <oetuaho@nvidia.com>
Olli Etuaho c26214de 2018-03-16T10:43:11 Move AST utilities to a subdirectory Move AST related utilities to compiler/translator/tree_util. BUG=angleproject:2409 TEST=angle_unittests Change-Id: I7567c2f6f2710292029263257c7ac26e2a144ac8 Reviewed-on: https://chromium-review.googlesource.com/966032 Reviewed-by: Jamie Madill <jmadill@chromium.org> Reviewed-by: Corentin Wallez <cwallez@chromium.org> Commit-Queue: Olli Etuaho <oetuaho@nvidia.com>
Olli Etuaho b391ec40 2018-03-12T17:04:59 Generate code for looking up built-ins Instead of storing built-ins in a std::unordered_map, we now generate a series of switch statements using the hash value of the look-up string. This works similarly to earlier implementation of looking up unmangled built-ins. Those built-ins that need to be initialized at run-time are stored as member variables of TSymbolTable. This increases compiler init performance significantly, as well as increasing compiler perf test scores around 1-2%. Binary size is larger than before though. BUG=angleproject:2267 TEST=angle_unittests Change-Id: If1dcd36f0d2b30c2ed315cdcf6e831ae9fe70c94 Reviewed-on: https://chromium-review.googlesource.com/960031 Reviewed-by: Jamie Madill <jmadill@chromium.org> Commit-Queue: Olli Etuaho <oetuaho@nvidia.com>
Olli Etuaho 2bfe9f6b 2018-03-02T16:53:29 Use function id to group functions in ParseContext This way we can do numeric comparisons instead of string comparisons. The effect on compiler perf test scores is fairly marginal, but this reduces binary size by a few kilobytes, and there may be a larger effect on shaders calling a lot of texture functions. BUG=angleproject:2267 TEST=angle_unittests Change-Id: I077db97b16b16b70b7e18ee037e06d7450d08dc9 Reviewed-on: https://chromium-review.googlesource.com/947952 Commit-Queue: Olli Etuaho <oetuaho@nvidia.com> Reviewed-by: Jamie Madill <jmadill@chromium.org>
Luc Ferron 4ea3b450 2018-03-13T11:48:26 run_code_generation.py: Add gen_load_functions_table.py to the list Bug:angleproject:2358 Change-Id: I93bdbbe65ac8109a5a506d88202715c51918613b Reviewed-on: https://chromium-review.googlesource.com/960574 Reviewed-by: Jamie Madill <jmadill@chromium.org> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Olli Etuaho 391bda23 2018-02-23T11:43:14 Generate code for initializing built-in variables gen_builtin_symbols.py now generates code for initializing built-in variable symbols as well. Some of the variable symbols are static, but some of them also get initialized dynamically based on values in ShBuiltInResources. The static symbols have get functions in a header file so they can be referenced from AST traversers as well without doing a lookup. BUG=angleproject:2267 TEST=angle_unittests, angle_end2end_tests Change-Id: Ida7f3aeb06d2bce0f737f1483b1bd5833aeddd2e Reviewed-on: https://chromium-review.googlesource.com/911768 Reviewed-by: Corentin Wallez <cwallez@chromium.org> Commit-Queue: Olli Etuaho <oetuaho@nvidia.com>
Olli Etuaho 7618eaf9 2018-03-07T11:25:40 Fix incorrect path in run_code_generation.py Adding missing "src". BUG=angleproject:2267 TEST=python scripts/run_code_generation.py Change-Id: I89d981c8f09e6d0839d0b796a2ea14a8466f98a7 Reviewed-on: https://chromium-review.googlesource.com/952925 Reviewed-by: Olli Etuaho <oetuaho@nvidia.com> Commit-Queue: Olli Etuaho <oetuaho@nvidia.com>
Olli Etuaho 065aa863 2018-02-22T15:30:27 Generate code for unmangled name lookup Instead of using an std::map at each symbol table level, use the gen_builtin_symbols.py script to build a function to query unmangled names. BUG=angleproject:2267 TEST=angle_unittests Change-Id: I4f1cf1df1f50fe9d909f3249150ee002ee6efb61 Reviewed-on: https://chromium-review.googlesource.com/931885 Commit-Queue: Olli Etuaho <oetuaho@nvidia.com> Reviewed-by: Jamie Madill <jmadill@chromium.org>
Olli Etuaho 140152e7 2018-02-08T14:46:44 Statically allocate built-in function symbols A script gen_builtin_symbols.py now generates code for initializing built-in function symbols. The TFunction objects are initialized at C++ compile time. The source file used for the functions is in a format that's similar to how functions are given out in the GLSL spec, so it is easy to maintain. The function symbols are still inserted to the symbol table levels same as before. Getting rid of inserting the symbols at runtime is intended to be done as follow-up. This speeds up angle_unittests on Linux in release mode by a bit less than half, and in debug mode by more than half. BUG=angleproject:2267 TEST=angle_unittests Change-Id: I11c9de98c74d28e7e8cdf024516e2f6ee30ca33e Reviewed-on: https://chromium-review.googlesource.com/924155 Commit-Queue: Olli Etuaho <oetuaho@nvidia.com> Reviewed-by: Jamie Madill <jmadill@chromium.org>
Luc Ferron 0aa1ffe3 2018-02-08T13:42:36 Vulkan: Autogen mandatory texture caps * This commit includes a JS file to execute on the spec and generate the JSON output of all the mandatory texture caps. Bug: angleproject:2348 Change-Id: I57e969915bdd0e7104e00a73fd3743ff1ecf0a6d Reviewed-on: https://chromium-review.googlesource.com/911615 Commit-Queue: Luc Ferron <lucferron@chromium.org> Reviewed-by: Jamie Madill <jmadill@chromium.org>
Frank Henigman dda048cd 2018-01-11T20:09:09 Make scripts executable by python2. Add #!/usr/bin/python2 and the executable permission bit to all scripts where missing. BUG=angleproject:2209 Change-Id: Ib33017c17e579c371b89bbfbdb7136b870027dc5 Reviewed-on: https://chromium-review.googlesource.com/862987 Reviewed-by: Corentin Wallez <cwallez@chromium.org> Commit-Queue: Frank Henigman <fjhenigman@chromium.org>
Jamie Madill 5ad52994 2017-11-14T12:43:40 Add generator for EGL proc table. This should improve ANGLE startup time by avoiding creating a large std::map filled with the entry points. BUG=chromium:781460 Change-Id: I20cfdb10b99844d0f60759dda73b729991dc60fe Reviewed-on: https://chromium-review.googlesource.com/768209 Reviewed-by: Geoff Lang <geofflang@chromium.org> Reviewed-by: Antoine Labour <piman@chromium.org> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Jamie Madill a8b73ed0 2017-11-02T09:22:29 Add a meta-script to run code generators. This script calls all the various GL and back-end python scripts to generate our internal format tables and entry points, etc. It uses a GYP-like scheme of inputs/outputs to check modified time before running the generators. It also will automatically call 'git cl format' if any generator was called. Also updates the copyright in a couple of touched files. BUG=angleproject:2207 Change-Id: I4187a7622accc1c97a8d779b8f87fe00b74855ea Reviewed-on: https://chromium-review.googlesource.com/742372 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Geoff Lang <geofflang@chromium.org>