gen_wgpu_format_table.py

Show log
Commit

Hash : 78ee33e4
Author :
Date : 2024-07-01T15:42:55

Roll third_party/dawn/ 6cdf3a1a1..eb249b37e (310 commits; 3 trivial rolls)

We need to roll in
https://dawn-review.googlesource.com/c/dawn/+/196054
to ublock rolling libc++ into ANGLE.

Add special handling of Undefined WGPU formats due to also rolling
https://dawn-review.googlesource.com/c/dawn/+/193360

https://dawn.googlesource.com/dawn.git/+log/6cdf3a1a195f..eb249b37e3e8

$ git log 6cdf3a1a1..eb249b37e --date=short --no-merges --format='%ad %ae %s'
2024-07-01 blundell [Vulkan] Set the Vk descriptor type correctly for YCbCr samplers
2024-07-01 blundell [Vulkan] Map FilterMode to Vulkan filter correctly for YCbCr info
2024-07-01 dawn-autoroll Roll vulkan-deps from 773555d3bd22 to 4a2ff9e87700 (1 revision)
2024-06-30 dsinclair [hlsl] Handle compute builtins for ShaderIO transform.
2024-06-30 dsinclair [hlsl] Add some existing transforms
2024-06-29 jrprice [msl] Regenerate test expectations
2024-06-29 dsinclair [hlsl] Split module var structs and arrays.
2024-06-29 dawn-autoroll Roll vulkan-deps from 2796aaff7eef to 773555d3bd22 (9 revisions)
2024-06-29 cwallez opengl: Add support for EGL_EXT_pixel_format_float.
2024-06-29 kainino Summarize CTS failures as first line of first max-severity log
2024-06-29 jaswant.panchumarti Do not turn off CMAKE_CXX_EXTENSIONS
2024-06-28 rafael.cintron Disable render passes on Windows until support can be detected
2024-06-28 enga [headers] Remove the Flags enums and make them 64-bit
2024-06-28 dsinclair [hlsl] Stub out the DecomposeMemoryAccess IR transform.
2024-06-28 cwallez opengl: Add support for EGL_KHR_no_config_context.
2024-06-28 jaswant.panchumarti Disable GLFW_INSTALL
2024-06-28 jaswant.panchumarti Add third_party/google_benchmark/src in fetch_dawn_dependencies.py
2024-06-28 jaswant.panchumarti Always build glslang as static library
2024-06-28 dawn-autoroll Roll vulkan-deps from 7ff358e64e2b to 2796aaff7eef (10 revisions)
2024-06-28 enga Fix Wayland build
2024-06-28 dawn-autoroll Roll ANGLE from 46dd6457f4bf to a674dc1dae8f (3 revisions)
2024-06-28 dawn-autoroll Roll DirectX Shader Compiler from 206133c9e7e1 to 49d1ae9a029c (1 revision)
2024-06-27 bsheedy Mark problematic expectation block as KEEP
2024-06-27 rharrison [tint][fuzz][ir] Implement test case disassembler
2024-06-27 dsinclair [hlsl] Add transform to remove struct and array initializers to lets.
2024-06-27 enga Re-number enum values part 2
2024-06-27 rharrison [tint][fuzz][ir] Batch process files when generating corpus
2024-06-27 cwallez GL: Add support for wgpu::Surface.
2024-06-27 jrprice [tint] Fix UB in FileContent copy constructor
2024-06-27 dsinclair [hlsl] Add ValueToLet transform.
2024-06-27 dsinclair [hlsl] Add ShaderIO transform.
2024-06-27 jaswant.panchumarti Use dawn_add_library function to create the dawn_glfw library
2024-06-27 jaswant.panchumarti Use dawn_add_library function to create the dawn_utils library
2024-06-27 jaswant.panchumarti Use dawn_add_library function to create the dawn_wire library
2024-06-27 jaswant.panchumarti Use dawn_add_library function to create the dawn_native library
2024-06-27 dsinclair [glsl] Extract textureBuiltinFromUniformData
2024-06-27 dawn-autoroll Roll vulkan-deps from 1c484942880c to 7ff358e64e2b (4 revisions)
2024-06-27 hans Fix std::vector<const T>
2024-06-27 dawn-autoroll Roll ANGLE from 0bf157f12bb8 to 46dd6457f4bf (5 revisions)
2024-06-27 dsinclair [glsl] Extract combinedSamplerInfo
2024-06-27 jrprice [fuzz] Skip PixelLocal with multiple entry points
2024-06-27 jrprice [tint] Pre-clone symbols in IR->AST pass
2024-06-27 jrprice [fuzz] Add TextureBuiltinsFromUniform precondition
2024-06-27 jrprice [tint] Fix collision check in multiplanar fuzzer
2024-06-27 jaswant.panchumarti Use dawn_add_library function to create the dawn_platform library
2024-06-27 jiawei.shao Fix crash when using constant expression as `@blend_src`
2024-06-26 rharrison [tint][ir][fuzz] Convert assembler to generating test case protos
2024-06-26 jaswant.panchumarti Use dawn_add_library function to create the dawn_proc library
2024-06-26 dawn-autoroll Roll ANGLE from 046327343595 to 0bf157f12bb8 (5 revisions)
2024-06-26 dawn-autoroll Roll DirectX Shader Compiler from e5183a06b913 to 206133c9e7e1 (1 revision)
(...)
2024-06-17 dawn-autoroll Roll SwiftShader from 085997ddb08b to cea33ab2d5ad (1 revision)
2024-06-17 jimblackler Integration test in the library.
2024-06-17 dawn-autoroll Roll vulkan-deps from 897d900fc1b1 to 7f05a36fb6d9 (1 revision)
2024-06-17 dawn-autoroll Roll DirectX Shader Compiler from 206b7c2e537a to 8c3f40c0ae12 (1 revision)
2024-06-17 dawn-autoroll Roll ANGLE from d521dd08bef4 to 7f70cd0da27c (1 revision)
2024-06-16 dawn-autoroll Roll vulkan-deps from 1bd6201121db to 897d900fc1b1 (1 revision)
2024-06-15 jimblackler Alternate way of generating the bindings inside Gradle.
2024-06-15 jimblackler Fix for incorrect import path for webgpu.h.
2024-06-15 jimblackler Move 'Util' to a new location, and convert to Kotlin.
2024-06-15 dawn-autoroll Roll vulkan-deps from dd296c842e50 to 1bd6201121db (7 revisions)
2024-06-15 dawn-autoroll Roll ANGLE from bcb567a7f8e4 to d521dd08bef4 (7 revisions)
2024-06-14 hitawala [YCbCr Samplers] Get supported SampleTypeBit for external format
2024-06-14 senorblanco OpenGLES: more end2end test cleanup.
2024-06-14 enga Fix wgpu::Status return value for wgpuSurfaceGetCapabilities
2024-06-14 kainino Use ityp::array for sVertexFormatTable
2024-06-14 dawn-autoroll Roll vulkan-deps from 42ee85cf2154 to dd296c842e50 (2 revisions)
2024-06-14 dawn-autoroll Roll Depot Tools from 88a72320ed30 to 1d1f17af898b (4 revisions)
2024-06-14 dawn-autoroll Roll DirectX Shader Compiler from 56f3c4038159 to 206b7c2e537a (1 revision)
2024-06-14 enga Remove 0-value enums that aren't ever valid
2024-06-14 dawn-autoroll Roll ANGLE from 973e5261a0fd to bcb567a7f8e4 (7 revisions)
2024-06-14 senorblanco Revert "Make a convenience cpp method for WaitAny with one future"
2024-06-14 dawn-autoroll Roll vulkan-deps from 7db4057407f4 to 42ee85cf2154 (8 revisions)
2024-06-14 enga Make a convenience cpp method for WaitAny with one future
2024-06-14 dawn-autoroll Roll Depot Tools from 43c6415bced9 to 88a72320ed30 (3 revisions)
2024-06-14 beaufort.francois Make GPUAdapter GetInfo() a suitable replacement for GetProperties
2024-06-14 rharrison [tint][ir][fuzz] Disable building IR fuzzer when ASAN is enabled
2024-06-14 enga AHB: Don't add storage binding unless AHARDWAREBUFFER_USAGE_GPU_DATA_BUFFER
2024-06-14 dsinclair [hlsl] Fix incorrect HLSL test output
2024-06-14 rharrison [tint][ir][fuzz] Fix breakage of fuzz_cmd target for CMake
2024-06-13 dsinclair [hlsl] Add HLSL IR printer var/let tests
2024-06-13 dsinclair [ir] Add a templated `zero` helper.
2024-06-13 dawn-autoroll Roll ANGLE from 65f8dc8e3ad8 to 973e5261a0fd (5 revisions)
2024-06-13 dsinclair [hlsl] Add generated IR expectations.
2024-06-13 jrprice [msl] Regenerate E2E expectations
2024-06-13 lehoangquyen D3D11: split Buffer class into GPUOnlyBuffer and StagingBuffer
2024-06-13 senorblanco OpenGLES: enable/triage misc end2end tests.
2024-06-13 chouinard Remove storage textures from vertex tests
2024-06-13 chouinard Disallow textureStore in vertex shader tests
2024-06-13 chouinard Replace textureStore test with frag shader
2024-06-13 jrprice [msl] Add polyfill for textureNumSamples
2024-06-13 jrprice [msl] Add polyfill for textureNumLayers
2024-06-13 jrprice [msl] Add polyfill for textureSampleCompare
2024-06-13 jrprice [ir] Strip all IO attributes in ShaderIO transform
2024-06-13 fdurso SampleUtils: Add missing newline in output message
2024-06-13 wong.kl.matthew dawn/node: Update documentation for ASan CMake builds.
2024-06-13 dawn-autoroll Roll DirectX Shader Compiler from 4353db3983e2 to 56f3c4038159 (2 revisions)
2024-06-13 ynovikov Revert "d3d: Change D3D references to PhysicalDevice to WeakRef instead of Ref"
2024-06-13 dsinclair [hlsl] Add HLSL IR printer constant tests
2024-06-13 dawn-autoroll Roll vulkan-deps from 7d45b6d174eb to 7db4057407f4 (13 revisions)
2024-06-13 dawn-autoroll Roll Depot Tools from c0883c509ea3 to 43c6415bced9 (3 revisions)

Created with:
  roll-dep third_party/dawn

Bug: chromium:323708866, chromium:349411322, dawn:42241166
Change-Id: Iad65ef83c0ec47a6aba3fdc7de511ff22f586d4d
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/5667168
Auto-Submit: Yuly Novikov <ynovikov@chromium.org>
Reviewed-by: Geoff Lang <geofflang@chromium.org>
Commit-Queue: Geoff Lang <geofflang@chromium.org>

Download

src/libANGLE/renderer/wgpu/gen_wgpu_format_table.py

#!/usr/bin/python3
# Copyright 2024 The ANGLE Project Authors. All rights reserved.
# Use of this source code is governed by a BSD-style license that can be
# found in the LICENSE file.
#
# gen_wgpu_format_table.py:
#  Code generation for wgpu format map. See wgpu_format_map.json for data source.
#  NOTE: don't run this script directly. Run scripts/run_code_generation.py.

import json
import math
import pprint
import os
import re
import sys

sys.path.append('..')
import angle_format

template_table_autogen_cpp = """// GENERATED FILE - DO NOT EDIT.
// Generated by {script_name} using data from {input_file_name}
//
// Copyright 2024 The ANGLE Project Authors. All rights reserved.
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.
//
// {out_file_name}:
//   Queries for full WebGPU format information based on GL format.

#include "libANGLE/renderer/wgpu/wgpu_format_utils.h"

#include "image_util/loadimage.h"

using namespace angle;

namespace rx
{{
namespace webgpu
{{

void Format::initialize(const angle::Format &angleFormat)
{{
    switch (angleFormat.id)
    {{
{format_case_data}
        default:
            UNREACHABLE();
            break;
    }}
}}

wgpu::TextureFormat GetWgpuTextureFormatFromFormatID(angle::FormatID formatID)
{{
    static constexpr angle::FormatMap<wgpu::TextureFormat> kMap = {{
{image_format_id_cases}
    }};

    return kMap[formatID];
}}

angle::FormatID GetFormatIDFromWgpuTextureFormat(wgpu::TextureFormat wgpuFormat)
{{
    switch (wgpuFormat)
    {{
{wgpu_image_format_cases}
        default:
            UNREACHABLE();
            return angle::FormatID::NONE;
    }}
}}

wgpu::VertexFormat GetWgpuVertexFormatFromFormatID(angle::FormatID formatID)
{{
    static constexpr angle::FormatMap<wgpu::VertexFormat> kMap = {{
{buffer_format_id_cases}
    }};

    return kMap[formatID];
}}

angle::FormatID GetFormatIDFromWgpuBufferFormat(wgpu::VertexFormat wgpuFormat)
{{
    switch (wgpuFormat)
    {{
{wgpu_buffer_format_cases}
        default:
            UNREACHABLE();
            return angle::FormatID::NONE;
    }}
}}
}}  // namespace webgpu
}}  // namespace rx
"""

empty_format_entry_template = """case angle::FormatID::{format_id}:
// This format is not implemented in WebGPU.
break;
"""

format_entry_template = """case angle::FormatID::{format_id}:
mIntendedGLFormat = {internal_format};
{image_template}
{buffer_template}
break;
"""

image_snorm_template = """mActualImageFormatID = {image};
mImageInitializerFunction = {image_initializer};
mIsRenderable = false;"""

image_basic_template = """mActualImageFormatID = {image};
mImageInitializerFunction = {image_initializer};
mIsRenderable = true;"""

image_struct_template = "{{{image}, {image_initializer}}}"

image_fallback_template = """{{
static constexpr ImageFormatInitInfo kInfo[] = {{{image_list}}};
initImageFallback(kInfo, ArraySize(kInfo));
}}"""

buffer_basic_template = """mActualBufferFormatID = {buffer};
mVertexLoadFunction = {vertex_load_function};
mVertexLoadRequiresConversion = {vertex_load_converts};"""

buffer_struct_template = """{{{buffer}, {vertex_load_function}, {vertex_load_converts}}}"""

buffer_fallback_template = """{{
static constexpr BufferFormatInitInfo kInfo[] = {{{buffer_list}}};
initBufferFallback(kInfo, ArraySize(kInfo));
}}"""


def verify_wgpu_image_map_keys(angle_to_gl, wgpu_json_data):
    """Verify that the keys in WebGPU format tables exist in the ANGLE table.  If they don't, the
    entry in the WebGPU file is incorrect and needs to be fixed."""

    no_error = True
    for table in ["image_map", "buffer_map", "fallbacks"]:
        for angle_format in wgpu_json_data[table].keys():
            if not angle_format in angle_to_gl.keys():
                print("Invalid format " + angle_format + " in wgpu_format_map.json in " + table)
                no_error = False

    return no_error


def get_vertex_copy_function(src_format, dst_format, wgpu_format):
    if 'R10G10B10A2' in src_format:
        # When the R10G10B10A2 type can't be used by the vertex buffer,
        # it needs to be converted to the type which can be used by it.
        is_signed = 'false' if 'UINT' in src_format or 'UNORM' in src_format or 'USCALED' in src_format else 'true'
        normalized = 'true' if 'NORM' in src_format else 'false'
        to_float = 'false' if 'INT' in src_format else 'true'
        to_half = to_float
        return 'CopyXYZ10W2ToXYZWFloatVertexData<%s, %s, %s, %s>' % (is_signed, normalized,
                                                                     to_float, to_half)
    return angle_format.get_vertex_copy_function(src_format, dst_format)


def gen_format_case(angle, internal_format, wgpu_json_data):
    wgpu_image_map = wgpu_json_data["image_map"]
    wgpu_buffer_map = wgpu_json_data["buffer_map"]
    wgpu_fallbacks = wgpu_json_data["fallbacks"]
    args = dict(
        format_id=angle, internal_format=internal_format, image_template="", buffer_template="")

    if ((angle not in wgpu_image_map) and (angle not in wgpu_buffer_map) and
        (angle not in wgpu_fallbacks)):
        return empty_format_entry_template.format(**args)

    # get_formats returns override format (if any) + fallbacks
    # this was necessary to support D32_UNORM. There is no appropriate override that allows
    # us to fallback to D32_FLOAT, so now we leave the image override empty and function will
    # give us the fallbacks.
    def get_formats(format, type):
        fallbacks = wgpu_fallbacks.get(format, {}).get(type, [])
        if not isinstance(fallbacks, list):
            fallbacks = [fallbacks]

        if (format in wgpu_image_map and type == "image") or (format in wgpu_buffer_map and
                                                              type == "buffer"):
            assert format not in fallbacks
            fallbacks = [format] + fallbacks

        return fallbacks

    def image_args(format):
        return dict(
            image="angle::FormatID::" + format,
            image_initializer=angle_format.get_internal_format_initializer(
                internal_format, format))

    def buffer_args(format):
        wgpu_buffer_format = wgpu_buffer_map[format]
        return dict(
            buffer="angle::FormatID::" + format,
            vertex_load_function=get_vertex_copy_function(angle, format, wgpu_buffer_format),
            vertex_load_converts='false' if angle == format else 'true',
        )

    images = get_formats(angle, "image")
    if len(images) == 1:
        if 'SNORM' in angle:
            args.update(image_template=image_snorm_template)
        else:
            args.update(image_template=image_basic_template)
        args.update(image_args(images[0]))
    elif len(images) > 1:
        args.update(
            image_template=image_fallback_template,
            image_list=", ".join(image_struct_template.format(**image_args(i)) for i in images))

    buffers = get_formats(angle, "buffer")
    if len(buffers) == 1 and buffers[0] in wgpu_buffer_map:
        args.update(buffer_template=buffer_basic_template)
        args.update(buffer_args(buffers[0]))
    elif len(buffers) > 1:
        args.update(
            buffer_template=buffer_fallback_template,
            buffer_list=", ".join(
                buffer_struct_template.format(**buffer_args(i)) for i in buffers))

    return format_entry_template.format(**args).format(**args)


def get_format_id_case(format_id, format_type, wgpu_format):
    # wgpu::VertexFormat::Undefined was replaced with wgpu::VertexFormat(0u)
    # in https://dawn-review.googlesource.com/c/dawn/+/193360
    if 'Undefined' in wgpu_format and 'VertexFormat' in format_type:
        return "{angle::FormatID::%s, wgpu::%s(0u)}" % (format_id, format_type)
    return "{angle::FormatID::%s, wgpu::%s::%s}" % (format_id, format_type, wgpu_format)


def get_wgpu_format_case(format_type, format_id, wgpu_format):
    # wgpu::VertexFormat::Undefined was replaced with wgpu::VertexFormat(0u)
    # in https://dawn-review.googlesource.com/c/dawn/+/193360
    # so there is no 'case' needed for it.
    if 'Undefined' in wgpu_format and 'VertexFormat' in format_type:
        return ''
    # don't generate the reverse mapping for the external format slots because they _all_ map
    # to WGPU_FORMAT_UNDEFINED and so clash with NONE
    if 'EXTERNAL' in format_id:
        return ''
    return """\
        case wgpu::%s::%s:
            return angle::FormatID::%s;
""" % (format_type, wgpu_format, format_id)


def main():

    input_file_name = 'wgpu_format_map.json'
    out_file_name = 'wgpu_format_table_autogen.cpp'

    # auto_script parameters.
    if len(sys.argv) > 1:
        inputs = ['../angle_format.py', '../angle_format_map.json', input_file_name]
        outputs = [out_file_name]

        if sys.argv[1] == 'inputs':
            print(','.join(inputs))
        elif sys.argv[1] == 'outputs':
            print(','.join(outputs))
        else:
            print('Invalid script parameters')
            return 1
        return 0

    angle_to_gl = angle_format.load_inverse_table(os.path.join('..', 'angle_format_map.json'))
    wgpu_json_data = angle_format.load_json(input_file_name)

    if not verify_wgpu_image_map_keys(angle_to_gl, wgpu_json_data):
        return 1

    image_format_id_cases = [
        get_format_id_case(format_id, "TextureFormat", wgpu_format)
        for format_id, wgpu_format in sorted(wgpu_json_data["image_map"].items())
    ]

    wgpu_image_format_cases = [
        get_wgpu_format_case("TextureFormat", format_id, wgpu_format)
        for format_id, wgpu_format in sorted(wgpu_json_data["image_map"].items())
    ]

    buffer_format_id_cases = [
        get_format_id_case(format_id, "VertexFormat", wgpu_format)
        for format_id, wgpu_format in sorted(wgpu_json_data["buffer_map"].items())
    ]

    wgpu_buffer_format_cases = [
        get_wgpu_format_case("VertexFormat", format_id, wgpu_format)
        for format_id, wgpu_format in sorted(wgpu_json_data["buffer_map"].items())
    ]

    wgpu_cases = [
        gen_format_case(angle, gl, wgpu_json_data) for angle, gl in sorted(angle_to_gl.items())
    ]

    output_cpp = template_table_autogen_cpp.format(
        format_case_data="\n".join(wgpu_cases),
        image_format_id_cases=",\n".join(image_format_id_cases),
        wgpu_image_format_cases="".join(wgpu_image_format_cases),
        buffer_format_id_cases=",\n".join(buffer_format_id_cases),
        wgpu_buffer_format_cases="".join(wgpu_buffer_format_cases),
        script_name=os.path.basename(__file__),
        out_file_name=out_file_name,
        input_file_name=input_file_name)

    with open(out_file_name, 'wt') as out_file:
        out_file.write(output_cpp)
        out_file.close()
    return 0


if __name__ == '__main__':
    sys.exit(main())

Download