run_code_generation.py

Branch :

Show log
Commit
Author : Olli Etuaho
Date : 2018-03-21 17:30:50
Hash : 89398b65
Message : Avoid mangled name comparisons of 3-parameter functions The hash values used for looking up built-ins now encode whether the mangled name contains arrays, structs or interface blocks in its parameters list. This is written in the most significant bit of the hash value. We check this bit at the start of built-in lookup and if the bit is set we exit early. After that we know that the lookup name doesn't contain array, struct or interface block parameters. When we find a hash that matches a hash of a built-in function, we now know 3 things: 1) the length of the mangled name matches 2) the open parentheses in the mangled name matches 3) the lookup doesn't contain array, struct or block parameters. Additionally, we have an if statement checking whether the function name matches. Collisions are only possible with functions that 1) have the same name 2) have the same number of parameters With these preconditions we can check beforehand whether collisions are possible for 3-parameter functions. If there are no collisions, we don't need to compare the full mangled name. This is similar to what was already being done with functions that had 0 to 2 parameters. This reduces shader_translator binary size by around 4 KB on Windows. Besides increased complexity, the tradeoff is that an exhaustive search of hash values for possible 3-parameter combinations is costly, so the gen_builtin_functions.py code generation script now takes around one minute to run on a high-end workstation. Due to this, the script now exits early if it detects it has already been run with the same inputs based on a hash value stored in builtin_symbols_hash_autogen.txt. BUG=angleproject:2267 BUG=chromium:823856 TEST=angle_unittests Change-Id: I3ff8c6eb85b90d3c4971ac8d73ee171a07a7e55f Reviewed-on: https://chromium-review.googlesource.com/973372 Reviewed-by: Corentin Wallez <cwallez@chromium.org> Commit-Queue: Olli Etuaho <oetuaho@nvidia.com>

scripts/run_code_generation.py

#!/usr/bin/python2
#
# Copyright 2017 The ANGLE Project Authors. All rights reserved.
# Use of this source code is governed by a BSD-style license that can be
# found in the LICENSE file.
#
# run_code_reneration.py:
#   Runs  ANGLE format table and other script run_code_renerationgeneration.

import os, subprocess, sys

# TODO(jmadill): Might be nice to have a standard way for scripts to return
# their inputs and outputs rather than listing them here.
generators = {
    'ANGLE format': {
        'inputs': [
            'src/libANGLE/renderer/angle_format.py',
            'src/libANGLE/renderer/angle_format_data.json',
            'src/libANGLE/renderer/angle_format_map.json',
        ],
        'outputs': [
            'src/libANGLE/renderer/Format_table_autogen.cpp',
            'src/libANGLE/renderer/Format_ID_autogen.inl',
        ],
        'script': 'src/libANGLE/renderer/gen_angle_format_table.py',
    },
    'ANGLE load functions table': {
        'inputs': [
            'src/libANGLE/renderer/load_functions_data.json',
        ],
        'outputs': [
            'src/libANGLE/renderer/load_functions_table_autogen.cpp',
        ],
        'script': 'src/libANGLE/renderer/gen_load_functions_table.py',
    },
    'D3D11 format': {
        'inputs': [
            'src/libANGLE/renderer/angle_format.py',
            'src/libANGLE/renderer/d3d/d3d11/texture_format_data.json',
            'src/libANGLE/renderer/d3d/d3d11/texture_format_map.json',
        ],
        'outputs': [
            'src/libANGLE/renderer/d3d/d3d11/texture_format_table_autogen.cpp',
        ],
        'script': 'src/libANGLE/renderer/d3d/d3d11/gen_texture_format_table.py',
    },
    'DXGI format': {
        'inputs': [
            'src/libANGLE/renderer/angle_format.py',
            'src/libANGLE/renderer/angle_format_map.json',
            'src/libANGLE/renderer/d3d/d3d11/dxgi_format_data.json',
        ],
        'outputs': [
            'src/libANGLE/renderer/d3d/d3d11/dxgi_format_map_autogen.cpp',
        ],
        'script': 'src/libANGLE/renderer/d3d/d3d11/gen_dxgi_format_table.py',
    },
    'DXGI format support': {
        'inputs': [
            'src/libANGLE/renderer/d3d/d3d11/dxgi_support_data.json',
        ],
        'outputs': [
            'src/libANGLE/renderer/d3d/d3d11/dxgi_support_table.cpp',
        ],
        'script': 'src/libANGLE/renderer/d3d/d3d11/gen_dxgi_support_tables.py',
    },
    'GL copy conversion table': {
        'inputs': [
            'src/libANGLE/es3_copy_conversion_formats.json',
        ],
        'outputs': [
            'src/libANGLE/es3_copy_conversion_table_autogen.cpp',
        ],
        'script': 'src/libANGLE/gen_copy_conversion_table.py',
    },
    'GL entry point': {
        'inputs': [
            'scripts/entry_point_packed_gl_enums.json',
            'scripts/gl.xml',
        ],
        'outputs': [
            'src/libGLESv2/entry_points_gles_2_0_autogen.cpp',
            'src/libGLESv2/entry_points_gles_2_0_autogen.h',
            'src/libGLESv2/entry_points_gles_3_0_autogen.cpp',
            'src/libGLESv2/entry_points_gles_3_0_autogen.h',
        ],
        'script': 'scripts/generate_entry_points.py',
    },
    'GL format map': {
        'inputs': [
            'src/libANGLE/es3_format_type_combinations.json',
            'src/libANGLE/format_map_data.json',
        ],
        'outputs': [
            'src/libANGLE/format_map_autogen.cpp',
        ],
        'script': 'src/libANGLE/gen_format_map.py',
    },
    'uniform type': {
        'inputs': [],
        'outputs': [
            'src/common/uniform_type_info_autogen.cpp',
        ],
        'script': 'src/common/gen_uniform_type_table.py',
    },
    'OpenGL dispatch table': {
        'inputs': [
            'scripts/gl.xml',
        ],
        'outputs': [
            'src/libANGLE/renderer/gl/DispatchTableGL_autogen.cpp',
            'src/libANGLE/renderer/gl/DispatchTableGL_autogen.h',
            'src/libANGLE/renderer/gl/null_functions.h',
            'src/libANGLE/renderer/gl/null_functions.cpp',
        ],
        'script': 'src/libANGLE/renderer/gl/generate_gl_dispatch_table.py',
    },
    'packed GLenum': {
        'inputs': [
            'src/libANGLE/packed_gl_enums.json',
        ],
        'outputs': [
            'src/libANGLE/PackedGLEnums_autogen.cpp',
            'src/libANGLE/PackedGLEnums_autogen.h',
        ],
        'script': 'src/libANGLE/gen_packed_gl_enums.py',
    },
    'proc table': {
        'inputs': [
            'src/libGLESv2/proc_table_data.json',
        ],
        'outputs': [
            'src/libGLESv2/proc_table_autogen.cpp',
        ],
        'script': 'src/libGLESv2/gen_proc_table.py',
    },
    'Vulkan format': {
        'inputs': [
            'src/libANGLE/renderer/angle_format.py',
            'src/libANGLE/renderer/angle_format_map.json',
            'src/libANGLE/renderer/vulkan/vk_format_map.json',
        ],
        'outputs': [
            'src/libANGLE/renderer/vulkan/vk_format_table_autogen.cpp',
        ],
        'script': 'src/libANGLE/renderer/vulkan/gen_vk_format_table.py',
    },
    'Vulkan mandatory format support table': {
        'inputs': [
            'src/libANGLE/renderer/angle_format.py',
            'third_party/vulkan-validation-layers/src/scripts/vk.xml',
            'src/libANGLE/renderer/vulkan/vk_mandatory_format_support_data.json',
        ],
        'outputs': [
            'src/libANGLE/renderer/vulkan/vk_mandatory_format_support_table_autogen.cpp',
        ],
        'script': 'src/libANGLE/renderer/vulkan/gen_vk_mandatory_format_support_table.py',
    },
    'ESSL static builtins': {
        'inputs': [
            'src/compiler/translator/builtin_function_declarations.txt',
            'src/compiler/translator/builtin_variables.json',
        ],
        'outputs': [
            'src/compiler/translator/tree_util/BuiltIn_autogen.h',
            'src/compiler/translator/builtin_symbols_hash_autogen.txt',
            'src/compiler/translator/ParseContext_autogen.h',
            'src/compiler/translator/SymbolTable_autogen.cpp',
            'src/compiler/translator/SymbolTable_autogen.h',
            'src/tests/compiler_tests/ImmutableString_test_autogen.cpp',
        ],
        'script': 'src/compiler/translator/gen_builtin_symbols.py',
    },
}

root_dir = os.path.abspath(os.path.join(os.path.dirname(os.path.abspath(__file__)), '..'))
any_dirty = False

for name, info in sorted(generators.iteritems()):

    # Set the CWD to the root ANGLE directory.
    os.chdir(root_dir)

    script = info['script']
    dirty = False

    for finput in info['inputs'] + [script]:
        input_mtime = os.path.getmtime(finput)
        for foutput in info['outputs']:
            if not os.path.exists(foutput):
                print('Output ' + foutput + ' not found for ' + name + ' table')
                dirty = True
            else:
                output_mtime = os.path.getmtime(foutput)
                if input_mtime > output_mtime:
                    dirty = True

    if dirty:
        any_dirty = True

        # Set the CWD to the script directory.
        os.chdir(os.path.dirname(os.path.abspath(script)))

        print('Running ' + name + ' code generator')
        if subprocess.call(['python', os.path.basename(script)]) != 0:
            sys.exit(1)

if any_dirty:
    args = []
    if os.name == 'nt':
        args += ['git.bat']
    else:
        args += ['git']
    args += ['cl', 'format']
    print('Calling git cl format')
    subprocess.call(args)

kc3-lang/angle/scripts/run_code_generation.py

Commit

kc3-lang/angle /scripts/run_code_generation.py