generate_test_spec_json.py

Show log
Commit

Hash : 55279449
Author :
Date : 2023-07-03T19:42:52

Roll chromium_revision 2e0371f07e..ad19957265 (1162850:1165395)

Don't update testing beyond
https://chromium.googlesource.com/chromium/src/testing/+/8e38c8e8a6b4251a2f9584d9d9bfbf261d0c1835

Remove $mixin_append usage due to
https://chromium.googlesource.com/chromium/src/testing/+/a5415f3c674a065cc541c618df12262073d4d10e

Change log: https://chromium.googlesource.com/chromium/src/+log/2e0371f07e..ad19957265
Full diff: https://chromium.googlesource.com/chromium/src/+/2e0371f07e..ad19957265

Changed dependencies
* build: https://chromium.googlesource.com/chromium/src/build.git/+log/e4262406c0..e0690c7c78
* buildtools: https://chromium.googlesource.com/chromium/src/buildtools.git/+log/f089c59d7c..6d00f8d8c5
* buildtools/third_party/libc++/trunk: https://chromium.googlesource.com/external/github.com/llvm/llvm-project/libcxx.git/+log/b272a1c128..84fb809dd6
* buildtools/third_party/libunwind/trunk: https://chromium.googlesource.com/external/github.com/llvm/llvm-project/libunwind.git/+log/b5a43ecdac..f1c687e0aa
* testing: https://chromium.googlesource.com/chromium/src/testing/+log/46c0754180..ac3cb62a5a
* third_party/abseil-cpp: https://chromium.googlesource.com/chromium/src/third_party/abseil-cpp/+log/66a5ea107e..7d4c4bf473
* third_party/android_build_tools/lint: kqQgSDAgfuKTUIT8gpYlhM9gvajbtIDs_ApDbh3wdNIC..DjoXGsdMeTwnppye9Nwe5ozhXIqKyJQDzgb0nTFH8SAC
* third_party/android_build_tools/manifest_merger: pRHDE8UAgipcDQINCUsRz94lgm5BHQjiL-BLF6d6xC4C..MN3CF2GQ8xeB6obj4qf5J6l15-NoA43u4__RQTTe8I4C
* third_party/android_toolchain: version:2@r25c.cr1..R_8suM8m0oHbZ1awdxGXvKEFpAOETscbfZxkkMthyk8C
* third_party/catapult: https://chromium.googlesource.com/catapult.git/+log/{catapult_..48b31edbfc
* third_party/depot_tools: https://chromium.googlesource.com/chromium/tools/depot_tools.git/+log/f0fba1d307..872ac9bd3d
* third_party/fuchsia-sdk/sdk: version:13.20230626.3.1..version:13.20230629.2.1
* third_party/turbine: G8Cku1fztaDd9to_8gk3PNWM2iRacAxD9zcUAgUPUEAC..9rSngS4o9OIlNubLcU1zXF_w0dPYif6AS-_k1pPAV1kC
* third_party/zlib: https://chromium.googlesource.com/chromium/src/third_party/zlib/+log/f81f385906..f999826d2d
* tools/android: https://chromium.googlesource.com/chromium/src/tools/android/+log/5c2008090e..39d4cc4ec8
* tools/clang: https://chromium.googlesource.com/chromium/src/tools/clang.git/+log/053092c268..36a80749cf
* tools/luci-go: git_revision:39f255d5875293d3e1d978888b819ac124a8b0cc..git_revision:15be1e7f953cb4ef6d88aba86f57ca28724d1c9a
* tools/luci-go: git_revision:39f255d5875293d3e1d978888b819ac124a8b0cc..git_revision:15be1e7f953cb4ef6d88aba86f57ca28724d1c9a
* tools/mb: https://chromium.googlesource.com/chromium/src/tools/mb/+log/f80577ef22..33899440a1
* tools/perf: https://chromium.googlesource.com/chromium/src/tools/perf/+log/daed2f7981..422351564c
* tools/skia_goldctl/linux: tVDltUu_xw7_tk4k-rnp7Ua8pKhf3EEHmTwX5c6BS4AC..209WoWuXITgke_BJDnvM7aTy7wu2jFOUPYs4uIoJ3CAC
* tools/skia_goldctl/mac_amd64: ov_DuOSHCneWQSTW6AZ9gWHEeoJxXEYDX9dl_0rA_d0C..vEHTSSZiGWhNdCjMGZ22sj4PfyPzgoIIkV97YXOJeKEC
* tools/skia_goldctl/mac_arm64: 3wDkZAVD4vV1dNhXfRNMOUBvODbzJ23AVmmsFbTsbkQC..CcHM6-LZ7_b-Api718RTCYQPD0v17lWKuQBaz42b8r4C
* tools/skia_goldctl/win: yW7os_HVMoE-wq04olb6B7GNgEmaYZYPN0ecTzPPBdkC..u5j7yik1xZdFc9dECFfFDX9vRP9go2GpqukfJwOLCbgC
DEPS diff: https://chromium.googlesource.com/chromium/src/+/2e0371f07e..ad19957265/DEPS

No update to Clang.

Bug: chromium:1456553
Change-Id: I1c18e6c6f68c7d24ae72a6754c707397a4c739d2
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4660052
Reviewed-by: Ian Elliott <ianelliott@google.com>
Commit-Queue: Yuly Novikov <ynovikov@chromium.org>

Download

infra/specs/generate_test_spec_json.py

#!/usr/bin/env vpython3
# Copyright 2021 The ANGLE Project Authors. All rights reserved.
# Use of this source code is governed by a BSD-style license that can be
# found in the LICENSE file.
"""Script to generate the test spec JSON files. Calls Chromium's generate_buildbot_json.

=== NOTE: DO NOT RUN THIS SCRIPT DIRECTLY. ===
Run scripts/run_code_generation.py instead to update necessary hashes.

"""

import os
import pprint
import sys
import subprocess
import tempfile

d = os.path.dirname
THIS_DIR = d(os.path.abspath(__file__))
TESTING_BBOT_DIR = os.path.join(d(d(THIS_DIR)), 'testing', 'buildbot')
sys.path.insert(0, TESTING_BBOT_DIR)

import generate_buildbot_json

# Add custom mixins here.
ADDITIONAL_MIXINS = {
    'angle_skia_gold_test': {
        'args': [
            '--git-revision=${got_angle_revision}',
            # BREAK GLASS IN CASE OF EMERGENCY
            # Uncommenting this argument will bypass all interactions with Skia
            # Gold in any tests that use it. This is meant as a temporary
            # emergency stop in case of a Gold outage that's affecting the bots.
            # '--bypass-skia-gold-functionality',
        ],
        'precommit_args': [
            '--gerrit-issue=${patch_issue}',
            '--gerrit-patchset=${patch_set}',
            '--buildbucket-id=${buildbucket_build_id}',
            # This normally evaluates to "0", but will evaluate to "1" if
            # "Use-Permissive-Angle-Pixel-Comparison: True" is present as a
            # CL footer.
            '--use-permissive-pixel-comparison=${use_permissive_angle_pixel_comparison}',
        ],
    },
    'timeout_120m': {
        'swarming': {
            'hard_timeout': 7200,
            'io_timeout': 300
        }
    },
}

MIXIN_FILE_NAME = os.path.join(THIS_DIR, 'mixins.pyl')
MIXINS_PYL_TEMPLATE = """\
# GENERATED FILE - DO NOT EDIT.
# Generated by {script_name} using data from {data_source}
#
# Copyright 2021 The ANGLE Project Authors. All rights reserved.
# Use of this source code is governed by a BSD-style license that can be
# found in the LICENSE file.
#
# This is a .pyl, or "Python Literal", file. You can treat it just like a
# .json file, with the following exceptions:
# * all keys must be quoted (use single quotes, please);
# * comments are allowed, using '#' syntax; and
# * trailing commas are allowed.
#
# For more info see Chromium's mixins.pyl in testing/buildbot.

{mixin_data}
"""


def main():
    if len(sys.argv) > 1:
        gen_bb_json = os.path.join(TESTING_BBOT_DIR, 'generate_buildbot_json.py')
        mixins_pyl = os.path.join(TESTING_BBOT_DIR, 'mixins.pyl')
        inputs = [
            'test_suite_exceptions.pyl', 'test_suites.pyl', 'variants.pyl', 'waterfalls.pyl',
            gen_bb_json, mixins_pyl
        ]
        outputs = ['angle.json', 'mixins.pyl']
        if sys.argv[1] == 'inputs':
            print(','.join(inputs))
            return 0
        if sys.argv[1] == 'outputs':
            print(','.join(outputs))
            return 0

    # --verify-only enables dirty checks without relying on checked in hashes.
    # Compares the content of the existing file with the generated content.
    verify_only = '--verify-only' in sys.argv

    if verify_only:
        with tempfile.TemporaryDirectory() as temp_dir:
            return run_generator(verify_only, temp_dir)
    else:
        return run_generator(verify_only, None)


def write_or_verify_file(filename, content, verify_only):
    if verify_only:
        try:
            with open(filename) as f:
                # Note: .gitattributes "* text=auto" handles LF <-> CRLF on Windows
                return f.read() == content
        except FileNotFoundError:
            return False
    else:
        with open(filename, 'w') as fout:
            fout.write(content)
            return True


def run_generator(verify_only, temp_dir):
    chromium_args = generate_buildbot_json.BBJSONGenerator.parse_args([])
    chromium_generator = generate_buildbot_json.BBJSONGenerator(chromium_args)
    chromium_generator.load_configuration_files()

    override_args = ['--pyl-files-dir', THIS_DIR]
    if verify_only:
        override_args += ['--output-dir', temp_dir]
    angle_args = generate_buildbot_json.BBJSONGenerator.parse_args(override_args)
    angle_generator = generate_buildbot_json.BBJSONGenerator(angle_args)
    angle_generator.load_configuration_files()
    angle_generator.resolve_configuration_files()

    seen_mixins = set()
    for waterfall in angle_generator.waterfalls:
        seen_mixins = seen_mixins.union(waterfall.get('mixins', set()))
        for bot_name, tester in waterfall['machines'].items():
            seen_mixins = seen_mixins.union(tester.get('mixins', set()))
    for suite in angle_generator.test_suites.values():
        if isinstance(suite, list):
            # Don't care about this, it's a composition, which shouldn't include a
            # swarming mixin.
            continue
        for test in suite.values():
            assert isinstance(test, dict)
            seen_mixins = seen_mixins.union(test.get('mixins', set()))

    found_mixins = ADDITIONAL_MIXINS.copy()
    for mixin in seen_mixins:
        if mixin in found_mixins:
            continue
        assert (mixin in chromium_generator.mixins), 'Error with %s mixin' % mixin
        found_mixins[mixin] = chromium_generator.mixins[mixin]

    pp = pprint.PrettyPrinter(indent=2)

    format_data = {
        'script_name': os.path.basename(__file__),
        'data_source': 'waterfall.pyl and Chromium\'s mixins.pyl',
        'mixin_data': pp.pformat(found_mixins),
    }
    generated_mixin_pyl = MIXINS_PYL_TEMPLATE.format(**format_data)

    if not write_or_verify_file(MIXIN_FILE_NAME, generated_mixin_pyl, verify_only):
        print('infra/specs/mixins.pyl dirty')
        return 1

    if angle_generator.main() != 0:
        print('buildbot (pyl to json) generation failed')
        return 1

    if verify_only:
        for waterfall in angle_generator.waterfalls:
            filename = waterfall['name'] + '.json'  # angle.json, might have more in future
            with open(os.path.join(temp_dir, filename)) as f:
                content = f.read()
            angle_filename = os.path.join(THIS_DIR, filename)
            if not write_or_verify_file(angle_filename, content, True):
                print('infra/specs/%s dirty' % filename)
                return 1

    return 0


if __name__ == '__main__':  # pragma: no cover
    sys.exit(main())

Download