glmark2Benchmark.cpp

Show log
Commit

Hash : d9c0aa4a
Author :
Date : 2025-01-09T18:08:08

Reland "Roll third_party/glmark2/src/ ca8de51fe..2054465f6 (77 commits)"

This is a reland of commit 68b594e859f036b01974f78f8ddca8c419bc8ce3

Includes fixes to glmark2Benchmark.cpp to parse updated output.

Original change's description:
> Roll third_party/glmark2/src/ ca8de51fe..2054465f6 (77 commits)
>
> https://chromium.googlesource.com/external/github.com/glmark2/glmark2/+log/ca8de51fedb7..2054465f6f7c
>
> $ git log ca8de51fe..2054465f6 --date=short --no-merges --format='%ad %ae %s'
> 2024-04-25 alexandros.frantzis Use std::filesystem throughout the codebase
> 2024-04-25 alexandros.frantzis android: Build native code with C++17
> 2024-04-25 alexandros.frantzis Doc: Recommend meson for building win32 flavors
> 2024-04-25 alexandros.frantzis github: Build win32 flavors with msvc in CI
> 2024-04-25 alexandros.frantzis github: Build win32 flavors with mingw in CI
> 2024-04-25 alexandros.frantzis include/getopt: Always use non-const char array to avoid MSVC errors
> 2024-04-25 alexandros.frantzis libmatrix: Add utility function to get idle time
> 2024-04-25 alexandros.frantzis libmatrix: Add utility function to get process times
> 2024-04-24 alexandros.frantzis libmatrix: Add utility function to get the number of processors
> 2024-04-25 alexandros.frantzis libmatrix: Use C++ std::chrono instead of clock_gettime
> 2024-04-25 alexandros.frantzis libmatrix: Use C++17 std::filesystem instead of dirent
> 2024-04-25 alexandros.frantzis Build: Add support for win32 flavors in meson
> 2024-04-24 alexandros.frantzis libmatrix: Provide a copy of the va_list to __android_log_vprint
> 2024-04-13 kode54 NativeStateX11: Fix WM_DELETE_WINDOW handling
> 2024-02-15 alexandros.frantzis NativeStateWayland: Explicitly destroy all Wayland resources at shutdown
> 2024-02-15 alexandros.frantzis Explicitly release GL related resources at shutdown
> 2023-06-03 prlw1 src/meson.build: add x11 dependency to glx
> 2024-02-07 timchen021 Fix memory leaks using smart pointers
> 2024-01-12 daniel.levin Add missing includes in scene-collection.h
> 2023-11-01 alexandros.frantzis GLVisualConfig: By default don't care about the stencil config component
> 2023-11-01 alexandros.frantzis Options,GLStateEGL,GLStateGLX: Add option to require a good visual config
> 2023-07-26 abs GLStateEGL,GLStateGLX: Change failure to get a "good" visual config to warning
> 2023-11-01 alexandros.frantzis libmatrix: Add Log::warning() function
> 2023-05-20 prlw1 meson.build: fix build and avoid sigsegv
> 2023-05-03 ofourdan NativeStateX11: Add winsys option for position
> 2023-01-19 alexandros.frantzis Build,Doc: Update files for 2023.01 release
> 2023-01-19 alexandros.frantzis NativeStateMir: Remove Mir/mirclient support
> 2023-01-03 alexandros.frantzis ResultsFile: Support storing results in an XML file
> 2023-01-03 alexandros.frantzis ResultsFile: Support storing results in a CSV file
> 2023-01-03 alexandros.frantzis Options,ResultsFile: Add option to set the results file
> 2023-01-03 alexandros.frantzis Introduce infrastructure to support saving results to a file
> 2023-01-03 alexandros.frantzis CanvasGeneric: Store individual info elements in separate strings
> 2023-01-03 alexandros.frantzis MainLoop: Store indivitual result values in separate strings
> 2023-01-03 alexandros.frantzis Scene: Remove trailing ':' from scene info string
> 2023-01-04 alexandros.frantzis SceneTerrain: Silence unused variable warning
> 2022-12-21 pierre-yves.mordret NativeStateGBM: Lock and release the front buffer on flip
> 2022-12-21 alexandros.frantzis MainLoop,Scene: Support reporting shader compilation time
> 2022-12-20 alexandros.frantzis MainLoop,Scene: Add busy percentage in CPU time results
> 2022-12-20 alexandros.frantzis MainLoop,Scene: Support reporting CPU time results
> 2022-12-20 alexandros.frantzis Options: Introduce option to specify benchmark results to show
> 2022-12-20 alexandros.frantzis Scene: Update elapsed times when a benchmark finishes
> 2022-12-19 alexandros.frantzis Scene: Introduce struct to keep track of elapsed time
> 2022-12-15 alexandros.frantzis Scene: Introduce infrastructure for more complex scene stats
> 2022-12-15 alexandros.frantzis Scene: Move common preparation logic to Scene::prepare
> 2022-12-13 alexandros.frantzis Scene: Introduce non-virtual methods to prepare and finish a scene for benchmark
> 2022-12-08 alexandros.frantzis GLVisualConfig: Support specifying the config id in --visual-config
> 2022-12-08 alexandros.frantzis libmatrix: Support prefix dependent input conversions from string
> 2022-12-08 alexandros.frantzis GLStateEGL,GLStateGLX: Fail if no suitable config is found
> 2022-12-08 alexandros.frantzis GLStateEGL,GLStateGLX: Don't limit the configs passed to our selection mechanism
> 2022-12-08 alexandros.frantzis GLVisualConfig: Penalize configs with components smaller than requested
> 2022-11-16 alexandros.frantzis github: Build x11-gl-egl flavor in CI
> 2022-11-14 rilian-la-te build: Add x11-gl-egl flavor
> 2022-11-10 alexandros.frantzis GLStateGLX: Support multisampled configs
> 2022-11-10 l.stach GLStateEGL: Support multisampled configs
> 2022-11-10 l.stach GLVisualConfig: Add multisampling support
> 2022-09-02 120989324 Add missing newline at end of log messages
> 2022-11-10 alexandros.frantzis NativeStateDRM,NativeStateGBM: Support configuring the DRM device
> 2022-11-10 alexandros.frantzis Options: Support passing window system specific options
> 2022-04-23 nunes.erico NativeStateGBM: Add GBM offscreen backend
> 2022-07-07 wuqianhai SceneJellyfish: Fix memory leak
> 2022-07-07 wuqianhai SceneRefract: Fix texture leak
> 2022-07-07 wuqianhai SceneIdeas: Fix texture leak
> 2022-06-14 Martin.Jansa waflib: fix compatibility with python-3.11
> 2022-05-12 jeffy.chen NativeStateDRM: Honor visual configuration
> 2022-05-13 alexandros.frantzis GLStateEGL,GLStateGLX: Use SwapInterval 1 for FIFO swap mode
> 2022-04-15 alexandros.frantzis NativeStateDrm: Implement swap interval 0
> 2022-02-14 alexandros.frantzis github: Run apt-get update to get latest packages
> 2022-02-14 alexandros.frantzis CanvasGeneric,CanvasAndroid: Display surface config info
> 2022-02-14 alexandros.frantzis Options: Support --version command line argument
> 2022-02-14 alexandros.frantzis libmatrix: Add missing <utility> include
> 2022-02-10 hillma libmatrix: avoid -Wimplicit-fallthrough
> 2021-12-23 alexandros.frantzis Build,Doc: Update files for 2021.12 release
> 2021-12-23 alexandros.frantzis NativeStateDrm: Fix crash when we can't become DRM master
> 2021-09-30 jf android: Fix benchmark parameter edition
> 2021-08-30 alexandros.frantzis Use highp precision modifier only on GLES
> 2021-08-30 alexandros.frantzis GLStateEGL: Support EGL versions < 1.4
> 2021-04-11 maahiuzeon Fix model loading on big endian.
>
> Created with:
>   roll-dep third_party/glmark2/src
>
> Bug: chromium:40263312
> No-Presubmit: true
> Change-Id: Ifee506badac67ff5af57605994daff01278c6d56
> Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/6162277
> Reviewed-by: Yuly Novikov <ynovikov@chromium.org>
> Auto-Submit: Takuto Ikuta <tikuta@chromium.org>
> Commit-Queue: Yuly Novikov <ynovikov@chromium.org>

Test: angle_perftests --gtest_filter="*GLMark2Benchmark*"
Bug: chromium:40263312
Change-Id: I54d8982cb37e488f80b7511ad0111d1ff111f004
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/6174747
Reviewed-by: Roman Lavrov <romanl@google.com>
Commit-Queue: Cody Northrop <cnorthrop@google.com>
Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org>

Download

src/tests/perf_tests/glmark2Benchmark.cpp

//
// Copyright 2019 The ANGLE Project Authors. All rights reserved.
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.
//
// glmark2Benchmark:
//   Runs the glmark2 benchmark.
//

#include <gtest/gtest.h>

#include <stdio.h>
#include <sstream>

#include "../perf_tests/third_party/perf/perf_result_reporter.h"
#include "ANGLEPerfTestArgs.h"
#include "common/platform.h"
#include "common/string_utils.h"
#include "common/system_utils.h"
#include "test_utils/angle_test_configs.h"
#include "test_utils/angle_test_instantiate.h"
#include "util/test_utils.h"

using namespace angle;

namespace
{

struct BenchmarkInfo
{
    const char *glmark2Config;
    const char *name;
};

// Each glmark2 scene is individually benchmarked.  If glmark2 is run without a specific benchmark,
// it can produce an aggregate score, which is not interesting at the moment.  Adding an empty
// string ("") to this list will enable a test where glmark2 is run with the default scenes and the
// score for each test as well as the overall score is output.
constexpr BenchmarkInfo kBenchmarks[] = {
    {"build:use-vbo=false", "build"},
    {"build:use-vbo=true", "build_vbo"},
    {"texture:texture-filter=nearest", "texture_nearest"},
    {"texture:texture-filter=linear", "texture_linear"},
    {"texture:texture-filter=mipmap", "texture_mipmap"},
    {"shading:shading=gouraud", "shading_gouraud"},
    {"shading:shading=blinn-phong-inf", "shading_blinn_phong"},
    {"shading:shading=phong", "shading_phong"},
    {"shading:shading=cel", "shading_cel"},
    {"bump:bump-render=high-poly", "bump_high_poly"},
    {"bump:bump-render=normals", "bump_normals"},
    {"bump:bump-render=height", "bump_height"},
    {"effect2d:kernel=0,1,0;1,-4,1;0,1,0;", "effect2d_edge"},
    {"effect2d:kernel=1,1,1,1,1;1,1,1,1,1;1,1,1,1,1;", "effect2d_blur"},
    {"pulsar:light=false:quads=5:texture=false", "pulsar"},
    {"desktop:blur-radius=5:effect=blur:passes=1:separable=true:windows=4", "desktop_blur"},
    {"desktop:effect=shadow:windows=4", "desktop_shadow"},
    {"buffer:columns=200:interleave=false:update-dispersion=0.9:update-fraction=0.5:update-method="
     "map",
     "buffer_map"},
    {"buffer:columns=200:interleave=false:update-dispersion=0.9:update-fraction=0.5:update-method="
     "subdata",
     "buffer_subdata"},
    {"buffer:columns=200:interleave=true:update-dispersion=0.9:update-fraction=0.5:update-method="
     "map",
     "buffer_map_interleave"},
    {"ideas:speed=duration", "ideas"},
    {"jellyfish", "jellyfish"},
    {"terrain", "terrain"},
    {"shadow", "shadow"},
    {"refract", "refract"},
    {"conditionals:fragment-steps=0:vertex-steps=0", "conditionals"},
    {"conditionals:fragment-steps=5:vertex-steps=0", "conditionals_fragment"},
    {"conditionals:fragment-steps=0:vertex-steps=5", "conditionals_vertex"},
    {"function:fragment-complexity=low:fragment-steps=5", "function"},
    {"function:fragment-complexity=medium:fragment-steps=5", "function_complex"},
    {"loop:fragment-loop=false:fragment-steps=5:vertex-steps=5", "loop_no_fsloop"},
    {"loop:fragment-steps=5:fragment-uniform=false:vertex-steps=5", "loop_no_uniform"},
    {"loop:fragment-steps=5:fragment-uniform=true:vertex-steps=5", "loop"},
};

struct GLMark2TestParams : public PlatformParameters
{
    BenchmarkInfo info;
};

std::ostream &operator<<(std::ostream &os, const GLMark2TestParams &params)
{
    os << static_cast<const PlatformParameters &>(params) << "_" << params.info.name;
    return os;
}

class GLMark2Benchmark : public testing::TestWithParam<GLMark2TestParams>
{
  public:
    GLMark2Benchmark()
    {
        switch (GetParam().getRenderer())
        {
            case EGL_PLATFORM_ANGLE_TYPE_D3D11_ANGLE:
                mBackend = "d3d11";
                break;
            case EGL_PLATFORM_ANGLE_TYPE_OPENGLES_ANGLE:
                mBackend = "gl";
                break;
            case EGL_PLATFORM_ANGLE_TYPE_VULKAN_ANGLE:
                mBackend = "vulkan";
                break;
            case EGL_PLATFORM_ANGLE_TYPE_METAL_ANGLE:
                mBackend = "metal";
                break;
            default:
                break;
        }
        std::string story = GetParam().info.name;
        mReporter = std::make_unique<perf_test::PerfResultReporter>("glmark2_" + mBackend, story);
        mReporter->RegisterImportantMetric(".fps", "fps");
        mReporter->RegisterImportantMetric(".score", "score");
    }

    void run()
    {
        // Only supported on Linux and Windows at the moment.
        if (!IsLinux() && !IsWindows())
        {
            return;
        }

        const BenchmarkInfo benchmarkInfo = GetParam().info;
        const char *benchmark             = benchmarkInfo.glmark2Config;
        const char *benchmarkName         = benchmarkInfo.name;
        bool completeRun                  = benchmark == nullptr || benchmark[0] == '\0';

        Optional<std::string> cwd = GetCWD();

        // Set the current working directory to the executable's, as the data path of glmark2 is
        // set relative to that path.
        std::string executableDir = GetExecutableDirectory();
        SetCWD(executableDir.c_str());
        SetEnvironmentVar("ANGLE_DEFAULT_PLATFORM", mBackend.c_str());

        std::vector<const char *> args = {
            "glmark2_angle",
        };
        if (OneFrame())
        {
            args.push_back("--validate");
        }
        if (!completeRun)
        {
            args.push_back("--benchmark");
            args.push_back(benchmark);
            fprintf(stderr, "Running benchmark: %s\n", benchmark);
        }
        args.push_back(nullptr);

        ProcessHandle process(args, ProcessOutputCapture::StdoutOnly);
        ASSERT_TRUE(process && process->started());
        ASSERT_TRUE(process->finish());

        // Restore the current working directory for the next tests.
        if (cwd.valid())
        {
            SetCWD(cwd.value().c_str());
        }

        ASSERT_EQ(EXIT_SUCCESS, process->getExitCode());

        if (!OneFrame())
        {
            std::string output = process->getStdout();
            parseOutput(output, benchmarkName, completeRun);
        }
    }

  private:
    void parseOutput(const std::string &output, const char *benchmarkName, bool completeRun)
    {
        // Output is in the following format:
        //
        // =======================================================
        //     glmark2 2017.07
        // =======================================================
        //     OpenGL Information
        //     GL_VENDOR:      ...
        //     GL_RENDERER:    ...
        //     GL_VERSION:     ...
        //     Surface Config: ...
        //     Surface Size:   ...
        // =======================================================
        // [test] config: FPS: uint FrameTime: float ms
        // [test] config: Not Supported
        // ...
        // =======================================================
        //                                   glmark2 Score: uint
        // =======================================================
        //
        // This function skips the header, prints one line for each test/config line where there's
        // an FPS value, and finally prints the overall score.
        std::istringstream glmark2Output(output);
        std::string line;

        // Forward any INFO: lines that may have been generated.
        while (std::getline(glmark2Output, line) && BeginsWith(line, "INFO:"))
        {
            fprintf(stderr, "%s\n", line.c_str());
        }

        // Expect ==== at the top of the header
        ASSERT_EQ('=', line[0]);

        // Skip one line
        std::getline(glmark2Output, line);

        // Expect ==== in the middle of the header
        std::getline(glmark2Output, line);
        ASSERT_EQ('=', line[0]);

        // Skip four lines
        std::getline(glmark2Output, line);
        std::getline(glmark2Output, line);
        std::getline(glmark2Output, line);
        std::getline(glmark2Output, line);

        // The fourth line is the GL_VERSION.  Expect it to include ANGLE, otherwise we are not
        // running against ANGLE.
        ASSERT_NE(line.find("ANGLE"), std::string::npos);

        // Skip two Surface lines
        std::getline(glmark2Output, line);
        std::getline(glmark2Output, line);

        // Expect ==== at the bottom of the header
        std::getline(glmark2Output, line);
        ASSERT_EQ('=', line[0]);

        // Read configs until the top of the footer is reached
        while (std::getline(glmark2Output, line) && line[0] != '=')
        {
            // Parse the line
            std::istringstream lin(line);

            std::string testName, testConfig;
            lin >> testName >> testConfig;
            EXPECT_TRUE(lin);

            std::string fpsTag, frametimeTag;
            size_t fps;
            float frametime;

            lin >> fpsTag >> fps >> frametimeTag >> frametime;

            // If the line is not in `FPS: uint FrameTime: Float ms` format, the test is not
            // supported.  It will be skipped.
            if (!lin)
            {
                continue;
            }

            EXPECT_EQ("FPS:", fpsTag);
            EXPECT_EQ("FrameTime:", frametimeTag);

            if (!completeRun)
            {
                mReporter->AddResult(".fps", fps);
            }
        }

        // Get the score line: `glmark2 Score: uint`
        std::string glmark2Tag, scoreTag;
        size_t score;
        glmark2Output >> glmark2Tag >> scoreTag >> score;
        EXPECT_TRUE(glmark2Output);
        EXPECT_EQ("glmark2", glmark2Tag);
        EXPECT_EQ("Score:", scoreTag);

        if (completeRun)
        {
            mReporter->AddResult(".score", score);
        }
    }

    std::string mBackend = "invalid";
    std::unique_ptr<perf_test::PerfResultReporter> mReporter;
};

TEST_P(GLMark2Benchmark, Run)
{
    run();
}

GLMark2TestParams CombineEGLPlatform(const GLMark2TestParams &in, EGLPlatformParameters eglParams)
{
    GLMark2TestParams out = in;
    out.eglParameters     = eglParams;
    return out;
}

GLMark2TestParams CombineInfo(const GLMark2TestParams &in, BenchmarkInfo info)
{
    GLMark2TestParams out = in;
    out.info              = info;
    return out;
}

using namespace egl_platform;

std::vector<GLMark2TestParams> gTestsWithInfo =
    CombineWithValues({GLMark2TestParams()}, kBenchmarks, CombineInfo);
std::vector<EGLPlatformParameters> gEGLPlatforms = {D3D11(), METAL(), OPENGLES(), VULKAN()};
std::vector<GLMark2TestParams> gTestsWithPlatform =
    CombineWithValues(gTestsWithInfo, gEGLPlatforms, CombineEGLPlatform);

ANGLE_INSTANTIATE_TEST_ARRAY(GLMark2Benchmark, gTestsWithPlatform);

}  // namespace

Download