kc3-lang/angle/src/tests/perf_tests/ANGLEPerfTestArgs.cpp

src/tests/perf_tests/ANGLEPerfTestArgs.cpp

Log

Author Commit Date CI Message
Shashank Sharma 0c9ce8c7 2023-09-08T21:34:07 Trace-based testing app Modifying angle_trace_tests app to run traces automatically on invocation and dumps fps information. Also, compares the screenshot with the golden images if provided and dumps the histogram of image diff. Golden image naming format: <trace_name>_golden.png - Run traces - Capture screenshot - Record FPS value - Record image diff histogram Bug: b/270421213 Doc: doc/StandaloneBenchmark.md Change-Id: I42b8d229e1e358d48887f29f2424b9e771093ce9 Commit-Queue: Shashank Sharma <shawshanks@google.com> Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/6072349 Commit-Queue: Cody Northrop <cnorthrop@google.com> Auto-Submit: Shashank Sharma <shawshanks@google.com> Reviewed-by: Roman Lavrov <romanl@google.com> Reviewed-by: Cody Northrop <cnorthrop@google.com>
Roman Lavrov c174aa70 2024-12-05T15:36:18 Trace perf: add a basic fps limiter Adds sleeps based on each 4 frames timing (4 instead of just last frame makes fps a bit closer to requested) Example: --fps-limit=30 *RESULT TracePerf_vulkan.wall_time: offscreen_bubble_shooter_and_friends= 33.3556469900 ms *RESULT TracePerf_vulkan.wall_time: bubble_shooter_and_friends= 33.4376432467 ms --fps-limit=50 *RESULT TracePerf_vulkan.wall_time: offscreen_bubble_shooter_and_friends= 20.0218095000 ms *RESULT TracePerf_vulkan.wall_time: bubble_shooter_and_friends= 20.0624947200 ms --fps-limit=100 *RESULT TracePerf_vulkan.wall_time: offscreen_bubble_shooter_and_friends= 10.1298787483 ms *RESULT TracePerf_vulkan.wall_time: bubble_shooter_and_friends= 10.1781205983 ms --fps-limit=200 *RESULT TracePerf_vulkan.wall_time: offscreen_bubble_shooter_and_friends= 5.1289075067 ms *RESULT TracePerf_vulkan.wall_time: bubble_shooter_and_friends= 5.1571364367 ms --fps-limit=400 *RESULT TracePerf_vulkan.wall_time: offscreen_bubble_shooter_and_friends= 2.6172564224 ms *RESULT TracePerf_vulkan.wall_time: bubble_shooter_and_friends= 2.7648291614 ms Bug: b/376300037 Change-Id: I63ef75fe342b149336c0e274bd9cbf6c3aa34c23 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/6072435 Commit-Queue: Roman Lavrov <romanl@google.com> Reviewed-by: Cody Northrop <cnorthrop@google.com>
Roman Lavrov d6fd7eaf 2023-08-25T16:33:27 Trace/perf tests: remove calibration and most warmup options This removes calibration and warmup except for --warmup (which does what --warmup-steps=-1 used to do: render each frame once with glFinish) Trace and perf tests now default to no warmup. Unchanged by this CL: --fixed-test-time=x runs a single trial for x seconds. --fixed-test-time-with-warmup=x is effectively the above flag coupled with --warmup-steps=-1 --run-to-key-frame runs to the key frame (no longer disables warmup as it's now off by default) Bug: b/297418214 Change-Id: I7a0d3e490067dbde57c5f519c1b9092ba70b1480 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4812049 Reviewed-by: Cody Northrop <cnorthrop@google.com> Commit-Queue: Roman Lavrov <romanl@google.com> Reviewed-by: Amirali Abdolrashidi <abdolrashidi@google.com>
Mark Lobodzinski d85de0e9 2023-08-09T14:15:56 Capture/Replay: Add optional replay of trimmed resources This CL: - Adds '--include-inactive-resources' option to angle_trace_tests - Removes the 'trim-enabled' option - Outputs all previously trimmed shaders/programs to trace file in a new Setup function, SetupReplayContextSharedInactive() which is executed only if the new option is specified - Modifies CaptureTest to add inactive resources, but does not set the include-inactive-resources flag Bug: b/296055694 Change-Id: I33b18d5da727d55c90c2012c2bf64b1413521429 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4781552 Reviewed-by: Cody Northrop <cnorthrop@google.com> Commit-Queue: Mark Łobodziński <mark@lunarg.com> Reviewed-by: Roman Lavrov <romanl@google.com>
Roman Lavrov 4e073356 2023-04-13T15:16:53 Trace Interpreter: gzip cpp and run gold tests. New test step: angle_restricted_trace_gold_interpreted_tests similar to existing angle_restricted_trace_gold_tests but runs a subset of newest traces that have the right format. Currently there are 8 of them and the filter is hard-coded in test_suites.pyl. rise_of_empires can also be run interpreted but happened to flake during testing so it's not included for now. Trace cpp files are concatenated and gzipped as a build step producing gen/tracegz_{trace}.gz and these gz files are then distributed as part of the build --trace-interpreter flag changes from boolean to string to support: --trace-interpreter=gz: * this uses the gz file above * gold tests added in this CL use this option --trace-interpreter=c: * using uncompressed c/cpp file * existing angle_trace_interpreter_tests runs a retrace saving .c files and uses this option Bug: b/276742336 Change-Id: I69544f25bda873af191978195d02ffbdd34363c0 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4424690 Reviewed-by: Cody Northrop <cnorthrop@google.com> Commit-Queue: Roman Lavrov <romanl@google.com>
Roman Lavrov 074b98fe 2023-04-14T12:28:29 Remove unnecessary and unused --interpreter flag. The one we're using is --trace-interpreter and is more clear. Bug: b/276742336 Change-Id: Iea17ca54a17f41a0445cbc4d192ff1d9f5315cf5 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4428750 Commit-Queue: Cody Northrop <cnorthrop@google.com> Commit-Queue: Roman Lavrov <romanl@google.com> Auto-Submit: Roman Lavrov <romanl@google.com> Reviewed-by: Cody Northrop <cnorthrop@google.com>
Cody Northrop d9dbc20e 2023-02-23T17:10:06 TraceTests: Add a screenshot frame member variable Instead of modifying a global when detecting KeyFrames are in use by a trace, update a new value tracked per trace. Also consolidate the option handling for gRunToKeyFrame since its needs align with gMaxStepsPerformed. Test: Batching traces that include one with a keyframe Bug: angleproject:8035 Bug: b/270426257 Change-Id: I2ed4215d9f310dddb317debf09921aa45c6822bf Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4288616 Reviewed-by: Yuxin Hu <yuxinhu@google.com> Commit-Queue: Cody Northrop <cnorthrop@google.com>
Cody Northrop b0739436 2023-02-13T14:28:17 Traces: Add key frame support Allow specifying a key frame in the trace JSON, i.e: "KeyFrames": [ 20 ], This allows our infra to use a frame other than 1 when taking screenshots for quality comparison. Adds new flag `--run-to-key-frame`, which will stop the trace once key frame has been reached. If no key frame in JSON, frame 1 will be used. Note the name in JSON is plural, but we only support one key frame for now. Multiple key frame support can come in the future. This CL also updates the code to allow ending traces early with `--max-steps-performed` which has been broken since http://crrev/c/4008998 It also removes `--one-frame-only` which is superseded by `--run-to-key-frame`, and can be replicated using `--max-steps-performed 1`. Test: angle_trace_tests --gtest_filter="*tmnt_shredders_revenge*" Bug: angleproject:8035 Bug: b/270426257 Change-Id: Ib02ef60d887ae5efb0288f5a9b8c2914dafc6efc Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4284637 Commit-Queue: Cody Northrop <cnorthrop@google.com> Reviewed-by: Yuly Novikov <ynovikov@chromium.org>
Roman Lavrov 2ce2146b 2022-12-12T15:31:29 Fix --render-test-output-dir to work in TestSuite. https://crrev.com/c/3956937 introduced the issue where this flag is consumed by ANGLEPerfTestArgs.cpp logic which runs before the TestSuite constructor and removes the flag so we end up with an empty mTestArtifactDirectory in TestSuite. Bug: angleproject:7755 Change-Id: Ic32d3a2ddceca7842a0aebe5a1e40cbedaedf2eb Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4097325 Reviewed-by: Jamie Madill <jmadill@chromium.org> Commit-Queue: Roman Lavrov <romanl@google.com>
Cody Northrop 1f69f580 2022-11-18T14:33:20 Perf tests: Add --fixed-test-time-with-warmup Warmup trials combined with fixed-test-time was broken recently. It was already fragile and only worked when parameters were provided in the right order. On Android we're relying on this combination to get steady results without hitting thermal limits. To address this: - Add a new parameter that encapsulates the combination - ASSERT that flags aren't provided in unsupported combinations Test: python3 restricted_trace_perf.py --fixedtime 10 Bug: angleproject:7755 Change-Id: I56fce1f65d4f0538e623951163a9c8736cb3fb03 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4027282 Reviewed-by: Roman Lavrov <romanl@google.com> Reviewed-by: Jamie Madill <jmadill@chromium.org> Commit-Queue: Cody Northrop <cnorthrop@google.com>
Sean Risser f3e047b5 2022-10-12T16:28:32 Add new retrace command get_min_reqs `retrace_restricted_traces.py get_min_reqs` finds the minimum required GLES version and extensions and record them to a trace's json file. TracePerfTest has also been modified to use the new "RequiredExtensions" json field to determine if it should run a test or not. This allows several lines to be removed from TracePerfTest's constructor. get_min_reqs uses a divide-and-conquer strategy to find the extensions. This approach should be faster for traces where the number of required extensions is less than 20. This adds a new command to retrace_restricted_traces.py called get_min_reqs. This runs an existing trace multiple times to find what the minimum required GLES version is, then records that version to the trace's json file. Bug: angleproject:7639 Change-Id: I42530e438054efec1ad79ababebe06d221d5c695 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3884175 Reviewed-by: Geoff Lang <geofflang@chromium.org> Reviewed-by: Cody Northrop <cnorthrop@google.com> Commit-Queue: Sean Risser <srisser@google.com>
Jamie Madill dc62b3ee 2022-10-10T21:00:16 Capture/Replay: Add trace interpreter. Also adds a self-test using the retrace script. Bug: angleproject:7752 Change-Id: I1985b47250bef99726d2ca2d90bef859208e357e Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3965128 Reviewed-by: Cody Northrop <cnorthrop@google.com> Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Yuxin Hu <yuxinhu@google.com>
Jamie Madill bc2e9eea 2022-10-15T14:38:15 Refine sampling of trace perf tests. - The warmup algorithm now computes an accurate steps estimate. - Bump up the trials, samples, and trial time in the runner. Bug: angleproject:7671 Change-Id: I89a73472fd72af73f695c9107119cbc559d2d6e1 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3956946 Reviewed-by: Cody Northrop <cnorthrop@google.com> Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Roman Lavrov <romanl@google.com>
Jamie Madill a41c6173 2022-10-16T08:28:53 Simplify trace test names. Originally we had a lot of conditionally enabled test configs in angle_perftests, that we enabled behind a flag because the total runtime would be too large if we left everything enabled. This CL switches to using a single test set, one for each trace, and using a the same command-line API we use in dEQP and the ANGLE samples to pick a single configuration. This makes it easier to pick exactly what configurations you want from the command line. Because we only run one configuration each time, we don't need to specialize the test name to include the configuration info. We can also simplify TracePerfTest -> TraceTest since these tests are run for both perf and correctness. Old name: TracePerfTest.Run/vulkan_trex_200 New name: TraceTest.trex_200 Example for tests that don't use default arguments: Old args: --enable-all-trace-tests Old name: TracePerfTest.Run/native_offscreen_trex_200 New args: --use-gl=native --offscreen New name: TraceTest.trex_200 Bug: angleproject:7755 Change-Id: Id93cf5725eff2e2c6b3b14804bc2902f38306e7e Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3966535 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Cody Northrop <cnorthrop@google.com> Reviewed-by: Roman Lavrov <romanl@google.com>
Jamie Madill e495e7fd 2022-10-14T13:52:47 Redo perf and trace test parameters. - Moves common argument parsing code into test_util. - Changes the perf test arg parsing to use the common functions. - Adds new --use-angle and --use-gl parameters to the trace tests. - Also adds new --offscreen and --vsync parameters to the traces. - Removes the now unneeded --enable-all-trace-tests argument. - Both --arg=value and --arg value work in test suites now. Now, instead of using --enable-all-trace-tests you can specify the backend with --use-angle=swiftshader, --offscreen, or combinations of those parameters. The test names are the same as they were before, but only the configured tests will run in a session. We could opt to simplify the test names in later CLs if we want to simplify the test running. Ideally we'd keep the perf reporting the same because then we'd keep the time series the same on the test infra. This also allows us to split up the trace tests into separate targets on the bots, which will better allow us to control the workloads and sampling of the tests. For example: - angle_perftests becomes - angle_perftests (microbenchmarks) - angle_trace_perf_vulkan_tests (traces with vulkan back-end) - angle_trace_perf_native_tests (traces with system GL) Bug: angleproject:7755 Change-Id: I537168f3a6de96425dfda05ed98220eff9b19b76 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3956937 Reviewed-by: Yuly Novikov <ynovikov@chromium.org> Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Cody Northrop <cnorthrop@google.com>
Jamie Madill 04f3ed80 2022-10-04T08:24:16 Perf Tests: Use specified trial time for warmup. When the user specifies a trial time, use this for the warmup time instead of using the calibration time. This will give a longer warmup time on our test infra and should help stabilize perf testing scores on our bots. Bug: angleproject:7671 Change-Id: I65fb8a7ae1f684d3f7834613d8341d17bd2df00f Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3933315 Reviewed-by: Yuly Novikov <ynovikov@chromium.org> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Jamie Madill 67af8d89 2022-09-19T15:19:57 Perf Tests: Make dynamic step count calibration optional. This uses a fixed test time for the tests, instead of using an extra step to calibrate a number of steps to run in an approximate maximum time. Before: 1. run a test for <test time>, return the number of steps 2. for <num samples> x <num trials> times, run the tests for that number of steps After: 1. for <num samples> x <num trials> times, run the tests for <test time> This saves some total time since we don't need to run the tests in a pre-process to calibrate the number of steps, and it also makes this test script more like the restricted trace perf script. Bug: angleproject:7671 Change-Id: I413cbf8d65f9f2f87781d8607f799d2450a71ea7 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3906144 Reviewed-by: Cody Northrop <cnorthrop@google.com> Reviewed-by: Roman Lavrov <romanl@google.com> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Roman Lavrov d6acced1 2022-07-25T18:11:20 Add flag --save-screenshots (implied by --screenshot-dir). --screenshot-dir and --render-test-output-dir behave differently on Android. --screenshot-dir is passed as is and therefore means a device directory, and --render-test-output-dir has special handling in Android wrappers that substitutes this flag with a temporary location on the device then copying files to the local directory set by this flag. Cody pointed out that there are many docs referring to --screenshot-dir so behavior should be preserved. So --save-screenshots is implied when this flag is set. Otherwise, it defaults to false, so --render-test-output-dir just on its own won't save screenshots, only when --save-screenshots is added as well. This also allows to produce other artifacts without necessarily producing screenshots by using --render-test-output-dir without --save-screenshots. Bug: angleproject:7299 Change-Id: Ib1a268ddeb7eba9f120e3f37a429ea0e4bbe2411 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3785296 Reviewed-by: Cody Northrop <cnorthrop@google.com> Reviewed-by: Jamie Madill <jmadill@chromium.org> Commit-Queue: Roman Lavrov <romanl@google.com>
Jamie Madill 23558369 2022-04-06T15:41:59 Update perf tests arguments. Bug: angleproject:6776 Change-Id: I3cf51cdf9ad05dc792126354c84e717574638408 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3566219 Reviewed-by: Charlie Lao <cclao@google.com> Reviewed-by: Yuxin Hu <yuxinhu@google.com> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Jamie Madill 3739a195 2022-03-09T13:56:36 perf tests: Record perf counter metrics. This adds a new command line argument that will allow the user to specify perf counters to record into the test output. Bug: angleproject:4918 Change-Id: Ia7432ff96eadf13ef681f67d2d503d00fd83e06e Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3516970 Reviewed-by: Lingfeng Yang <lfy@google.com> Reviewed-by: Charlie Lao <cclao@google.com> Reviewed-by: Yuxin Hu <yuxinhu@google.com> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Jamie Madill 9c05f55a 2021-07-15T10:58:10 Capture/Replay: Add replay state validation. We can use this to validate traces that have slight differences when retracing. The valdation works by embedding the "expected" JSON into a string at the end of each captured frame. The replay also embeds a callback which fires right before the swap in the replay harness. The harness then gets the "actual" JSON and runs a comparison. On a mismatch it calls "diff" externally. Currently the diff call is hard-coded to work on Linux only. Note that when running validation it's important to replay on SwiftShader since that's what we use to retrace. Bug: angleproject:5133 Change-Id: Icbf0031d07be8bd916607c537dec235f9a512c43 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3066008 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Tim Van Patten <timvp@google.com> Reviewed-by: Cody Northrop <cnorthrop@google.com>
Cody Northrop 634d156f 2021-07-08T12:42:21 TracePerfTests: Add ScreenShotFrame option Allows the user to specify a frame to capture instead of defaulting to the first one. Usage: --screenshot-frame <frame> Bug: angleproject:4615 Change-Id: I000ee17aa9b3019188816aa6203f3266b39155b8 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3016356 Reviewed-by: Tim Van Patten <timvp@google.com> Reviewed-by: Jamie Madill <jmadill@chromium.org> Commit-Queue: Cody Northrop <cnorthrop@google.com>
Jamie Madill 69c2d820 2021-06-29T11:58:41 Perf Tests: Register trace tests programatically. Using testing::RegisterTests lets us filter out the disabled configs. This helps sharding the tests because some shards were getting a very uneven distribution of real tests vs skipped tests. A couple shards were running 20 minutes, while most were about 6 minutes. More work might be needed to even that out. Bug: angleproject:6090 Change-Id: Ibdf35677cfea86dfc374a2b18bb57cd9f289ddee Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2994726 Reviewed-by: Tim Van Patten <timvp@google.com> Reviewed-by: Cody Northrop <cnorthrop@google.com> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Cody Northrop 913bc473 2021-05-07T11:37:01 Tests: Add --fixed-test-time option Similar to --max-steps-performed, don't do any calibration and just run flat out for a fixed amount of time. Also rename --test-time to --max-trial-time to reflect what the parameter actually does, limiting each trial to a ceiling. Also remove a redundant check in doRunLoop. Test: angle_perftests --fixed-test-time 20 Test: angle_perftests --max-trial-time 2 Bug: angleproject:5935 Change-Id: I854799a734cd6c019399f5a1444b4ef5f659ddf4 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2878937 Reviewed-by: Jamie Madill <jmadill@chromium.org> Reviewed-by: Charlie Lao <cclao@google.com> Commit-Queue: Cody Northrop <cnorthrop@google.com>
Charlie Lao 4340fcfa 2021-04-02T13:59:17 Vulkan: Add command option to run app traces with minimum GPU work This CL interposes the draw call API entries. If enabled by command line option "--minimize-gpu-work", it will override all draw calls to render a single GL_POINT instead. It forces view port and scissor size to 1x1. It forces all data copy to 1 texel or 1 byte. It ignores glGenerateMipmap call. The goal is to make the frame time reflect the driver CPU performance as much as possible so that we can use it to compare ANGLE vs native GLES on the driver overhead, as well as identifying CPU hot spot. Bug: b/184766477 Change-Id: Ie8b5b585e0ed55320fed7863da37cce419acc65f Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2802858 Commit-Queue: Charlie Lao <cclao@google.com> Reviewed-by: Cody Northrop <cnorthrop@google.com> Reviewed-by: Jamie Madill <jmadill@chromium.org>
Jamie Madill 78dde332 2021-03-29T17:31:52 Move restricted traces to CIPD. All traces are now stored as DEPS entries in CIPD. The auto-generation script generates the DEPS entries. Note that we don't include DEPS in the list of generated outputs to simplify other rollers. Also we update auto-generation to include full sources list to allow 'gn analyze' to work successfully. Usees a trace fixture for common code. This will enable a more compact trace without as much repeated code. We must land a set of re-trace traces to avoid breakage. Also includes a python script for uploading new traces to CIPD. The script first checks if traces are already present in the cloud, and if so it skips the upload. It will take a while to complete as the number of traces grows larger as it takes a few seconds per trace. The traces in this patch are also re-traced to use the common fixture code instead of including duplicated code in each trace. They now form a simple common interface and the autogenerated cpp is now simply a list of properties. I've also updated the capture/replay tests to use the simpler common entry point integration. There is less auto-generated glue code now. We now use a new serialized string query extension instead of calling directly into ANGLE's internals. Also includes a docs update. The capture/replay sample is broken and we'll need to update it in a follow-up CL. Also includes a few necessary fixes to the retracing script. Bug: angleproject:5811 Change-Id: I977bc6dc56843c8966377fc445ae97e91e17319a Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2797833 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Cody Northrop <cnorthrop@google.com> Reviewed-by: Tim Van Patten <timvp@google.com>
Jamie Madill 2c685a41 2021-02-03T11:20:20 Fix the perf test runner calibration. This path was broken for the white box unit tests. Also adds argparse handling to the runner so we can more flexibly override command line arguments. Previously the broken calibration was causing some of the tests to run only a single test iteration when measuring. This could lead to low quality measurements. Bug: angleproject:5573 Change-Id: Ic1cb2b2553774a361325f290440c40b2ff90db5e Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2672702 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Mohan Maiya <m.maiya@samsung.com> Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org>
Jamie Madill 4323ed0c 2020-11-06T12:45:12 Perf Tests: Fix test time with max steps performed. This lets the default test time be 10 minutes when using a maximum steps value. This will prevent the current 10 second limit from kicking in and aborting. Also changes the --test-time override to be more clearly a calibration time, and adds a new --test-time that lets the user override the actual trial test time. Bug: b/172480147 Change-Id: I630aa8d3c9e118bb5f0100b38f37e717e12f3538 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2523371 Reviewed-by: Paul Thomson <paulthomson@google.com> Reviewed-by: Cody Northrop <cnorthrop@google.com> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Jamie Madill af16c638 2020-11-06T09:50:51 Perf Tests: Add --max-steps-performed argument. This will limit the total number of steps run in the test. When using this argument we skip warmp & calibration. Renames the prior --steps argument to --steps-per-trial. Bug: b/172480147 Change-Id: Iae7cce26d66ca0bbdb21c1f511a6bfa246bd204e Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2519404 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Cody Northrop <cnorthrop@google.com>
Jamie Madill 77defeb0 2020-10-27T18:31:53 Add the ability to re-trace existing traces. This new script runs the ANGLE trace tests with capture enabled to generate updated replay cpp files. This allows us to update our traces files to a new file format in one step. Trace metadata (currently only the default FBO format) is preserved between re-traces. Currently only desktop-based retracing is supported. This means a couple traces that require specific extensions don't run on all platforms. Bug: angleproject:5134 Change-Id: I7c923d89e33c18285ab36a7cee91f2fb735758eb Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2488130 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Courtney Goeltzenleuchter <courtneygo@google.com> Reviewed-by: Cody Northrop <cnorthrop@google.com>
Jamie Madill 51ff0d05 2020-10-16T11:54:04 Trace Tests: Add vsync-limited mode with a GPU timer. Local testing on Linux shows that ANGLE and native have the same wall-time, and ANGLE has a 10 ms vs native 6 ms GPU time. Bug: angleproject:5183 Change-Id: I26da678b984af916bfb40e1db6416f5df4f85c08 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2480782 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Cody Northrop <cnorthrop@google.com> Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org>
Jamie Madill ecbac31c 2020-10-16T10:36:06 Perf Tests: Add offscreen mode. This lets the trace perf tests run configurable number of frames within a single swap. The offscreen config is similar to how gfxbench works. It renders to a user FBO (by overriding calls to BindFramebuffer) and then composits multiple frames into the real backbuffer. This allows us to get a perf measurement with less overhead from composition and display. Adds emulation for some APIs that operate on Framebuffers like BindFramebuffer, Invalidate, DrawBuffers and ReadBuffer. Bug: angleproject:4845 Change-Id: I1044c1d52c82f1c215a68a6c46d74c52ed0f3d2a Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2300207 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Cody Northrop <cnorthrop@google.com> Reviewed-by: Courtney Goeltzenleuchter <courtneygo@google.com>
Jamie Madill a22a19a4 2020-10-04T13:16:46 Add a --no-finish option to the perf tests. We can use this to re-capture replays. Bug: angleproject:5134 Change-Id: I765e0bbb66cfdd822ffa981f2b5533157d024cac Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2449161 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Cody Northrop <cnorthrop@google.com> Reviewed-by: Courtney Goeltzenleuchter <courtneygo@google.com>
Jamie Madill f8e5b97e 2020-09-28T23:31:41 Additional perf test cleanup & functionality. Adds several new command line arguments and documents all args in the README. Changes the trace tests to output milliseconds. Bug: b/169600938 Change-Id: Ie8c7840a6a8958951992c806979dde6dc7d8a709 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2438194 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Courtney Goeltzenleuchter <courtneygo@google.com> Reviewed-by: Cody Northrop <cnorthrop@google.com>
Jamie Madill 5ec560fb 2020-09-17T17:18:25 Add restricted trace golden image script test. The script reads the list of tests from the restricted traces JSON and runs angle_perftests once for each test. Running the tests in a batch seems unstable on some platfroms so running each test separately works around that limitation. It does not yet interact with Skia Gold. The Gold integration will come later once this basic skeleton is running on the bots. Also updates the perf tests to exit more quickly with --one-frame-only. This will speed up the screenshots. Example run with trigger.py: https://chromium-swarm.appspot.com/task?id=4ecb985330a33910 Bug: angleproject:4090 Bug: b/168049670 Test: tested with trigger.py Change-Id: I52820dfba2acfbc5d638673c37297877e8476adb Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2417641 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Yuly Novikov <ynovikov@chromium.org> Reviewed-by: Cody Northrop <cnorthrop@google.com>
Jamie Madill 4e7a6a69 2020-07-15T19:35:43 Perf Tests: Fix trace test calibration. The trace tests were including the startup time in their calibration calculations. Then was forcing them to always render one step. This fixes the calibration to attempt to get a more consistent measurement. The trace tests now render many more steps than they did before. It should increase the stability and decrease variance in our measurements as they will be runnnig many more frames than before. Also adds a verbose logging flag to help debugging. Noticed this bug when working on the trace test offscreen mode. Bug: angleproject:4845 Change-Id: Iff0c987008a935e7051fca34ef12f4433eb46092 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2300205 Reviewed-by: Yuly Novikov <ynovikov@chromium.org> Reviewed-by: Cody Northrop <cnorthrop@google.com> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Jamie Madill d08f1d8d 2020-05-05T12:54:07 Perf tests: Add screenshot capture mode. This adds a "--screenshot-dir" argument to capture screenshots. If we're running with screenshot capture then the test will early exit after the first capture. The screenshots use the same naming pattern as the test name: TracePerfTest.Run/vulkan_trex_200 -> angle_vulkan_trex_200.png Note the screenshot dir is relative to the test binary directory, not the CWD. Also adds a PNG saving utility function. Bug: angleproject:4615 Change-Id: I1de8ae6a6e6892586bb0b743e7b9a842f90f98e5 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2184834 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Cody Northrop <cnorthrop@google.com>
Jamie Madill 42c453fe 2020-03-15T18:23:15 Move perf test common files to a source set. Similarly to a prior patch this organizes the common perf test sources into a single target. This will allow us to more easily support setting compiler flags on particular perf test sources. This also fixes some build errors that popped up when new warnings were enabled. Bug: angleproject:3630 Change-Id: I866885199f93a5055e8d160e39f72d1f8209bc75 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2103086 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Yuly Novikov <ynovikov@chromium.org>
Shahbaz Youssefi f576a708 2019-02-04T16:52:12 Add glmark2 to angle_perftests Bug: angleproject:3125 Change-Id: I9242743c6b5c6e18d0a23ff853ef6b9b370865a6 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/1452956 Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org> Reviewed-by: Cody Northrop <cnorthrop@google.com>
kmx.io     mail     discord kmxgit v0.4.0