|
2ce2146b
|
2022-12-12T15:31:29
|
|
Fix --render-test-output-dir to work in TestSuite.
https://crrev.com/c/3956937 introduced the issue where this flag is
consumed by ANGLEPerfTestArgs.cpp logic which runs before the TestSuite
constructor and removes the flag so we end up with an empty
mTestArtifactDirectory in TestSuite.
Bug: angleproject:7755
Change-Id: Ic32d3a2ddceca7842a0aebe5a1e40cbedaedf2eb
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4097325
Reviewed-by: Jamie Madill <jmadill@chromium.org>
Commit-Queue: Roman Lavrov <romanl@google.com>
|
|
e495e7fd
|
2022-10-14T13:52:47
|
|
Redo perf and trace test parameters.
- Moves common argument parsing code into test_util.
- Changes the perf test arg parsing to use the common functions.
- Adds new --use-angle and --use-gl parameters to the trace tests.
- Also adds new --offscreen and --vsync parameters to the traces.
- Removes the now unneeded --enable-all-trace-tests argument.
- Both --arg=value and --arg value work in test suites now.
Now, instead of using --enable-all-trace-tests you can specify the
backend with --use-angle=swiftshader, --offscreen, or combinations
of those parameters. The test names are the same as they were
before, but only the configured tests will run in a session. We
could opt to simplify the test names in later CLs if we want to
simplify the test running. Ideally we'd keep the perf reporting
the same because then we'd keep the time series the same on the
test infra.
This also allows us to split up the trace tests into separate
targets on the bots, which will better allow us to control the
workloads and sampling of the tests. For example:
- angle_perftests becomes
- angle_perftests (microbenchmarks)
- angle_trace_perf_vulkan_tests (traces with vulkan back-end)
- angle_trace_perf_native_tests (traces with system GL)
Bug: angleproject:7755
Change-Id: I537168f3a6de96425dfda05ed98220eff9b19b76
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3956937
Reviewed-by: Yuly Novikov <ynovikov@chromium.org>
Commit-Queue: Jamie Madill <jmadill@chromium.org>
Reviewed-by: Cody Northrop <cnorthrop@google.com>
|
|
916a35dd
|
2022-06-28T14:40:53
|
|
Merge getTempPath and CreateTemporaryFile
Also includes moving these two (and some helpers) to
common/system_utils.h
Bug: angleproject:5095
Change-Id: I7d2fcbd27a67dc13215265ac5b5e8391b15a980b
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3733519
Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org>
Commit-Queue: Eddie Hatfield <eddiehatfield@google.com>
Reviewed-by: Cody Northrop <cnorthrop@google.com>
|
|
|
ca96cba9
|
2022-06-16T11:33:14
|
|
Make ReadEntireFileToString return a std::string.
Note: this makes the method unavailable for export from angle_util,
which is probably why it was designed the way it was in the first
place. However, we import the test utils source file as a static
lib into each test executable and test shared module, so it works.
Bug: angleproject:7404
Change-Id: Ia957268882c2b8529643660d7d4f34d142c0dc43
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3708602
Commit-Queue: Roman Lavrov <romanl@google.com>
Reviewed-by: Roman Lavrov <romanl@google.com>
|
|
e9f9fa17
|
2021-11-09T18:13:15
|
|
Add cpu time measurement to perf tests
Bug: angleproject:6667
Change-Id: I16570a7b51d363dd1c0f35789b985520cb76d488
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3270601
Commit-Queue: Lingfeng Yang <lfy@google.com>
Reviewed-by: Tim Van Patten <timvp@google.com>
Reviewed-by: Jamie Madill <jmadill@chromium.org>
Reviewed-by: Charlie Lao <cclao@google.com>
|
|
|
666fcf38
|
2021-10-08T12:19:15
|
|
Rename functions that overlap with Windows APIs.
Bug: angleproject:6283
Change-Id: Ifcd9ea9e3bf729fd2066178eb9429050b2f10518
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3212894
Reviewed-by: Bruce Dawson <brucedawson@chromium.org>
Reviewed-by: Jamie Madill <jmadill@chromium.org>
Commit-Queue: Jamie Madill <jmadill@chromium.org>
|
|
|
9c05f55a
|
2021-07-15T10:58:10
|
|
Capture/Replay: Add replay state validation.
We can use this to validate traces that have slight differences
when retracing. The valdation works by embedding the "expected"
JSON into a string at the end of each captured frame. The replay
also embeds a callback which fires right before the swap in the
replay harness. The harness then gets the "actual" JSON and runs
a comparison. On a mismatch it calls "diff" externally.
Currently the diff call is hard-coded to work on Linux only.
Note that when running validation it's important to replay on
SwiftShader since that's what we use to retrace.
Bug: angleproject:5133
Change-Id: Icbf0031d07be8bd916607c537dec235f9a512c43
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3066008
Commit-Queue: Jamie Madill <jmadill@chromium.org>
Reviewed-by: Tim Van Patten <timvp@google.com>
Reviewed-by: Cody Northrop <cnorthrop@google.com>
|
|
72d2bd0c
|
2021-06-21T09:59:21
|
|
Allow capturing process stdout and stderr interleaved
The test utils are enhanced to allow redirecting stderr to stdout. This
is in preparation for a change that makes the test runner capture stderr
together with stdout. Currently, on failure logs originating from
UNIMPLEMENTED() and other such macros are not captured.
Bug: angleproject:6077
Change-Id: I7a3c6c4732a59dac3ff0cc20a7835d5ed6f0f22e
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2976183
Reviewed-by: Jamie Madill <jmadill@chromium.org>
Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org>
|
|
cb8903b1
|
2020-12-08T01:08:00
|
|
Metal: Ignore OS's internal shader cache when testing.
Internal shader cache caused timeout in some dEQP tests.
Work-around: ignore the cache by hooking fopen function and return null
when the cache related files are accessed.
Bug: angleproject:5354
Change-Id: I12ca228540925e67454bf24ce1ba83d703882c87
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2580918
Commit-Queue: Le Hoang Quyen <le.hoang.q@gmail.com>
Reviewed-by: Jamie Madill <jmadill@chromium.org>
Reviewed-by: Kenneth Russell <kbr@chromium.org>
|
|
|
11e26fa0
|
2020-06-11T13:32:10
|
|
Allow running Samples against native GL.
Useful for perf testing/comparison.
Bug: angleproject:4729
Change-Id: Ic46424570dcef0a30d506962f546910ba7440595
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2241620
Reviewed-by: Jonah Ryan-Davis <jonahr@google.com>
Reviewed-by: Geoff Lang <geofflang@chromium.org>
Commit-Queue: Jamie Madill <jmadill@chromium.org>
|
|
78f2e4b8
|
2020-03-24T16:21:40
|
|
Add gles1 covegl test compile
Covegl test is hard to suit gtest, so just compile it without gtest
(angle_gles1_covegl_no_gtest), we can get the test result by exit code.
Bug: angleproject:2303
Change-Id: I67a930c91f23c02e72022329c712a0749e4a903c
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2126587
Commit-Queue: Geoff Lang <geofflang@chromium.org>
Reviewed-by: Geoff Lang <geofflang@chromium.org>
|
|
|
9190f49c
|
2019-12-16T16:03:57
|
|
GN: Make new angle_test_utils target.
This source set target lets other targets import the test utils without
needing to export them. They get built into angle_util. They also get
compiled into the various tests and samples. The change also fixes
export issues.
Moves some of the GN logic into the util/ subfolder.
Bug: angleproject:3162
Bug: chromium:1030192
Change-Id: If99d201092ad8541c0de60b3bd893ac9b5875270
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/1968259
Reviewed-by: Yuly Novikov <ynovikov@chromium.org>
Reviewed-by: Jamie Madill <jmadill@chromium.org>
Commit-Queue: Jamie Madill <jmadill@chromium.org>
|
|
|
95a736bd
|
2019-10-09T10:26:25
|
|
Reland "Add more test_utils functions."
This is a reland of 5fcfcea4a9379633a83a67fc1d94938cb31f2a9c
Re-land uses static linking with angle_util. The root cause
of the CFI error wasn't solved. Static linking works around
the problem by not using any export rules.
Second re-land fixes missing imports for libEGL and libGLESv2
static varieties.
Original change's description:
> Add more test_utils functions.
>
> Includes methods for creating temporary files, deleting files, and
> reading files into a string. Also renames GetPathSeparator to mention
> it's only used for environment variables. Includes a new virtual type
> angle::Process that will be used to implement cross-platform async
> Process launching for tests. Also includes a way to specify a custom
> crash handler callback.
>
> Also adds a few unit tests for the new functionality. They are disabled
> on Android because the functions are not needed by the new test runner.
>
> Bug: angleproject:3162
> Change-Id: I3e2c2e9837608884c98379fa0f78c9ffbe158d73
> Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/1821940
> Commit-Queue: Jamie Madill <jmadill@chromium.org>
> Reviewed-by: Jonah Ryan-Davis <jonahr@google.com>
Bug: chromium:1015810
Bug: angleproject:3162
Change-Id: I2a18b819b0f91df610ad12ffedea2b38349fe7cf
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/1879859
Commit-Queue: Jamie Madill <jmadill@chromium.org>
Reviewed-by: Yuly Novikov <ynovikov@chromium.org>
|
|
|
f939cb63
|
2019-10-20T21:44:36
|
|
Revert "Reland "Add more test_utils functions.""
This reverts commit 7923e234bc458eda5936b8cb4f09fa19e28c0452.
Reason for revert: Fails compilation on rollers:
[2097/6048] SOLINK ./lib_angle_unittests__library.cr.so
FAILED: lib_angle_unittests__library.cr.so lib_angle_unittests__library.cr.so.TOC lib.unstripped/lib_angle_unittests__library.cr.so
python "../../build/toolchain/gcc_solink_wrapper.py" --readelf="../../third_party/android_ndk/toolch...(too long)
ld.lld: error: undefined symbol: glCreateShader
>>> referenced by shader_utils.cpp
>>> angle_util_static/shader_utils.o:(CompileShader(unsigned int, char const*)) in archive obj/third_party/angle/libangle_util_static.a
ld.lld: error: undefined symbol: glShaderSource
Original change's description:
> Reland "Add more test_utils functions."
>
> This is a reland of 5fcfcea4a9379633a83a67fc1d94938cb31f2a9c
>
> Re-land uses static linking with angle_util. The root cause
> of the CFI error wasn't solved. Static linking works around
> the problem by not using any export rules.
>
> Original change's description:
> > Add more test_utils functions.
> >
> > Includes methods for creating temporary files, deleting files, and
> > reading files into a string. Also renames GetPathSeparator to mention
> > it's only used for environment variables. Includes a new virtual type
> > angle::Process that will be used to implement cross-platform async
> > Process launching for tests. Also includes a way to specify a custom
> > crash handler callback.
> >
> > Also adds a few unit tests for the new functionality. They are disabled
> > on Android because the functions are not needed by the new test runner.
> >
> > Bug: angleproject:3162
> > Change-Id: I3e2c2e9837608884c98379fa0f78c9ffbe158d73
> > Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/1821940
> > Commit-Queue: Jamie Madill <jmadill@chromium.org>
> > Reviewed-by: Jonah Ryan-Davis <jonahr@google.com>
>
> Bug: chromium:1015810
> Bug: angleproject:3162
> Change-Id: I6a2c1e7b585a13ca846759f32da0777c00d7f7e6
> Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/1869541
> Commit-Queue: Jamie Madill <jmadill@chromium.org>
> Reviewed-by: Jamie Madill <jmadill@chromium.org>
TBR=ynovikov@chromium.org,jonahr@google.com,jmadill@chromium.org
Change-Id: I975b2214411906cb981bffa04fa50e0a65ff8b4e
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: chromium:1015810, angleproject:3162
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/1870811
Reviewed-by: Jamie Madill <jmadill@chromium.org>
Commit-Queue: Jamie Madill <jmadill@chromium.org>
|
|
|
7923e234
|
2019-10-09T10:26:25
|
|
Reland "Add more test_utils functions."
This is a reland of 5fcfcea4a9379633a83a67fc1d94938cb31f2a9c
Re-land uses static linking with angle_util. The root cause
of the CFI error wasn't solved. Static linking works around
the problem by not using any export rules.
Original change's description:
> Add more test_utils functions.
>
> Includes methods for creating temporary files, deleting files, and
> reading files into a string. Also renames GetPathSeparator to mention
> it's only used for environment variables. Includes a new virtual type
> angle::Process that will be used to implement cross-platform async
> Process launching for tests. Also includes a way to specify a custom
> crash handler callback.
>
> Also adds a few unit tests for the new functionality. They are disabled
> on Android because the functions are not needed by the new test runner.
>
> Bug: angleproject:3162
> Change-Id: I3e2c2e9837608884c98379fa0f78c9ffbe158d73
> Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/1821940
> Commit-Queue: Jamie Madill <jmadill@chromium.org>
> Reviewed-by: Jonah Ryan-Davis <jonahr@google.com>
Bug: chromium:1015810
Bug: angleproject:3162
Change-Id: I6a2c1e7b585a13ca846759f32da0777c00d7f7e6
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/1869541
Commit-Queue: Jamie Madill <jmadill@chromium.org>
Reviewed-by: Jamie Madill <jmadill@chromium.org>
|
|
0cb09633
|
2019-10-18T15:50:10
|
|
Revert "Add more test_utils functions."
This reverts commit 5fcfcea4a9379633a83a67fc1d94938cb31f2a9c.
Reason for revert:
1 Test Suite(s) failed. angle_unittests failed because of:
TestUtils.RunApp on Linux CFI
https://ci.chromium.org/p/chromium/builders/ci/Linux%20CFI
Original change's description:
> Add more test_utils functions.
>
> Includes methods for creating temporary files, deleting files, and
> reading files into a string. Also renames GetPathSeparator to mention
> it's only used for environment variables. Includes a new virtual type
> angle::Process that will be used to implement cross-platform async
> Process launching for tests. Also includes a way to specify a custom
> crash handler callback.
>
> Also adds a few unit tests for the new functionality. They are disabled
> on Android because the functions are not needed by the new test runner.
>
> Bug: angleproject:3162
> Change-Id: I3e2c2e9837608884c98379fa0f78c9ffbe158d73
> Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/1821940
> Commit-Queue: Jamie Madill <jmadill@chromium.org>
> Reviewed-by: Jonah Ryan-Davis <jonahr@google.com>
TBR=ynovikov@chromium.org,jonahr@google.com,jmadill@chromium.org
Change-Id: I1441bfbae31712f72b4aebeeea9cd711c3975a5d
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: angleproject:3162
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/1869254
Reviewed-by: Jamie Madill <jmadill@chromium.org>
Commit-Queue: Jamie Madill <jmadill@chromium.org>
|
|
|
5fcfcea4
|
2019-10-09T10:26:25
|
|
Add more test_utils functions.
Includes methods for creating temporary files, deleting files, and
reading files into a string. Also renames GetPathSeparator to mention
it's only used for environment variables. Includes a new virtual type
angle::Process that will be used to implement cross-platform async
Process launching for tests. Also includes a way to specify a custom
crash handler callback.
Also adds a few unit tests for the new functionality. They are disabled
on Android because the functions are not needed by the new test runner.
Bug: angleproject:3162
Change-Id: I3e2c2e9837608884c98379fa0f78c9ffbe158d73
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/1821940
Commit-Queue: Jamie Madill <jmadill@chromium.org>
Reviewed-by: Jonah Ryan-Davis <jonahr@google.com>
|
|
|
57b37b6b
|
2019-09-25T18:29:28
|
|
Rename util/system_utils to util/test_utils.
This removes a GN naming conflict between util/system_utils and
common/system_utils. This conflict was preventing us from adding
unit tests to utils' version of system_utils. Since these functions are
only useful to tests and samples rename them test_utils for simplicity.
Will enable further development of ANGLE's standalone testing harness.
Bug: angleproject:3162
Change-Id: I9e34fb69f96c5de6dc2453fce4148a0f285e15ed
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/1825268
Reviewed-by: Jonah Ryan-Davis <jonahr@google.com>
Reviewed-by: Yuly Novikov <ynovikov@chromium.org>
Commit-Queue: Jamie Madill <jmadill@chromium.org>
|