src/tests/deqp_support/angle_deqp_libtester_main.cpp


Log

Author Commit Date CI Message
Shahbaz Youssefi 62628799 2021-07-21T22:55:22 Vulkan: SPIR-V Gen: Support dEQP A temporary condition for direct SPIR-V generation is added to test expectations while this work is in progress, so test suites can be partially enabled. 32 tests are currently failing. Bug: angleproject:4889 Change-Id: Icf0f6ef80d30fdb7564d8ecf928e67ca58ace86f Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3042556 Reviewed-by: Jamie Madill <jmadill@chromium.org> Reviewed-by: Tim Van Patten <timvp@google.com> Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org>
Shahbaz Youssefi 7df7fc7f 2021-06-20T00:05:28 Tests: Add support for --renderdoc This change adds support for a new flag namely `--renderdoc` to end2end and deqp tests. With this flag, each test automatically starts and ends a frame capture in renderdoc, working around an issue where renderdoc refuses to capture a test frame that doesn't start or end with a swap. With end2end tests, the capture starts before test set up, and ends after test tear down. With deqp tests, it starts before init, ends and restarts after each test iteration and ends after deinit. Bug: angleproject:6072 Change-Id: Ib41b816aff121bf922d9147044cc363c33a62181 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2971835 Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org> Reviewed-by: Cody Northrop <cnorthrop@google.com> Reviewed-by: Jamie Madill <jmadill@chromium.org>
Jamie Madill 85efb9d5 2021-01-26T14:39:19 Log dEQP QPA files as test artifacts. This adds artifact output to the test runner. We add a fake test at the start of a test run that owns the artifacts. Bug: angleproject:5236 Change-Id: Ice8001bf1f2aafbd8123fee76e0e7fcc3e5a8a0c Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2657535 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org> Reviewed-by: Mohan Maiya <m.maiya@samsung.com> Reviewed-by: Yuly Novikov <ynovikov@chromium.org>
Jamie Madill 00643e7d 2020-07-22T20:58:49 Use Android API to get storage path. This pipes through to a system call via JNI. This will allow Chromium changes to land that were prevented because ANGLE hard-coded some parts of this path. This in turn will allow us to more easily override these paths for changes needed for Android R support. Bug: chromium:1094062 Change-Id: I20d75b8ee40d418ba5c057f618640ef896248299 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2315483 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Tim Van Patten <timvp@google.com> Reviewed-by: Jonah Ryan-Davis <jonahr@google.com>
Shahbaz Youssefi 09932e04 2020-10-29T00:31:49 Vulkan: Support emulated pre-rotation in dEQP testing Similarly to end2end tests, the window dimensions are swapped with emulated prerotation at 90 and 270 degrees, while maintaining to the application that dimensions are as requested. The following new command line argument can be used to select an emulated prerotation: --emulated-pre-rotation=90 --emulated-pre-rotation=180 --emulated-pre-rotation=270 For example: $ ./angle_deqp_gles2_tests --use-angle=vulkan \ --deqp-case=*draw* \ --emulated-pre-rotation=270 Additionally, the deqp test expectations can be marked with the following new tags to add suppressions for failing tests under prerotation: PREROTATION PREROTATION90 PREROTATION180 PREROTATION270 Bug: angleproject:4901 Change-Id: I7a68c1a1e7da4366cde981469c589d8d900c40c5 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2506810 Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org> Reviewed-by: Yuly Novikov <ynovikov@chromium.org> Reviewed-by: Jamie Madill <jmadill@chromium.org>
Jamie Madill 0df92012 2020-06-03T17:08:43 Rename Platform.h to PlatformMethods.h. "platform.h" is too common a name and causes headers to be included incorrectly. Disambiguate the header using a more specific name. Solves a problem that came up with the GLES 1 tests and the standalone test harness. Bug: angleproject:3162 Change-Id: I88229a2c9407e0db57f5beee44daa11a4075f700 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2229065 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Yuly Novikov <ynovikov@chromium.org>
Courtney Goeltzenleuchter febdcf51 2019-10-28T10:54:03 Roll dEQP to VK-GL-CTS Switch from Android dEQP to Khronos GL CTS. We now mirror from the public Khronos Github github.com/KhronosGroup/VK-GL-CTS This includes a number of text fixes for GLES 2.0 conformance. Bug: angleproject:3914 Bug: angleproject:1656 Change-Id: I4cccd33e47752742cb989127c5883c0afd075b9e Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/1879579 Reviewed-by: Jamie Madill <jmadill@chromium.org> Commit-Queue: Courtney Goeltzenleuchter <courtneygo@google.com>
Jamie Madill 57b37b6b 2019-09-25T18:29:28 Rename util/system_utils to util/test_utils. This removes a GN naming conflict between util/system_utils and common/system_utils. This conflict was preventing us from adding unit tests to utils' version of system_utils. Since these functions are only useful to tests and samples rename them test_utils for simplicity. Will enable further development of ANGLE's standalone testing harness. Bug: angleproject:3162 Change-Id: I9e34fb69f96c5de6dc2453fce4148a0f285e15ed Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/1825268 Reviewed-by: Jonah Ryan-Davis <jonahr@google.com> Reviewed-by: Yuly Novikov <ynovikov@chromium.org> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Jamie Madill 7a026133 2019-05-20T10:34:08 Fix Android data dir for KHR dEQP tests. Also includes a fix for case list paths on developer machines. Also includes a test expectation update for GLES2. Bug: angleproject:3353 Change-Id: Idb01eb8d806067739253a6676a6d2d8ee1199929 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/1616853 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Yuly Novikov <ynovikov@chromium.org>
Jamie Madill cfffc138 2019-05-16T11:35:01 Add KHR-GLES dEQP tests. This reorganizes the dEQP build targets to allow for better reuse with the new KHR test set. It also adds a single new test target called "angle_deqp_khr_tests" that houses all the new test sets. The new test targets can be compiled now but cannot yet be executed. That new functionality will follow in a subsequent CL. Along with test expectations and integration. Uses a new template type in the build to conslidate GN customization. Adds the GLES 2.0 and 3.0 tests for now. We can follow up with additional work for the GLES 3.1+ tests. Also adds test expectations. Bug: angleproject:3353 Change-Id: Ib0673e97f679c8aa46e27dfa616845a2014a75fe Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/1604070 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Yuly Novikov <ynovikov@chromium.org>
Jamie Madill b4ca09ac 2019-05-15T15:23:12 Add IsDirectory helper function to system_utils. This was previously a method in dEQP only. Cleanup change only. Bug: angleproject:3353 Change-Id: I9ac58ab52516f75efbb08bddb39466a40c07cbb8 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/1613892 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Courtney Goeltzenleuchter <courtneygo@google.com> Reviewed-by: Yuly Novikov <ynovikov@chromium.org>
Jamie Madill ba319ba3 2018-12-29T10:29:33 Re-land "Load entry points dynamically in tests and samples." Fixes the Android/ChromeOS/Fuchsia builds by using consistent EGL headers. This CL adds a dynamic loader generator based on XML files. It also refactors the entry point generation script to move the XML parsing into a helper class. Additionally this includes a new GLES 1.0 base header. The new header allows for function pointer types and hiding prototypes. All tests and samples now load ANGLE dynamically. In the future this will be extended to load entry points from the driver directly when possible. This will allow us to perform more accurate A/B testing. The new build configuration leads to some tests having more warnings applied. The CL includes fixes for the new warnings. Bug: angleproject:2995 Change-Id: I5a8772f41a0f89570b3736b785f44b7de1539b57 Reviewed-on: https://chromium-review.googlesource.com/c/1392382 Reviewed-by: Jamie Madill <jmadill@chromium.org> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Yuly Novikov 9f088621 2018-12-29T20:46:15 Revert "Load entry points dynamically in tests and samples." This reverts commit 03923558a7103827ffec6a4d2a1453ed91f01c6f. Reason for revert: fails compilation on Android, ChromeOS and Fuchsia during roll https://chromium-review.googlesource.com/c/chromium/src/+/1392624 Original change's description: > Load entry points dynamically in tests and samples. > > This CL adds a dynamic loader generator based on XML files. It also > refactors the entry point generation script to move the XML parsing > into a helper class. > > Additionally this includes a new GLES 1.0 base header. The new > header allows for function pointer types and hiding prototypes. > > All tests and samples now load ANGLE dynamically. In the future this > will be extended to load entry points from the driver directly when > possible. This will allow us to perform more accurate A/B testing. > > The new build configuration leads to some tests having more warnings > applied. The CL includes fixes for the new warnings. > > Bug: angleproject:2995 > Change-Id: I6726d4163f7a6e54d2482f094c0a952f59702a05 > Reviewed-on: https://chromium-review.googlesource.com/c/1359516 > Commit-Queue: Jamie Madill <jmadill@chromium.org> > Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org> TBR=ynovikov@chromium.org,jmadill@chromium.org,syoussefi@chromium.org Change-Id: I902bec2d733c2b879be29c02ab52a0b7d4eaa077 No-Presubmit: true No-Tree-Checks: true No-Try: true Bug: angleproject:2995 Reviewed-on: https://chromium-review.googlesource.com/c/1392381 Reviewed-by: Yuly Novikov <ynovikov@chromium.org> Commit-Queue: Yuly Novikov <ynovikov@chromium.org>
Jamie Madill 03923558 2018-12-29T10:29:33 Load entry points dynamically in tests and samples. This CL adds a dynamic loader generator based on XML files. It also refactors the entry point generation script to move the XML parsing into a helper class. Additionally this includes a new GLES 1.0 base header. The new header allows for function pointer types and hiding prototypes. All tests and samples now load ANGLE dynamically. In the future this will be extended to load entry points from the driver directly when possible. This will allow us to perform more accurate A/B testing. The new build configuration leads to some tests having more warnings applied. The CL includes fixes for the new warnings. Bug: angleproject:2995 Change-Id: I6726d4163f7a6e54d2482f094c0a952f59702a05 Reviewed-on: https://chromium-review.googlesource.com/c/1359516 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org>
Jamie Madill b980c563 2018-11-27T11:34:27 Reformat all cpp and h files. This applies git cl format --full to all ANGLE sources. Bug: angleproject:2986 Change-Id: Ib504e618c1589332a37e97696cdc3515d739308f Reviewed-on: https://chromium-review.googlesource.com/c/1351367 Reviewed-by: Jamie Madill <jmadill@chromium.org> Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org>
Jamie Madill a2f043d8 2018-07-10T17:21:20 dEQP: Implement platform error handling. Also downgrades several Vulkan UNIMPLEMENTED() to WARN(). Also downgrades a couple D3D-specific errors to warnings. Also downgrades an undefined behaviour integer clear error to warning. Also includes suppressions for failing D3D11 ES 3.1 SSBO tests. Also includes suppressions for failing Android GLES format tests. Also includes suppressions for failing Android Vulkan buffer tests. Bug: angleproject:2552 Bug: angleproject:2567 Bug: angleproject:1951 Bug: angleproject:2405 Change-Id: Ie619085021d42012cd578b669f7ff4252ca41a58 Reviewed-on: https://chromium-review.googlesource.com/1062791 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Yuly Novikov <ynovikov@chromium.org>
Edward Lemur cbab2750 2018-05-31T18:49:31 DEPS.chromium: Sync dEQP to ANGLE (not Chromium) third_party. Bug: 782846 Change-Id: I88af578dfd27fb69751138d9b263a148da9ecf23 Reviewed-on: https://chromium-review.googlesource.com/1081113 Reviewed-by: Jamie Madill <jmadill@chromium.org> Reviewed-by: Yuly Novikov <ynovikov@chromium.org> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Jamie Madill 4428d395 2018-05-17T15:16:27 dEQP: Abort gtest run on exception. Prevents massive infra failures when the tests are all broken. This also gives the GoogleTest harness the ability to handle different test result types. Bug: angleproject:2552 Change-Id: Id9832e8557e2ee4e6a248a27729f67a81bf2d830 Reviewed-on: https://chromium-review.googlesource.com/1064014 Reviewed-by: Yuly Novikov <ynovikov@chromium.org> Reviewed-by: Geoff Lang <geofflang@chromium.org> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Geoff Lang baa60994 2018-05-14T12:46:17 DEQP: Print not supported messages from tests. BUG=angleproject:2546 Change-Id: I74b376c07f94d77abe28129ff5e97bd19a26d0d0 Reviewed-on: https://chromium-review.googlesource.com/1057831 Reviewed-by: Jamie Madill <jmadill@chromium.org> Commit-Queue: Geoff Lang <geofflang@chromium.org>
Jamie Madill 23c107e8 2018-01-30T09:54:39 Roll dEQP 455d82c60b0...5bc346ba2. Inclues several new ES 3.1 and EGL test sets. https://android.googlesource.com/platform/external/deqp/+log/455d82c60b0..5bc346ba2d5 Bug: angleproject:2319 Bug: angleproject:1961 Bug: angleproject:1028 Bug: angleproject:2320 Bug: angleproject:2321 Bug: angleproject:2329 Bug: angleproject:2330 Change-Id: I26bfb11a41073e94ecc75d20b07055eb4e946d43 Reviewed-on: https://chromium-review.googlesource.com/703159 Reviewed-by: Geoff Lang <geofflang@chromium.org>
Jamie Madill e5c53e3e 2016-11-07T12:22:48 dEQP: Add command-line parsing of back-end. Also use the requested renderer to initialize expectations. BUG=angleproject:1442 Change-Id: Idf54072dac5f7ad9deea70e97d65e36a6e883b1c Reviewed-on: https://chromium-review.googlesource.com/407802 Commit-Queue: Geoff Lang <geofflang@chromium.org> Reviewed-by: Corentin Wallez <cwallez@chromium.org>
Jamie Madill d08163d7 2016-09-01T13:56:24 Fix standalone build. BUG=angleproject:1471 Change-Id: I52aadeb58f75d5b944421bd0cd4a5dddead3f6d1 Reviewed-on: https://chromium-review.googlesource.com/379916 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Yuly Novikov <ynovikov@chromium.org> Reviewed-by: Jamie Madill <jmadill@chromium.org>
Yuly Novikov 4628f386 2016-08-13T14:13:16 Add dEQP search paths on Android devices Including: test expectations and list, dEQP data and test results. Hardcode chromium_tests_root for now, to avoid adding a new API to pass the path from test runner to ANGLE. BUG=angleproject:1471 Change-Id: I654fa4727c436d743b3d6344ad17cb030934cde2 Reviewed-on: https://chromium-review.googlesource.com/368984 Commit-Queue: Yuly Novikov <ynovikov@chromium.org> Reviewed-by: Jamie Madill <jmadill@chromium.org>
Yuly Novikov ca05a081 2016-08-12T14:46:30 Compile deqp_gles2 on Android Add some missing functions, modify preprocessor conditions and build files to include some other missing functions when building for Android. BUG=angleproject:1471 Change-Id: Iadc0a0b9fed2444b8bc9a894ee65c8b66ea7f3c9 Reviewed-on: https://chromium-review.googlesource.com/368982 Commit-Queue: Yuly Novikov <ynovikov@chromium.org> Reviewed-by: Corentin Wallez <cwallez@chromium.org> Reviewed-by: Jamie Madill <jmadill@chromium.org>
Frank Henigman bb51c35c 2016-03-24T16:27:37 Insert slash when concatenating paths. BUG=angleproject:1343 Change-Id: I04180135dcd3aac2d40dc3f6d32b1aad362c69d1 Reviewed-on: https://chromium-review.googlesource.com/334920 Reviewed-by: Jamie Madill <jmadill@chromium.org> Reviewed-by: Geoff Lang <geofflang@chromium.org> Commit-Queue: Frank Henigman <fjhenigman@chromium.org>
Corentin Wallez 6ad306cb 2016-02-11T17:06:41 dEQP: search for the data directory relative to the exe path. BUG=580045 Change-Id: I58bfa9ab6766eaae756b3c0f4d28f545e86d543b Reviewed-on: https://chromium-review.googlesource.com/327306 Reviewed-by: Jamie Madill <jmadill@chromium.org> Reviewed-by: Geoff Lang <geofflang@chromium.org> Reviewed-by: Corentin Wallez <cwallez@chromium.org> Commit-Queue: Corentin Wallez <cwallez@chromium.org>
Austin Kinross d0925fd2 2015-10-23T10:22:01 Add "../../../third_party/deqp/src/data" to dEQP data directories Change-Id: I9bd4127d2eb24657e45b3a3b4668794d86b9d0e3 Reviewed-on: https://chromium-review.googlesource.com/308481 Tested-by: Austin Kinross <aukinros@microsoft.com> Reviewed-by: Jamie Madill <jmadill@chromium.org>
Corentin Wallez 68a9d748 2015-09-15T08:21:10 Compile the dEQP test targets on Mac BUG=angleproject:891 Change-Id: I4bc2817106c8170684a76972badd38c02c0abf0e Reviewed-on: https://chromium-review.googlesource.com/299852 Reviewed-by: Geoff Lang <geofflang@chromium.org> Tested-by: Corentin Wallez <cwallez@chromium.org>
Austin Kinross cc54ab36 2015-09-02T10:27:28 Generate tests.sln for WinRT, and make it include dEQP projects Change-Id: I34284965f7b4f9d11aa7bab263292d4f41312d04 Reviewed-on: https://chromium-review.googlesource.com/297307 Tested-by: Austin Kinross <aukinros@microsoft.com> Reviewed-by: Jamie Madill <jmadill@chromium.org> Reviewed-by: Geoff Lang <geofflang@chromium.org>
Corentin Wallez c491e0fb 2015-07-02T10:41:11 Fix command line being ignored in non-gtest dEQP tests BUG=angleproject:1051 Change-Id: If10abaea31ad29af9f8d438403f67b9dfbfd7e45 Reviewed-on: https://chromium-review.googlesource.com/283167 Reviewed-by: Jamie Madill <jmadill@chromium.org> Tested-by: Corentin Wallez <cwallez@chromium.org>
Corentin Wallez a55e4b7a 2015-06-09T13:30:25 Enable compilation of dEQP tests on Linux BUG=angleproject:1051 Change-Id: I158e59ca581b580825848e831d05064cdcc30d96 Reviewed-on: https://chromium-review.googlesource.com/276202 Reviewed-by: Jamie Madill <jmadill@chromium.org> Tested-by: Corentin Wallez <cwallez@chromium.org>
Jamie Madill 4d06a24c 2015-06-26T11:12:24 dEQP: Extend GoogleTest support. *re-land after rebase on top of Chrome build fix* This patch will enable a Chromium-side CL to build dEQP/ANGLE with GoogleTest. BUG=angleproject:998 BUG=500736 Change-Id: Id02c66a260b0a7bab3de9cda624fe097322b2d3c Reviewed-on: https://chromium-review.googlesource.com/281824 Tested-by: Jamie Madill <jmadill@chromium.org> Reviewed-by: Corentin Wallez <cwallez@google.com> Reviewed-by: Kenneth Russell <kbr@chromium.org>
Jamie Madill 57cdc69c 2015-06-23T21:03:06 Revert "dEQP: Extend GoogleTest support." Causing runhooks failures: gyp: Undefined variable deqp_path in C:\b\build\slave\GPU_Win_Builder\build\src\third_party\angle\src\tests\tests.gyp BUG=angleproject:998 BUG=500736 This reverts commit 8edd75d1bb6680e7c2018ece9f5c81e3b87bf938. Change-Id: I8cd83bb5871dea81abea61554b182825145298e6 Reviewed-on: https://chromium-review.googlesource.com/281208 Reviewed-by: Jamie Madill <jmadill@chromium.org> Tested-by: Jamie Madill <jmadill@chromium.org>
Jamie Madill 8edd75d1 2015-06-22T15:35:53 dEQP: Extend GoogleTest support. This patch will enable a Chromium-side CL to build dEQP/ANGLE with GoogleTest. BUG=angleproject:998 BUG=500736 Change-Id: Ie480e3319c1bd77dd1de2c964f78ef97297fcfe9 Reviewed-on: https://chromium-review.googlesource.com/280856 Tested-by: Jamie Madill <jmadill@chromium.org> Reviewed-by: Corentin Wallez <cwallez@chromium.org> Reviewed-by: Zhenyao Mo <zmo@chromium.org>
Corentin Wallez 44b07e94 2015-06-09T18:00:46 Fix compilation of libtester_main on Linux Replace usage of std::exception by std::runtime_error BUG=angleproject:892 Change-Id: I214fe6a93944dcfb39b0633f46caf3b1187bd5d3 Reviewed-on: https://chromium-review.googlesource.com/275815 Reviewed-by: Geoff Lang <geofflang@chromium.org> Reviewed-by: Jamie Madill <jmadill@chromium.org> Tested-by: Corentin Wallez <cwallez@chromium.org>
Jamie Madill df785dfa 2015-06-09T09:55:30 dEQP: Add GoogleTest integration. Use value-parameterized tests, loading a full test list from case files. BUG=angleproject:998 Change-Id: Iff65a2d722e9d105e22649b17a18cc74dccea2b1 Reviewed-on: https://chromium-review.googlesource.com/274421 Reviewed-by: Kenneth Russell <kbr@chromium.org> Tested-by: Jamie Madill <jmadill@chromium.org> Reviewed-by: Geoff Lang <geofflang@chromium.org>
Jamie Madill 3cd438d1 2015-03-17T11:25:27 Add dEQP tests. We integrate dEQP as a console application and a shared library which runs all of the test logic. Using a shared library lets us compile dEQP with all the specific compiler options it needs, without conflicting with the compile settings in ANGLE proper. Currently we only support Windows D3D11, ES 2 and 3. We can add other targets in the future. We also have a few bugs preventing us from running the test suite in full. We run into infinite loop problems in some shader tests, and have crashes or UNIMPLEMENTED in others. BUG=angleproject:901 Change-Id: Ib6fe66041a6fe547eb2cba497c52de7fd080d667 Reviewed-on: https://chromium-review.googlesource.com/238084 Reviewed-by: Kenneth Russell <kbr@chromium.org> Reviewed-by: Olli Etuaho <oetuaho@nvidia.com> Tested-by: Jamie Madill <jmadill@chromium.org> Reviewed-by: Geoff Lang <geofflang@chromium.org>