src/tests/deqp_support/angle_deqp_gtest.cpp


Log

Author Commit Date CI Message
Jonah Ryan-Davis 518c679d 2019-03-25T16:25:10 Clean up test_expectations to merge with angle style/convention. Can rename/clean up the library to move from chrome style to angle style. Can also start pulling out chrome-specific definitions. More work to be done when moving gpu_info to use Angle's SystemInfo. Bug: angleproject:2677 Change-Id: I75fd4446aa4be85fba590cd00cd3a213a0b3752e Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/1538677 Commit-Queue: Jonah Ryan-Davis <jonahr@google.com> Reviewed-by: Geoff Lang <geofflang@chromium.org>
Jonah Ryan-Davis 0449a902 2019-03-22T16:03:54 Move gpu_test_expectations from third_party into ANGLE. 1. Copy the code from src/tests/third_party/gpu_test_expectations to src/test/test_expectations, rename .cc files to .cpp. Put these in a new static library and update dEQP to link against it in src/tests/BUILD.gn. 2. Merge the code in angle_config.h into the rest of the expectations parser, this code was added so that the rest of the parser would compile in ANGLE's tree with minimal modification. Still need to follow up with the third step to close the issue. Bug: angleproject:2677 Change-Id: Icf09b4eeed83a6d09b1964ad2adcfa85cabb4b63 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/1536312 Commit-Queue: Jonah Ryan-Davis <jonahr@google.com> Reviewed-by: Geoff Lang <geofflang@chromium.org>
Jamie Madill 4ae2c12f 2019-03-23T18:07:26 Use Khronos master lists for dEQP tests. There are an number of valid tests that are excluded from the Android test lists. Likely some older and low spec devices weren't passing these tests and for historical reasons they were omitted from the Android CTS. ANGLE however needs to pass these tests to be conformant and up until now we weren't testing them on our bots. The EGL tests still use the same Android version of the test list. It seems the Khronos version just lists wildcards. ANGLE needs an explicit test list. Bug: angleproject:3242 Change-Id: Ie0aa2aa94689be0a40e7cc39e05f0042c3f03f5d Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/1520986 Reviewed-by: Tim Van Patten <timvp@google.com> Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Victor Costan ffd39978 2019-02-20T10:45:24 test: Replace _TEST_CASE_ with _TEST_SUITE_. Googletest is (at last) converging with industry-standard terminology [1]. We previously called test suites "test cases", which was rather confusing for folks coming from any other testing framework. Chrome now has a googletest version that supports _TEST_SUITE_ macros instead of _TEST_CASE_, so this CL cleans up some of the outdated usage. [1] https://github.com/google/googletest/blob/master/googletest/docs/primer.md#beware-of-the-nomenclature Bug: chromium:925652 Change-Id: Ia0deec0bc4216ef1adabc33985a7cbda89682608 Reviewed-on: https://chromium-review.googlesource.com/c/1477418 Reviewed-by: Geoff Lang <geofflang@chromium.org> Commit-Queue: Victor Costan <pwnall@chromium.org>
Jamie Madill ba319ba3 2018-12-29T10:29:33 Re-land "Load entry points dynamically in tests and samples." Fixes the Android/ChromeOS/Fuchsia builds by using consistent EGL headers. This CL adds a dynamic loader generator based on XML files. It also refactors the entry point generation script to move the XML parsing into a helper class. Additionally this includes a new GLES 1.0 base header. The new header allows for function pointer types and hiding prototypes. All tests and samples now load ANGLE dynamically. In the future this will be extended to load entry points from the driver directly when possible. This will allow us to perform more accurate A/B testing. The new build configuration leads to some tests having more warnings applied. The CL includes fixes for the new warnings. Bug: angleproject:2995 Change-Id: I5a8772f41a0f89570b3736b785f44b7de1539b57 Reviewed-on: https://chromium-review.googlesource.com/c/1392382 Reviewed-by: Jamie Madill <jmadill@chromium.org> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Yuly Novikov 9f088621 2018-12-29T20:46:15 Revert "Load entry points dynamically in tests and samples." This reverts commit 03923558a7103827ffec6a4d2a1453ed91f01c6f. Reason for revert: fails compilation on Android, ChromeOS and Fuchsia during roll https://chromium-review.googlesource.com/c/chromium/src/+/1392624 Original change's description: > Load entry points dynamically in tests and samples. > > This CL adds a dynamic loader generator based on XML files. It also > refactors the entry point generation script to move the XML parsing > into a helper class. > > Additionally this includes a new GLES 1.0 base header. The new > header allows for function pointer types and hiding prototypes. > > All tests and samples now load ANGLE dynamically. In the future this > will be extended to load entry points from the driver directly when > possible. This will allow us to perform more accurate A/B testing. > > The new build configuration leads to some tests having more warnings > applied. The CL includes fixes for the new warnings. > > Bug: angleproject:2995 > Change-Id: I6726d4163f7a6e54d2482f094c0a952f59702a05 > Reviewed-on: https://chromium-review.googlesource.com/c/1359516 > Commit-Queue: Jamie Madill <jmadill@chromium.org> > Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org> TBR=ynovikov@chromium.org,jmadill@chromium.org,syoussefi@chromium.org Change-Id: I902bec2d733c2b879be29c02ab52a0b7d4eaa077 No-Presubmit: true No-Tree-Checks: true No-Try: true Bug: angleproject:2995 Reviewed-on: https://chromium-review.googlesource.com/c/1392381 Reviewed-by: Yuly Novikov <ynovikov@chromium.org> Commit-Queue: Yuly Novikov <ynovikov@chromium.org>
Jamie Madill 03923558 2018-12-29T10:29:33 Load entry points dynamically in tests and samples. This CL adds a dynamic loader generator based on XML files. It also refactors the entry point generation script to move the XML parsing into a helper class. Additionally this includes a new GLES 1.0 base header. The new header allows for function pointer types and hiding prototypes. All tests and samples now load ANGLE dynamically. In the future this will be extended to load entry points from the driver directly when possible. This will allow us to perform more accurate A/B testing. The new build configuration leads to some tests having more warnings applied. The CL includes fixes for the new warnings. Bug: angleproject:2995 Change-Id: I6726d4163f7a6e54d2482f094c0a952f59702a05 Reviewed-on: https://chromium-review.googlesource.com/c/1359516 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org>
Jamie Madill b980c563 2018-11-27T11:34:27 Reformat all cpp and h files. This applies git cl format --full to all ANGLE sources. Bug: angleproject:2986 Change-Id: Ib504e618c1589332a37e97696cdc3515d739308f Reviewed-on: https://chromium-review.googlesource.com/c/1351367 Reviewed-by: Jamie Madill <jmadill@chromium.org> Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org>
Jamie Madill a2f043d8 2018-07-10T17:21:20 dEQP: Implement platform error handling. Also downgrades several Vulkan UNIMPLEMENTED() to WARN(). Also downgrades a couple D3D-specific errors to warnings. Also downgrades an undefined behaviour integer clear error to warning. Also includes suppressions for failing D3D11 ES 3.1 SSBO tests. Also includes suppressions for failing Android GLES format tests. Also includes suppressions for failing Android Vulkan buffer tests. Bug: angleproject:2552 Bug: angleproject:2567 Bug: angleproject:1951 Bug: angleproject:2405 Change-Id: Ie619085021d42012cd578b669f7ff4252ca41a58 Reviewed-on: https://chromium-review.googlesource.com/1062791 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Yuly Novikov <ynovikov@chromium.org>
Edward Lemur cbab2750 2018-05-31T18:49:31 DEPS.chromium: Sync dEQP to ANGLE (not Chromium) third_party. Bug: 782846 Change-Id: I88af578dfd27fb69751138d9b263a148da9ecf23 Reviewed-on: https://chromium-review.googlesource.com/1081113 Reviewed-by: Jamie Madill <jmadill@chromium.org> Reviewed-by: Yuly Novikov <ynovikov@chromium.org> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Jamie Madill 4428d395 2018-05-17T15:16:27 dEQP: Abort gtest run on exception. Prevents massive infra failures when the tests are all broken. This also gives the GoogleTest harness the ability to handle different test result types. Bug: angleproject:2552 Change-Id: Id9832e8557e2ee4e6a248a27729f67a81bf2d830 Reviewed-on: https://chromium-review.googlesource.com/1064014 Reviewed-by: Yuly Novikov <ynovikov@chromium.org> Reviewed-by: Geoff Lang <geofflang@chromium.org> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Jamie Madill 9b1a9084 2018-05-17T15:16:26 dEQP: Format display factory and clean up style. This fixes a bunch of minor style issues. Bug: angleproject:2552 Change-Id: If0e8130cf85deb176a5dcbca568cca477d2c1068 Reviewed-on: https://chromium-review.googlesource.com/1062789 Reviewed-by: Yuly Novikov <ynovikov@chromium.org> Reviewed-by: Geoff Lang <geofflang@chromium.org> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Jamie Madill 1ce8aede 2018-05-17T09:19:05 dEQP: Shorten test case names. Instead of dEQP_GLES2.Default/functional.blah... Use dEQP.GLES2/functional.blah... This makes them a bit easier to parse and reconstruct from memory. Bug: angleproject:2557 Change-Id: I092dc6ef10d4451ddafe373e699a5d35247f8785 Reviewed-on: https://chromium-review.googlesource.com/1062065 Reviewed-by: Geoff Lang <geofflang@chromium.org> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Jamie Madill 21386a34 2018-05-16T12:34:06 dEQP: Implement --deqp-case for GTest. This parses the dEQP test name into a GoogleTest-friendly gtest_filter automatically. This makes pasting in test case names from the expectations file more straight-forward. Bug: angleproject:2557 Change-Id: Ib5fbb46ad487a89352d728aebedc1752ddb0673d Reviewed-on: https://chromium-review.googlesource.com/1058252 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Geoff Lang <geofflang@chromium.org>
Jamie Madill b0dde4b9 2018-05-14T18:17:21 dEQP: Clean up some style in gTest source. Bug: angleproject:2557 Change-Id: Ica29c686ad3a71da3f8564f8a78bbe4f57ce304f Reviewed-on: https://chromium-review.googlesource.com/1058251 Reviewed-by: Luc Ferron <lucferron@chromium.org> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Luc Ferron feb87852 2018-03-22T13:41:25 dEQP: Enable depth/stencil backbuffer. This will make all the dEQP tests no longer return "not supported" when trying to do any depth/stencil work. With this comes a bunch of new tests suppressions to be able to proceed forward. Bug: angleproject:2425 Bug: angleproject:2426 Bug: angleproject:2427 Bug: angleproject:2428 Bug: angleproject:2429 Change-Id: I923026bb29c4ed1f4586b89176eea848fd2ceb71 Reviewed-on: https://chromium-review.googlesource.com/976339 Commit-Queue: Luc Ferron <lucferron@chromium.org> Reviewed-by: Jamie Madill <jmadill@chromium.org>
Geoff Lang bb514004 2017-04-28T14:08:38 Request RGBA8 configs by default for gtest dEQP. Previously, dEQP would just use the first config returned which was RGBA4 which is not very representitive of what our users would request. BUG=angleproject:2337 BUG=angleproject:1095 BUG=angleproject:2222 Change-Id: I5c9defc7ce1feaa49407557b88d9bb6b98571771 Reviewed-on: https://chromium-review.googlesource.com/490726 Reviewed-by: Luc Ferron <lucferron@chromium.org> Reviewed-by: Jamie Madill <jmadill@chromium.org> Reviewed-by: Geoff Lang <geofflang@chromium.org> Commit-Queue: Geoff Lang <geofflang@chromium.org>
Geoff Lang c7295f4d 2018-02-01T13:05:25 Vulkan: Add dEQP ES2 expectations. Only run the info tests for now, skip everything else. Rename angle-vk config to angle-vulkan BUG=angleproject:2161 Change-Id: I0a57f1f8fbfa9d27c2c376d38685184622f0779b Reviewed-on: https://chromium-review.googlesource.com/896552 Commit-Queue: Geoff Lang <geofflang@chromium.org> Reviewed-by: Jamie Madill <jmadill@chromium.org>
Geoff Lang 513bc77d 2018-01-29T15:50:24 Vulkan: Allow dEQP to create Vulkan displays. BUG=angleproject:2161 Change-Id: Ia2d4b4edcf9a40621573f124d14147224bbd8431 Reviewed-on: https://chromium-review.googlesource.com/891803 Commit-Queue: Geoff Lang <geofflang@chromium.org> Reviewed-by: Jamie Madill <jmadill@chromium.org>
Geoff Lang 0742c8c9 2016-12-01T11:18:22 Pass the --deqp-egl-display-type= flag to dEQP tests. BUG=angleproject:1442 Change-Id: I49299d2345f67b7ed3e2c80db0ec85589fc7583c Reviewed-on: https://chromium-review.googlesource.com/415526 Reviewed-by: Jamie Madill <jmadill@chromium.org> Commit-Queue: Geoff Lang <geofflang@chromium.org>
Geoff Lang cec2b7f8 2016-11-18T14:11:58 Allow the null backend to be used by dEQP. BUG=angleproject:1468 Change-Id: Ife6f0001b9a6ae06c98385df47fc3887036f3a36 Reviewed-on: https://chromium-review.googlesource.com/412863 Commit-Queue: Geoff Lang <geofflang@chromium.org> Reviewed-by: Corentin Wallez <cwallez@chromium.org>
Jamie Madill e5c53e3e 2016-11-07T12:22:48 dEQP: Add command-line parsing of back-end. Also use the requested renderer to initialize expectations. BUG=angleproject:1442 Change-Id: Idf54072dac5f7ad9deea70e97d65e36a6e883b1c Reviewed-on: https://chromium-review.googlesource.com/407802 Commit-Queue: Geoff Lang <geofflang@chromium.org> Reviewed-by: Corentin Wallez <cwallez@chromium.org>
Yuly Novikov 4628f386 2016-08-13T14:13:16 Add dEQP search paths on Android devices Including: test expectations and list, dEQP data and test results. Hardcode chromium_tests_root for now, to avoid adding a new API to pass the path from test runner to ANGLE. BUG=angleproject:1471 Change-Id: I654fa4727c436d743b3d6344ad17cb030934cde2 Reviewed-on: https://chromium-review.googlesource.com/368984 Commit-Queue: Yuly Novikov <ynovikov@chromium.org> Reviewed-by: Jamie Madill <jmadill@chromium.org>
Jamie Madill fbcd96db 2016-08-03T17:12:34 dEQP: Add simple stats reporting to GTest runs. BUG=None Change-Id: Ia2b59916ebb0de5a09c1f039200d6c8885dd294c Reviewed-on: https://chromium-review.googlesource.com/365940 Reviewed-by: Yuly Novikov <ynovikov@chromium.org> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Martin Radev 28ff4fd8 2016-07-06T11:54:21 Add deqp gles 3.1 tests to the build files BUG=angleproject:1442 Change-Id: I2ef27478573ec362bff57b532f7fee5731333b06 Reviewed-on: https://chromium-review.googlesource.com/360121 Reviewed-by: Jamie Madill <jmadill@chromium.org> Commit-Queue: Olli Etuaho <oetuaho@nvidia.com>
Corentin Wallez a78d12c5 2016-02-08T14:02:52 Make the dEQP GTEST initialization failures red on swarming. On swarming the dEQP GTEST initialization failures would show as green because the FAIL GTEST macro was called outside of a test. Use exit(-1) instead that shows up as red on the bots. BUG=580045 Change-Id: Icf82a8593a11fe7e992778ffd8c682f6738c39a6 Reviewed-on: https://chromium-review.googlesource.com/326690 Reviewed-by: Geoff Lang <geofflang@chromium.org>
Corentin Wallez af344a1b 2016-02-05T14:51:00 Fail the dEQP tests early if the case list is not found. BUG=580045 Change-Id: I917f62dca047ca163bfdbcc5437b2fc1d7a1e020 Reviewed-on: https://chromium-review.googlesource.com/326401 Reviewed-by: Jamie Madill <jmadill@chromium.org> Commit-Queue: Corentin Wallez <cwallez@chromium.org>
Jamie Madill 5111a062 2015-11-30T10:31:36 Update dEQP integration. Intead of generating a case list from a build dEQP executable and packing it into a gzip file, we can use the Android CTS mustpass list. This has the benefit of allowing us to remove some failure expectations for broken tests. It also means we don't need to ever regenerate our case list files. Also remove the old dEQP test integration, since it is no longer supported or possibly even working. And update our test expectations to match the new Android CTS must-pass lists. This change will likely need to be landed simulatanously with a Chromium-side CL, due to the nature of the gyp/DEPS configuration. BUG=angleproject:1236 Change-Id: I352e4a651dfb88193a1c4991974502dee1edb67b Reviewed-on: https://chromium-review.googlesource.com/313792 Reviewed-by: Corentin Wallez <cwallez@chromium.org> Tested-by: Jamie Madill <jmadill@chromium.org>
Jamie Madill 0c1d4403 2015-10-29T10:13:49 dEQP: Use full GTest name in non-standalone. With crbug.com/509883 fixed, we can use the full test names. BUG=509883 BUG=angleproject:1153 Change-Id: I62bf60fb5066de8c0cc118512ba0d9e9bbd94303 Reviewed-on: https://chromium-review.googlesource.com/309653 Tested-by: Jamie Madill <jmadill@chromium.org> Reviewed-by: Corentin Wallez <cwallez@chromium.org>
Corentin Wallez 02f98029 2015-10-13T11:13:34 Use named value-parameterized tests for dEQP gtests This replace the non-descriptive digit at the end of the test name by the name of the dEQP test being run. Reland, enabling the named tests only in standalone. BUG=angleproject:1153 Change-Id: I1336d38425c19305fae8279a42d4931cc516215c Reviewed-on: https://chromium-review.googlesource.com/304171 Reviewed-by: Geoff Lang <geofflang@chromium.org> Tested-by: Jamie Madill <jmadill@chromium.org>
Corentin Wallez 6026538a 2015-09-04T14:36:16 Revert "Use named value-parameterized tests for dEQP gtests" This reverts commit 0db7b15554f30c528a69285ec19150f36be263f7. BUG= Change-Id: Ia2de8268bd51d865ad87875c55cb5bfcc51e29f6 Reviewed-on: https://chromium-review.googlesource.com/297760 Reviewed-by: Corentin Wallez <cwallez@chromium.org> Tested-by: Corentin Wallez <cwallez@chromium.org>
Corentin Wallez 0db7b155 2015-09-03T15:25:41 Use named value-parameterized tests for dEQP gtests This replace the non-descriptive digit at the end of the test name by the name of the dEQP test being run. BUG=angleproject:1153 Change-Id: Ibc9e5fbf46ffbf3a8c783807161b2bdc63119f54 Reviewed-on: https://chromium-review.googlesource.com/296686 Reviewed-by: Jamie Madill <jmadill@chromium.org> Tested-by: Corentin Wallez <cwallez@chromium.org> Tryjob-Request: Corentin Wallez <cwallez@chromium.org>
Geoff Lang 5f57cc6e 2015-08-26T13:41:37 Add targets that run the EGL dEQP tests. Generates lots of assertion failures when run, expectations and assertion fixes to come. BUG=angleproject:1139 Change-Id: Ia55b4b2ca3a7844d8d9d05cb7043cd42d55b86ed Reviewed-on: https://chromium-review.googlesource.com/295243 Reviewed-by: Jamie Madill <jmadill@chromium.org> Reviewed-by: Corentin Wallez <cwallez@chromium.org> Tested-by: Geoff Lang <geofflang@chromium.org>
Jamie Madill cce24e9c 2015-07-29T15:14:29 dEQP-GLES2: Reland of "Skip flush/finish test on Windows." This test is having flaky timeouts on the bots. Disable it for now. The re-land fixes the test syntax, and ensures we will see any errors in test config as test failures on the bots. BUG=angleproject:1107 TEST=angle_deqp_gles2_tests runs Change-Id: Ia4fa8affb70e4474d8e43ff87ee86328aa47ebeb Reviewed-on: https://chromium-review.googlesource.com/289580 Tested-by: Jamie Madill <jmadill@chromium.org> Reviewed-by: Geoff Lang <geofflang@chromium.org>
Jamie Madill bc4acb4e 2015-07-22T13:37:42 dEQP: Update dEQP-GLES3 test expectations. Tests are all green on my local Nvidia configuration. BUG=510822 BUG=angleproject:1071 Change-Id: I56d30089d574e795cfbd012ce7430896856d32ee Reviewed-on: https://chromium-review.googlesource.com/287517 Tested-by: Jamie Madill <jmadill@chromium.org> Reviewed-by: Kenneth Russell <kbr@chromium.org> Reviewed-by: Corentin Wallez <cwallez@chromium.org>
Jamie Madill 86170f33 2015-07-21T13:01:12 Add a GLES3 GoogleTest target. BUG=510822 BUG=angleproject:1071 Change-Id: Ia2e6d0ac71e61dbed75e55994d2bf874a75fabd6 Reviewed-on: https://chromium-review.googlesource.com/286831 Reviewed-by: Corentin Wallez <cwallez@chromium.org> Reviewed-by: Kenneth Russell <kbr@chromium.org> Tested-by: Jamie Madill <jmadill@chromium.org>
Jamie Madill 4d06a24c 2015-06-26T11:12:24 dEQP: Extend GoogleTest support. *re-land after rebase on top of Chrome build fix* This patch will enable a Chromium-side CL to build dEQP/ANGLE with GoogleTest. BUG=angleproject:998 BUG=500736 Change-Id: Id02c66a260b0a7bab3de9cda624fe097322b2d3c Reviewed-on: https://chromium-review.googlesource.com/281824 Tested-by: Jamie Madill <jmadill@chromium.org> Reviewed-by: Corentin Wallez <cwallez@google.com> Reviewed-by: Kenneth Russell <kbr@chromium.org>
Jamie Madill 57cdc69c 2015-06-23T21:03:06 Revert "dEQP: Extend GoogleTest support." Causing runhooks failures: gyp: Undefined variable deqp_path in C:\b\build\slave\GPU_Win_Builder\build\src\third_party\angle\src\tests\tests.gyp BUG=angleproject:998 BUG=500736 This reverts commit 8edd75d1bb6680e7c2018ece9f5c81e3b87bf938. Change-Id: I8cd83bb5871dea81abea61554b182825145298e6 Reviewed-on: https://chromium-review.googlesource.com/281208 Reviewed-by: Jamie Madill <jmadill@chromium.org> Tested-by: Jamie Madill <jmadill@chromium.org>
Jamie Madill 8edd75d1 2015-06-22T15:35:53 dEQP: Extend GoogleTest support. This patch will enable a Chromium-side CL to build dEQP/ANGLE with GoogleTest. BUG=angleproject:998 BUG=500736 Change-Id: Ie480e3319c1bd77dd1de2c964f78ef97297fcfe9 Reviewed-on: https://chromium-review.googlesource.com/280856 Tested-by: Jamie Madill <jmadill@chromium.org> Reviewed-by: Corentin Wallez <cwallez@chromium.org> Reviewed-by: Zhenyao Mo <zmo@chromium.org>
Jamie Madill 7f165fd9 2015-06-10T16:46:11 dEQP: Hook up GPU test expectations code. With the list of failures, this gives a 100% passing GLES2 test suite. BUG=angleproject:998 Change-Id: I20985c729104d7a114352d156c238bca3ea1698c Reviewed-on: https://chromium-review.googlesource.com/274423 Reviewed-by: Kenneth Russell <kbr@chromium.org> Tested-by: Jamie Madill <jmadill@chromium.org>