src/tests/deqp_support/angle_deqp_gtest.cpp


Log

Author Commit Date CI Message
Jonah Ryan-Davis 6e4004dd 2019-12-06T14:41:17 Add metal display type to angle_deqp_tests Adds --deqp-egl-display-type=angle-metal for running the deqp tests on Mac. Note: does not add any test suppressions for metal. Bug: angleproject:2634 Change-Id: I8f8d3a83cf93a5b930b05af9f0075a9ce435823a Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/1954612 Reviewed-by: Jonah Ryan-Davis <jonahr@google.com> Commit-Queue: Jonah Ryan-Davis <jonahr@google.com>
Courtney Goeltzenleuchter 2c703b1a 2019-10-24T17:53:24 Print the testConfig in dEQP tests. This will help debug failures on the bots that could be caused by the wrong test config being used. As a side effect this changes the condition array of GPUTestConfig to use an angle::BitSetT BUG=chromium:997351 Change-Id: If742cd3d416694d5df6c3e6c1d722a24f487a960 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/1879585 Commit-Queue: Courtney Goeltzenleuchter <courtneygo@google.com> Reviewed-by: Ian Elliott <ianelliott@google.com>
Courtney Goeltzenleuchter febdcf51 2019-10-28T10:54:03 Roll dEQP to VK-GL-CTS Switch from Android dEQP to Khronos GL CTS. We now mirror from the public Khronos Github github.com/KhronosGroup/VK-GL-CTS This includes a number of text fixes for GLES 2.0 conformance. Bug: angleproject:3914 Bug: angleproject:1656 Change-Id: I4cccd33e47752742cb989127c5883c0afd075b9e Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/1879579 Reviewed-by: Jamie Madill <jmadill@chromium.org> Commit-Queue: Courtney Goeltzenleuchter <courtneygo@google.com>
Jamie Madill 57b37b6b 2019-09-25T18:29:28 Rename util/system_utils to util/test_utils. This removes a GN naming conflict between util/system_utils and common/system_utils. This conflict was preventing us from adding unit tests to utils' version of system_utils. Since these functions are only useful to tests and samples rename them test_utils for simplicity. Will enable further development of ANGLE's standalone testing harness. Bug: angleproject:3162 Change-Id: I9e34fb69f96c5de6dc2453fce4148a0f285e15ed Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/1825268 Reviewed-by: Jonah Ryan-Davis <jonahr@google.com> Reviewed-by: Yuly Novikov <ynovikov@chromium.org> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Jamie Madill a3f1981d 2019-09-23T17:04:03 Add SwiftShader to command line arguments. Also adds parsing SwiftShader to the test expectations code. Bug: angleproject:3876 Change-Id: I70e1475b62f86160d5a150218e792801fb0259bc Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/1817109 Reviewed-by: Alexis Hétu <sugoi@chromium.org> Reviewed-by: Yuly Novikov <ynovikov@chromium.org> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Shahbaz Youssefi f0ea83a4 2019-06-26T10:17:26 Vulkan: Initial support for ES3.1 Bug: angleproject:3520 Bug: angleproject:3521 Change-Id: I49d67cae0bba3e147c7a83c10e09fe1ee5ff0824 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/1678400 Reviewed-by: Geoff Lang <geofflang@chromium.org> Reviewed-by: Jonah Ryan-Davis <jonahr@google.com> Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org>
Jamie Madill 7a026133 2019-05-20T10:34:08 Fix Android data dir for KHR dEQP tests. Also includes a fix for case list paths on developer machines. Also includes a test expectation update for GLES2. Bug: angleproject:3353 Change-Id: Idb01eb8d806067739253a6676a6d2d8ee1199929 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/1616853 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Yuly Novikov <ynovikov@chromium.org>
Jamie Madill cafd7736 2019-05-17T09:33:41 Fix KHR-GLES dEQP test paths. Our GN was using the incorrect path for the new test sets. This was showing up as an isolate_tests failure when running the new tests. Also fixes incorrect search paths for the case list directory when used in a Chromium checkout. And adds a few other missing data dependencies. Bug: angleproject:3353 Change-Id: I602db1e5debdad88b6eace76416fc2c10c059f8b Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/1616021 Reviewed-by: Yuly Novikov <ynovikov@chromium.org> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Jamie Madill cfffc138 2019-05-16T11:35:01 Add KHR-GLES dEQP tests. This reorganizes the dEQP build targets to allow for better reuse with the new KHR test set. It also adds a single new test target called "angle_deqp_khr_tests" that houses all the new test sets. The new test targets can be compiled now but cannot yet be executed. That new functionality will follow in a subsequent CL. Along with test expectations and integration. Uses a new template type in the build to conslidate GN customization. Adds the GLES 2.0 and 3.0 tests for now. We can follow up with additional work for the GLES 3.1+ tests. Also adds test expectations. Bug: angleproject:3353 Change-Id: Ib0673e97f679c8aa46e27dfa616845a2014a75fe Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/1604070 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Yuly Novikov <ynovikov@chromium.org>
Jamie Madill 6ddb7768 2019-05-15T15:25:36 Remove angle:: prefix in angle_deqp_gtest. Refactor and cleanup only. Bug: angleproject:3353 Change-Id: I2a01ddf71ba44b8f1a1fc2c4ccbac9edc7c80174 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/1613893 Reviewed-by: Courtney Goeltzenleuchter <courtneygo@google.com> Reviewed-by: Yuly Novikov <ynovikov@chromium.org> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Jonah Ryan-Davis 1ad7a072 2019-04-26T14:05:25 Clean up unexpected passed/failed test behavior for generate_stats The lists of tests can't be added to the sheets if more than 50,000 lines. Since the goal is to get rid of all unexpected passes/failures, it's reasonable to clip the lists by length. Other formatting changes too. Bug: angleproject:3398 Change-Id: Ice461862fb4266e0ad1280f20de85224fc6d7e97 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/1585612 Reviewed-by: Geoff Lang <geofflang@chromium.org> Reviewed-by: Yuly Novikov <ynovikov@chromium.org> Commit-Queue: Jonah Ryan-Davis <jonahr@google.com>
Jonah Ryan-Davis 51ac37ee 2019-04-25T13:32:16 Print unexpected passed/failed tests with formatting for generate_stats The stats generation should pick up a list of unexpected failed/passed tests, so the tests should be printed with the format used by this script. Bug: angleproject:3398 Change-Id: I96a8eba6c2e294f64ca59ec60042cc688d1133c5 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/1584363 Reviewed-by: Geoff Lang <geofflang@chromium.org> Commit-Queue: Jonah Ryan-Davis <jonahr@google.com>
Jonah Ryan-Davis 3b775865 2019-04-18T15:59:15 Skipped tests not accumulated in dEQP stats output. The sSkippedTests is not incremented in the kGpuTestSkip check in runTest, when it should be. Bug: angleproject:3390 Change-Id: I4aca73ba1bbe3060fc3c0cd27679c2276a30a741 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/1574620 Reviewed-by: Jamie Madill <jmadill@chromium.org> Commit-Queue: Jonah Ryan-Davis <jonahr@google.com>
Jonah Ryan-Davis 0f7a2330 2019-04-17T11:13:41 Modify test result output from dEQP tests for visualization script. Test results will now be prepended with [TESTSTATS], and there are two new outputs added for unexpected passes and unexpected failures. Bug: angleproject:3369 Change-Id: Iec4dafe695531dcf56d496083036ccf1e286ead7 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/1571388 Commit-Queue: Jonah Ryan-Davis <jonahr@google.com> Reviewed-by: Jamie Madill <jmadill@chromium.org>
Jonah Ryan-Davis c832cdd7 2019-04-15T14:53:01 dEQP test_expectations parser should output an error on unused lines. dEQP test_expectations parser should allow nested overrides. When an expectation line is not used for any test, it's likely duplicated or misspelled, or the test was removed. The parser should output these errors to make it easier to keep the expectations up to date. Also necessary to allow longer/more specific expectations to override shorter ones, so the scenario where you FAIL one set of tests across platforms, and SKIP a subset of that group on one platform will still be allowed. Bug: angleproject:3368 Change-Id: If0470cb6fb12f8d3daa97df0c701185fa086668c Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/1564725 Commit-Queue: Jonah Ryan-Davis <jonahr@google.com> Reviewed-by: Geoff Lang <geofflang@chromium.org>
Jamie Madill 23667b7e 2019-04-10T12:49:56 Update test result output from ANGLE dEQP gtests. * prints lines for total, passed, failed, not supported, skipped, and exception results. * prints unexpected passed and failed tests each individually using the dEQP name of the test. * moves skipped test detection into the test loop. skipped tests will now print a message that they've been skipped. Bug: angleproject:3369 Change-Id: Idbfc0154a658eda3f52b5bfd30fbff0860a00133 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/1561970 Reviewed-by: Jamie Madill <jmadill@chromium.org> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Jonah Ryan-Davis 34cbc544 2019-04-05T11:30:10 Use ANGLE's gpu_info_util libraries for test expectations 3. Remove gpu_info.h and gpu_info.cc and use ANGLE's own gpu_info_util to to gather device info instead. - Support collecting info for and parsing expectations of specific Android devices such as Nexus 5X and Pixel 2. - Change parser behavior to more closely follow: bit.ly/chromium-test-list-format Bug: angleproject:2677 Change-Id: I4c0b9342142b718e884484a6bcaac2dff3ac575a Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/1553822 Commit-Queue: Jonah Ryan-Davis <jonahr@google.com> Reviewed-by: Jamie Madill <jmadill@chromium.org>
Jonah Ryan-Davis 518c679d 2019-03-25T16:25:10 Clean up test_expectations to merge with angle style/convention. Can rename/clean up the library to move from chrome style to angle style. Can also start pulling out chrome-specific definitions. More work to be done when moving gpu_info to use Angle's SystemInfo. Bug: angleproject:2677 Change-Id: I75fd4446aa4be85fba590cd00cd3a213a0b3752e Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/1538677 Commit-Queue: Jonah Ryan-Davis <jonahr@google.com> Reviewed-by: Geoff Lang <geofflang@chromium.org>
Jonah Ryan-Davis 0449a902 2019-03-22T16:03:54 Move gpu_test_expectations from third_party into ANGLE. 1. Copy the code from src/tests/third_party/gpu_test_expectations to src/test/test_expectations, rename .cc files to .cpp. Put these in a new static library and update dEQP to link against it in src/tests/BUILD.gn. 2. Merge the code in angle_config.h into the rest of the expectations parser, this code was added so that the rest of the parser would compile in ANGLE's tree with minimal modification. Still need to follow up with the third step to close the issue. Bug: angleproject:2677 Change-Id: Icf09b4eeed83a6d09b1964ad2adcfa85cabb4b63 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/1536312 Commit-Queue: Jonah Ryan-Davis <jonahr@google.com> Reviewed-by: Geoff Lang <geofflang@chromium.org>
Jamie Madill 4ae2c12f 2019-03-23T18:07:26 Use Khronos master lists for dEQP tests. There are an number of valid tests that are excluded from the Android test lists. Likely some older and low spec devices weren't passing these tests and for historical reasons they were omitted from the Android CTS. ANGLE however needs to pass these tests to be conformant and up until now we weren't testing them on our bots. The EGL tests still use the same Android version of the test list. It seems the Khronos version just lists wildcards. ANGLE needs an explicit test list. Bug: angleproject:3242 Change-Id: Ie0aa2aa94689be0a40e7cc39e05f0042c3f03f5d Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/1520986 Reviewed-by: Tim Van Patten <timvp@google.com> Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Victor Costan ffd39978 2019-02-20T10:45:24 test: Replace _TEST_CASE_ with _TEST_SUITE_. Googletest is (at last) converging with industry-standard terminology [1]. We previously called test suites "test cases", which was rather confusing for folks coming from any other testing framework. Chrome now has a googletest version that supports _TEST_SUITE_ macros instead of _TEST_CASE_, so this CL cleans up some of the outdated usage. [1] https://github.com/google/googletest/blob/master/googletest/docs/primer.md#beware-of-the-nomenclature Bug: chromium:925652 Change-Id: Ia0deec0bc4216ef1adabc33985a7cbda89682608 Reviewed-on: https://chromium-review.googlesource.com/c/1477418 Reviewed-by: Geoff Lang <geofflang@chromium.org> Commit-Queue: Victor Costan <pwnall@chromium.org>
Jamie Madill ba319ba3 2018-12-29T10:29:33 Re-land "Load entry points dynamically in tests and samples." Fixes the Android/ChromeOS/Fuchsia builds by using consistent EGL headers. This CL adds a dynamic loader generator based on XML files. It also refactors the entry point generation script to move the XML parsing into a helper class. Additionally this includes a new GLES 1.0 base header. The new header allows for function pointer types and hiding prototypes. All tests and samples now load ANGLE dynamically. In the future this will be extended to load entry points from the driver directly when possible. This will allow us to perform more accurate A/B testing. The new build configuration leads to some tests having more warnings applied. The CL includes fixes for the new warnings. Bug: angleproject:2995 Change-Id: I5a8772f41a0f89570b3736b785f44b7de1539b57 Reviewed-on: https://chromium-review.googlesource.com/c/1392382 Reviewed-by: Jamie Madill <jmadill@chromium.org> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Yuly Novikov 9f088621 2018-12-29T20:46:15 Revert "Load entry points dynamically in tests and samples." This reverts commit 03923558a7103827ffec6a4d2a1453ed91f01c6f. Reason for revert: fails compilation on Android, ChromeOS and Fuchsia during roll https://chromium-review.googlesource.com/c/chromium/src/+/1392624 Original change's description: > Load entry points dynamically in tests and samples. > > This CL adds a dynamic loader generator based on XML files. It also > refactors the entry point generation script to move the XML parsing > into a helper class. > > Additionally this includes a new GLES 1.0 base header. The new > header allows for function pointer types and hiding prototypes. > > All tests and samples now load ANGLE dynamically. In the future this > will be extended to load entry points from the driver directly when > possible. This will allow us to perform more accurate A/B testing. > > The new build configuration leads to some tests having more warnings > applied. The CL includes fixes for the new warnings. > > Bug: angleproject:2995 > Change-Id: I6726d4163f7a6e54d2482f094c0a952f59702a05 > Reviewed-on: https://chromium-review.googlesource.com/c/1359516 > Commit-Queue: Jamie Madill <jmadill@chromium.org> > Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org> TBR=ynovikov@chromium.org,jmadill@chromium.org,syoussefi@chromium.org Change-Id: I902bec2d733c2b879be29c02ab52a0b7d4eaa077 No-Presubmit: true No-Tree-Checks: true No-Try: true Bug: angleproject:2995 Reviewed-on: https://chromium-review.googlesource.com/c/1392381 Reviewed-by: Yuly Novikov <ynovikov@chromium.org> Commit-Queue: Yuly Novikov <ynovikov@chromium.org>
Jamie Madill 03923558 2018-12-29T10:29:33 Load entry points dynamically in tests and samples. This CL adds a dynamic loader generator based on XML files. It also refactors the entry point generation script to move the XML parsing into a helper class. Additionally this includes a new GLES 1.0 base header. The new header allows for function pointer types and hiding prototypes. All tests and samples now load ANGLE dynamically. In the future this will be extended to load entry points from the driver directly when possible. This will allow us to perform more accurate A/B testing. The new build configuration leads to some tests having more warnings applied. The CL includes fixes for the new warnings. Bug: angleproject:2995 Change-Id: I6726d4163f7a6e54d2482f094c0a952f59702a05 Reviewed-on: https://chromium-review.googlesource.com/c/1359516 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org>
Jamie Madill b980c563 2018-11-27T11:34:27 Reformat all cpp and h files. This applies git cl format --full to all ANGLE sources. Bug: angleproject:2986 Change-Id: Ib504e618c1589332a37e97696cdc3515d739308f Reviewed-on: https://chromium-review.googlesource.com/c/1351367 Reviewed-by: Jamie Madill <jmadill@chromium.org> Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org>
Jamie Madill a2f043d8 2018-07-10T17:21:20 dEQP: Implement platform error handling. Also downgrades several Vulkan UNIMPLEMENTED() to WARN(). Also downgrades a couple D3D-specific errors to warnings. Also downgrades an undefined behaviour integer clear error to warning. Also includes suppressions for failing D3D11 ES 3.1 SSBO tests. Also includes suppressions for failing Android GLES format tests. Also includes suppressions for failing Android Vulkan buffer tests. Bug: angleproject:2552 Bug: angleproject:2567 Bug: angleproject:1951 Bug: angleproject:2405 Change-Id: Ie619085021d42012cd578b669f7ff4252ca41a58 Reviewed-on: https://chromium-review.googlesource.com/1062791 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Yuly Novikov <ynovikov@chromium.org>
Edward Lemur cbab2750 2018-05-31T18:49:31 DEPS.chromium: Sync dEQP to ANGLE (not Chromium) third_party. Bug: 782846 Change-Id: I88af578dfd27fb69751138d9b263a148da9ecf23 Reviewed-on: https://chromium-review.googlesource.com/1081113 Reviewed-by: Jamie Madill <jmadill@chromium.org> Reviewed-by: Yuly Novikov <ynovikov@chromium.org> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Jamie Madill 4428d395 2018-05-17T15:16:27 dEQP: Abort gtest run on exception. Prevents massive infra failures when the tests are all broken. This also gives the GoogleTest harness the ability to handle different test result types. Bug: angleproject:2552 Change-Id: Id9832e8557e2ee4e6a248a27729f67a81bf2d830 Reviewed-on: https://chromium-review.googlesource.com/1064014 Reviewed-by: Yuly Novikov <ynovikov@chromium.org> Reviewed-by: Geoff Lang <geofflang@chromium.org> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Jamie Madill 9b1a9084 2018-05-17T15:16:26 dEQP: Format display factory and clean up style. This fixes a bunch of minor style issues. Bug: angleproject:2552 Change-Id: If0e8130cf85deb176a5dcbca568cca477d2c1068 Reviewed-on: https://chromium-review.googlesource.com/1062789 Reviewed-by: Yuly Novikov <ynovikov@chromium.org> Reviewed-by: Geoff Lang <geofflang@chromium.org> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Jamie Madill 1ce8aede 2018-05-17T09:19:05 dEQP: Shorten test case names. Instead of dEQP_GLES2.Default/functional.blah... Use dEQP.GLES2/functional.blah... This makes them a bit easier to parse and reconstruct from memory. Bug: angleproject:2557 Change-Id: I092dc6ef10d4451ddafe373e699a5d35247f8785 Reviewed-on: https://chromium-review.googlesource.com/1062065 Reviewed-by: Geoff Lang <geofflang@chromium.org> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Jamie Madill 21386a34 2018-05-16T12:34:06 dEQP: Implement --deqp-case for GTest. This parses the dEQP test name into a GoogleTest-friendly gtest_filter automatically. This makes pasting in test case names from the expectations file more straight-forward. Bug: angleproject:2557 Change-Id: Ib5fbb46ad487a89352d728aebedc1752ddb0673d Reviewed-on: https://chromium-review.googlesource.com/1058252 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: Geoff Lang <geofflang@chromium.org>
Jamie Madill b0dde4b9 2018-05-14T18:17:21 dEQP: Clean up some style in gTest source. Bug: angleproject:2557 Change-Id: Ica29c686ad3a71da3f8564f8a78bbe4f57ce304f Reviewed-on: https://chromium-review.googlesource.com/1058251 Reviewed-by: Luc Ferron <lucferron@chromium.org> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Luc Ferron feb87852 2018-03-22T13:41:25 dEQP: Enable depth/stencil backbuffer. This will make all the dEQP tests no longer return "not supported" when trying to do any depth/stencil work. With this comes a bunch of new tests suppressions to be able to proceed forward. Bug: angleproject:2425 Bug: angleproject:2426 Bug: angleproject:2427 Bug: angleproject:2428 Bug: angleproject:2429 Change-Id: I923026bb29c4ed1f4586b89176eea848fd2ceb71 Reviewed-on: https://chromium-review.googlesource.com/976339 Commit-Queue: Luc Ferron <lucferron@chromium.org> Reviewed-by: Jamie Madill <jmadill@chromium.org>
Geoff Lang bb514004 2017-04-28T14:08:38 Request RGBA8 configs by default for gtest dEQP. Previously, dEQP would just use the first config returned which was RGBA4 which is not very representitive of what our users would request. BUG=angleproject:2337 BUG=angleproject:1095 BUG=angleproject:2222 Change-Id: I5c9defc7ce1feaa49407557b88d9bb6b98571771 Reviewed-on: https://chromium-review.googlesource.com/490726 Reviewed-by: Luc Ferron <lucferron@chromium.org> Reviewed-by: Jamie Madill <jmadill@chromium.org> Reviewed-by: Geoff Lang <geofflang@chromium.org> Commit-Queue: Geoff Lang <geofflang@chromium.org>
Geoff Lang c7295f4d 2018-02-01T13:05:25 Vulkan: Add dEQP ES2 expectations. Only run the info tests for now, skip everything else. Rename angle-vk config to angle-vulkan BUG=angleproject:2161 Change-Id: I0a57f1f8fbfa9d27c2c376d38685184622f0779b Reviewed-on: https://chromium-review.googlesource.com/896552 Commit-Queue: Geoff Lang <geofflang@chromium.org> Reviewed-by: Jamie Madill <jmadill@chromium.org>
Geoff Lang 513bc77d 2018-01-29T15:50:24 Vulkan: Allow dEQP to create Vulkan displays. BUG=angleproject:2161 Change-Id: Ia2d4b4edcf9a40621573f124d14147224bbd8431 Reviewed-on: https://chromium-review.googlesource.com/891803 Commit-Queue: Geoff Lang <geofflang@chromium.org> Reviewed-by: Jamie Madill <jmadill@chromium.org>
Geoff Lang 0742c8c9 2016-12-01T11:18:22 Pass the --deqp-egl-display-type= flag to dEQP tests. BUG=angleproject:1442 Change-Id: I49299d2345f67b7ed3e2c80db0ec85589fc7583c Reviewed-on: https://chromium-review.googlesource.com/415526 Reviewed-by: Jamie Madill <jmadill@chromium.org> Commit-Queue: Geoff Lang <geofflang@chromium.org>
Geoff Lang cec2b7f8 2016-11-18T14:11:58 Allow the null backend to be used by dEQP. BUG=angleproject:1468 Change-Id: Ife6f0001b9a6ae06c98385df47fc3887036f3a36 Reviewed-on: https://chromium-review.googlesource.com/412863 Commit-Queue: Geoff Lang <geofflang@chromium.org> Reviewed-by: Corentin Wallez <cwallez@chromium.org>
Jamie Madill e5c53e3e 2016-11-07T12:22:48 dEQP: Add command-line parsing of back-end. Also use the requested renderer to initialize expectations. BUG=angleproject:1442 Change-Id: Idf54072dac5f7ad9deea70e97d65e36a6e883b1c Reviewed-on: https://chromium-review.googlesource.com/407802 Commit-Queue: Geoff Lang <geofflang@chromium.org> Reviewed-by: Corentin Wallez <cwallez@chromium.org>
Yuly Novikov 4628f386 2016-08-13T14:13:16 Add dEQP search paths on Android devices Including: test expectations and list, dEQP data and test results. Hardcode chromium_tests_root for now, to avoid adding a new API to pass the path from test runner to ANGLE. BUG=angleproject:1471 Change-Id: I654fa4727c436d743b3d6344ad17cb030934cde2 Reviewed-on: https://chromium-review.googlesource.com/368984 Commit-Queue: Yuly Novikov <ynovikov@chromium.org> Reviewed-by: Jamie Madill <jmadill@chromium.org>
Jamie Madill fbcd96db 2016-08-03T17:12:34 dEQP: Add simple stats reporting to GTest runs. BUG=None Change-Id: Ia2b59916ebb0de5a09c1f039200d6c8885dd294c Reviewed-on: https://chromium-review.googlesource.com/365940 Reviewed-by: Yuly Novikov <ynovikov@chromium.org> Commit-Queue: Jamie Madill <jmadill@chromium.org>
Martin Radev 28ff4fd8 2016-07-06T11:54:21 Add deqp gles 3.1 tests to the build files BUG=angleproject:1442 Change-Id: I2ef27478573ec362bff57b532f7fee5731333b06 Reviewed-on: https://chromium-review.googlesource.com/360121 Reviewed-by: Jamie Madill <jmadill@chromium.org> Commit-Queue: Olli Etuaho <oetuaho@nvidia.com>
Corentin Wallez a78d12c5 2016-02-08T14:02:52 Make the dEQP GTEST initialization failures red on swarming. On swarming the dEQP GTEST initialization failures would show as green because the FAIL GTEST macro was called outside of a test. Use exit(-1) instead that shows up as red on the bots. BUG=580045 Change-Id: Icf82a8593a11fe7e992778ffd8c682f6738c39a6 Reviewed-on: https://chromium-review.googlesource.com/326690 Reviewed-by: Geoff Lang <geofflang@chromium.org>
Corentin Wallez af344a1b 2016-02-05T14:51:00 Fail the dEQP tests early if the case list is not found. BUG=580045 Change-Id: I917f62dca047ca163bfdbcc5437b2fc1d7a1e020 Reviewed-on: https://chromium-review.googlesource.com/326401 Reviewed-by: Jamie Madill <jmadill@chromium.org> Commit-Queue: Corentin Wallez <cwallez@chromium.org>
Jamie Madill 5111a062 2015-11-30T10:31:36 Update dEQP integration. Intead of generating a case list from a build dEQP executable and packing it into a gzip file, we can use the Android CTS mustpass list. This has the benefit of allowing us to remove some failure expectations for broken tests. It also means we don't need to ever regenerate our case list files. Also remove the old dEQP test integration, since it is no longer supported or possibly even working. And update our test expectations to match the new Android CTS must-pass lists. This change will likely need to be landed simulatanously with a Chromium-side CL, due to the nature of the gyp/DEPS configuration. BUG=angleproject:1236 Change-Id: I352e4a651dfb88193a1c4991974502dee1edb67b Reviewed-on: https://chromium-review.googlesource.com/313792 Reviewed-by: Corentin Wallez <cwallez@chromium.org> Tested-by: Jamie Madill <jmadill@chromium.org>
Jamie Madill 0c1d4403 2015-10-29T10:13:49 dEQP: Use full GTest name in non-standalone. With crbug.com/509883 fixed, we can use the full test names. BUG=509883 BUG=angleproject:1153 Change-Id: I62bf60fb5066de8c0cc118512ba0d9e9bbd94303 Reviewed-on: https://chromium-review.googlesource.com/309653 Tested-by: Jamie Madill <jmadill@chromium.org> Reviewed-by: Corentin Wallez <cwallez@chromium.org>
Corentin Wallez 02f98029 2015-10-13T11:13:34 Use named value-parameterized tests for dEQP gtests This replace the non-descriptive digit at the end of the test name by the name of the dEQP test being run. Reland, enabling the named tests only in standalone. BUG=angleproject:1153 Change-Id: I1336d38425c19305fae8279a42d4931cc516215c Reviewed-on: https://chromium-review.googlesource.com/304171 Reviewed-by: Geoff Lang <geofflang@chromium.org> Tested-by: Jamie Madill <jmadill@chromium.org>
Corentin Wallez 6026538a 2015-09-04T14:36:16 Revert "Use named value-parameterized tests for dEQP gtests" This reverts commit 0db7b15554f30c528a69285ec19150f36be263f7. BUG= Change-Id: Ia2de8268bd51d865ad87875c55cb5bfcc51e29f6 Reviewed-on: https://chromium-review.googlesource.com/297760 Reviewed-by: Corentin Wallez <cwallez@chromium.org> Tested-by: Corentin Wallez <cwallez@chromium.org>
Corentin Wallez 0db7b155 2015-09-03T15:25:41 Use named value-parameterized tests for dEQP gtests This replace the non-descriptive digit at the end of the test name by the name of the dEQP test being run. BUG=angleproject:1153 Change-Id: Ibc9e5fbf46ffbf3a8c783807161b2bdc63119f54 Reviewed-on: https://chromium-review.googlesource.com/296686 Reviewed-by: Jamie Madill <jmadill@chromium.org> Tested-by: Corentin Wallez <cwallez@chromium.org> Tryjob-Request: Corentin Wallez <cwallez@chromium.org>
Geoff Lang 5f57cc6e 2015-08-26T13:41:37 Add targets that run the EGL dEQP tests. Generates lots of assertion failures when run, expectations and assertion fixes to come. BUG=angleproject:1139 Change-Id: Ia55b4b2ca3a7844d8d9d05cb7043cd42d55b86ed Reviewed-on: https://chromium-review.googlesource.com/295243 Reviewed-by: Jamie Madill <jmadill@chromium.org> Reviewed-by: Corentin Wallez <cwallez@chromium.org> Tested-by: Geoff Lang <geofflang@chromium.org>
Jamie Madill cce24e9c 2015-07-29T15:14:29 dEQP-GLES2: Reland of "Skip flush/finish test on Windows." This test is having flaky timeouts on the bots. Disable it for now. The re-land fixes the test syntax, and ensures we will see any errors in test config as test failures on the bots. BUG=angleproject:1107 TEST=angle_deqp_gles2_tests runs Change-Id: Ia4fa8affb70e4474d8e43ff87ee86328aa47ebeb Reviewed-on: https://chromium-review.googlesource.com/289580 Tested-by: Jamie Madill <jmadill@chromium.org> Reviewed-by: Geoff Lang <geofflang@chromium.org>
Jamie Madill bc4acb4e 2015-07-22T13:37:42 dEQP: Update dEQP-GLES3 test expectations. Tests are all green on my local Nvidia configuration. BUG=510822 BUG=angleproject:1071 Change-Id: I56d30089d574e795cfbd012ce7430896856d32ee Reviewed-on: https://chromium-review.googlesource.com/287517 Tested-by: Jamie Madill <jmadill@chromium.org> Reviewed-by: Kenneth Russell <kbr@chromium.org> Reviewed-by: Corentin Wallez <cwallez@chromium.org>
Jamie Madill 86170f33 2015-07-21T13:01:12 Add a GLES3 GoogleTest target. BUG=510822 BUG=angleproject:1071 Change-Id: Ia2e6d0ac71e61dbed75e55994d2bf874a75fabd6 Reviewed-on: https://chromium-review.googlesource.com/286831 Reviewed-by: Corentin Wallez <cwallez@chromium.org> Reviewed-by: Kenneth Russell <kbr@chromium.org> Tested-by: Jamie Madill <jmadill@chromium.org>
Jamie Madill 4d06a24c 2015-06-26T11:12:24 dEQP: Extend GoogleTest support. *re-land after rebase on top of Chrome build fix* This patch will enable a Chromium-side CL to build dEQP/ANGLE with GoogleTest. BUG=angleproject:998 BUG=500736 Change-Id: Id02c66a260b0a7bab3de9cda624fe097322b2d3c Reviewed-on: https://chromium-review.googlesource.com/281824 Tested-by: Jamie Madill <jmadill@chromium.org> Reviewed-by: Corentin Wallez <cwallez@google.com> Reviewed-by: Kenneth Russell <kbr@chromium.org>
Jamie Madill 57cdc69c 2015-06-23T21:03:06 Revert "dEQP: Extend GoogleTest support." Causing runhooks failures: gyp: Undefined variable deqp_path in C:\b\build\slave\GPU_Win_Builder\build\src\third_party\angle\src\tests\tests.gyp BUG=angleproject:998 BUG=500736 This reverts commit 8edd75d1bb6680e7c2018ece9f5c81e3b87bf938. Change-Id: I8cd83bb5871dea81abea61554b182825145298e6 Reviewed-on: https://chromium-review.googlesource.com/281208 Reviewed-by: Jamie Madill <jmadill@chromium.org> Tested-by: Jamie Madill <jmadill@chromium.org>
Jamie Madill 8edd75d1 2015-06-22T15:35:53 dEQP: Extend GoogleTest support. This patch will enable a Chromium-side CL to build dEQP/ANGLE with GoogleTest. BUG=angleproject:998 BUG=500736 Change-Id: Ie480e3319c1bd77dd1de2c964f78ef97297fcfe9 Reviewed-on: https://chromium-review.googlesource.com/280856 Tested-by: Jamie Madill <jmadill@chromium.org> Reviewed-by: Corentin Wallez <cwallez@chromium.org> Reviewed-by: Zhenyao Mo <zmo@chromium.org>
Jamie Madill 7f165fd9 2015-06-10T16:46:11 dEQP: Hook up GPU test expectations code. With the list of failures, this gives a 100% passing GLES2 test suite. BUG=angleproject:998 Change-Id: I20985c729104d7a114352d156c238bca3ea1698c Reviewed-on: https://chromium-review.googlesource.com/274423 Reviewed-by: Kenneth Russell <kbr@chromium.org> Tested-by: Jamie Madill <jmadill@chromium.org>