|
b681e939
|
2022-09-19T13:20:23
|
|
Turn on trace data hash checking automatically.
This should save time uploading trace data already present on the
device.
Bug: angleproject:7671
Change-Id: Ie655cba284b6b3aec2e264cd4d64e4d18ebff8a7
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3905902
Commit-Queue: Jamie Madill <jmadill@chromium.org>
Reviewed-by: Cody Northrop <cnorthrop@google.com>
Reviewed-by: Roman Lavrov <romanl@google.com>
|
|
|
094b49db
|
2022-09-19T10:13:28
|
|
Android Test Runner: Skip APK install if hash matches.
Bug: angleproject:7670
Change-Id: I984fe9377728e5a575fd04be2f5cb3ed1cdbafb5
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3904425
Reviewed-by: Roman Lavrov <romanl@google.com>
Commit-Queue: Jamie Madill <jmadill@chromium.org>
Reviewed-by: Cody Northrop <cnorthrop@google.com>
|
|
f30cfd18
|
2022-08-16T16:50:33
|
|
Perf and gold tests detect test SKIPs from json results.
Make angle_test_util.RunTestSuite() gets and returns json results
(using --isolated-script-test-output).
perf tests has an additional special case
(filed https://anglebug.com/7578):
Previously when metrics were missing it was assumed that this was due to
tests being skipped - which seems to currently be the case except for
the special case of GLMark2Benchmark* tests. Those produce .fps/.score
metrics instead of the metrics expected by this script. This CL keeps
those tests running (once) but then marks them as SKIP in test results
when no metrics are found (which is the behavior before this CL; it
might actually make more sense to count this as PASS because everything
matches our expectations).
We're still running them once, so if they FAIL we'll get notified.
gold tests:
Use json results file instead of checking for '[ SKIPPED ]' in stdout.
Bug: angleproject:7299
Bug: angleproject:7578
Change-Id: Ia751784ad1aa94dc855c8b58ebfe5ba3e06e462f
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3826167
Reviewed-by: Amirali Abdolrashidi <abdolrashidi@google.com>
Commit-Queue: Roman Lavrov <romanl@google.com>
|
|
|
2de8bb42
|
2022-08-03T18:15:52
|
|
Converge run wrappers into angle_test_util.RunTestSuite
Bug: angleproject:7299
Change-Id: Ifa882028a6f49579b4a9c6893053cb3bee8d83fd
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3808302
Reviewed-by: Jamie Madill <jmadill@chromium.org>
Reviewed-by: Amirali Abdolrashidi <abdolrashidi@google.com>
Commit-Queue: Roman Lavrov <romanl@google.com>
|
|
|
88b05d3c
|
2022-08-03T15:21:00
|
|
Use globals in android_helper to simplify flow.
Explicit .Initialize(suite_name) call to detect Android
Track suite (apk) that was installed. We use the same package
com.android.angle.test in multiple apks (eg system info and perftests)
so keeping track of which one is currently installed avoids the need to
take care of it manually (with PrepareTestSuite).
Bug: angleproject:7299
Change-Id: I4b96f0fb24e38fc42d3814fb0eeda5aae527de5a
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3808292
Reviewed-by: Jamie Madill <jmadill@chromium.org>
Commit-Queue: Roman Lavrov <romanl@google.com>
|
|
|
b7bd0808
|
2022-06-06T10:54:30
|
|
Support running end2end tests.
There is no batching though so all tests matching the filter will be ran
at once which will block without any output until evertyhing is done
(android runner runs tests in batches of 256 IIRC).
Also make suite arg positional and explicitly list supported tests.
Bug: angleproject:6854
Change-Id: Ibe108f24d6c3ec8b966eda1cf9ae92119a517548
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3691048
Commit-Queue: Roman Lavrov <romanl@google.com>
Reviewed-by: Jamie Madill <jmadill@chromium.org>
|
|
|
b71c6427
|
2022-05-31T16:23:51
|
|
Add a wrapper script to run traces using android_helper.
Also add missing `appops set MANAGE_EXTERNAL_STORAGE` that appears to be
necessary on newer builds (catapult does it starting from Android 11).
Add a hash check to skip transferring .gz files that are already there.
Bug: angleproject:6854
Change-Id: Ib612d1235fe1274b8fea47718af8389e8810d34e
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3679486
Reviewed-by: Jamie Madill <jmadill@chromium.org>
Commit-Queue: Roman Lavrov <romanl@google.com>
Auto-Submit: Roman Lavrov <romanl@google.com>
|
|
|
c9794d6c
|
2022-04-27T14:20:47
|
|
Run a smoke test in each shard and crash on timeouts.
The smoke test (words_with_friends_2) typically takes only a few seconds
to run. Run it first to check that everything is working as intended. As
we know it's a short test, we can set a small timeout (60s) and fail
early in case something goes wrong. This will speed up tests failures
when something is completely broken.
I also changed the main shard run to crash when timeout is detected
instead of just dumping debug info as if the tests didn't finish in 10
minutes they're also unlikely to finish in the 20 minutes after which
swarming times out the shard.
Bug: angleproject:7242
Change-Id: Ia99ca4b2e1676d09b9ff7d67ab116aba94f66512
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3611446
Reviewed-by: Jamie Madill <jmadill@chromium.org>
Commit-Queue: Roman Lavrov <romanl@google.com>
Reviewed-by: Cody Northrop <cnorthrop@google.com>
|
|
|
0c59b5d9
|
2022-04-26T19:20:49
|
|
Log debuggerd $PID output when instrumentation gets stuck.
Bug: angleproject:7242
Change-Id: I2b12e94177bce65acf68e4420fd1fcabffdb993e
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3607499
Auto-Submit: Roman Lavrov <romanl@google.com>
Reviewed-by: Yuxin Hu <yuxinhu@google.com>
Commit-Queue: Yuxin Hu <yuxinhu@google.com>
|
|
|
d0c8363e
|
2022-04-26T11:51:47
|
|
Dump logcat if instrumentation doesn't finish in 10 minutes.
Let's see if this helps us understand stuckness as described in
https://anglebug.com/6854#c46
Normally each instrumentation call takes way less than 10 minutes as we
run tests individually or in small batches. When stuck, runner times out
after 20 minutes.
Bug: angleproject:6854
Change-Id: I20074db0fb3e783555a530b9ecef3765f1911fd9
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3605763
Commit-Queue: Roman Lavrov <romanl@google.com>
Reviewed-by: Jamie Madill <jmadill@chromium.org>
|
|
|
c0ac374f
|
2022-04-25T14:06:58
|
|
Replace `adb exec-out cat` with pull to temp location.
Flaked here:
https://chromium-swarm.appspot.com/task?id=5a7a6cbff20e8510
File "/b/swarming/w/ir/src/tests/py_utils/android_helper.py", line 86, in _ReadDeviceFile
assert len(out) == expected_size, 'exec-out mismatch'
AssertionError: exec-out mismatch
Bug: angleproject:6854
Change-Id: I846474cb64c436a67bbc559839d41967108be8c1
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3606328
Commit-Queue: Roman Lavrov <romanl@google.com>
Reviewed-by: Jamie Madill <jmadill@chromium.org>
|
|
|
d1788614
|
2022-04-22T15:52:16
|
|
Use adb instead of catapult to run perf tests on Android.
Add support for replacing flags:
--isolated-script-test-output
--isolated-script-test-perf-output
--render-test-output-dir
Always set --isolated-script-test-output to check for test failures
using num_failures_by_type, interrupted, is_unexpected fields.
Bug: angleproject:6854
Change-Id: Ia701f4d4d5d1adc9a24c057bb5c58fd5457cbd65
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3602834
Reviewed-by: Jamie Madill <jmadill@chromium.org>
Commit-Queue: Roman Lavrov <romanl@google.com>
|
|
|
e6fc5e62
|
2022-04-22T12:45:33
|
|
Use _FindAdb (cached) directly instead of Adb wrapper.
A little simpler, no need to pass around an additional `apk` arg.
Also fix missed _GetAdbRoot() call.
Bug: angleproject:6854
Change-Id: Id64e5f35fb21a6aef7d99e614dc0eae753c24317
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3600912
Reviewed-by: Jamie Madill <jmadill@chromium.org>
Commit-Queue: Roman Lavrov <romanl@google.com>
|
|
|
d988db39
|
2022-04-22T11:14:44
|
|
Fall back to adb on PATH if platform-tools not present.
Log adb --version. Example:
INFO:root:adb --version: Android Debug Bridge version 1.0.41
Version 31.0.3-7562133
Installed as /{snip}/third_party/android_sdk/public/platform-tools/adb
When not found -> "Installed as /usr/bin/adb"
Bug: angleproject:6854
Change-Id: I3c19c4eda2211480f2c93b7d14f72f5641b3bfc2
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3602821
Reviewed-by: Jamie Madill <jmadill@chromium.org>
Commit-Queue: Roman Lavrov <romanl@google.com>
|
|
|
7c091cb3
|
2022-04-19T21:12:35
|
|
Use adb directly (instead of catapult) in gold tests.
Android detected by angle_perftests apk presence relative to pwd.
All of the restricted_traces/*/*.json files are copied but those are
small. Only the necessary .angledata.gz files are copied.
angle_system_info_test is also handled via adb.
Bug: angleproject:6854
Change-Id: I7a89ff57fcdd8ce5dc63a5e3a8f5c0132f766894
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/3595241
Reviewed-by: Jamie Madill <jmadill@chromium.org>
Commit-Queue: Roman Lavrov <romanl@google.com>
|