summaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2016-07-13Install required APKs for UI tests before tests start.android-wear-7.1.1_r1android-n-preview-5android-n-iot-preview-2nougat-mr1-wear-releasen-iot-preview-2Xin Chen
UI tests used to install APKs programmatically during tests, which is not stable as observed due to security issues. Besides, that way also deviates from manual test cases. Instead, we will install APKs before tests and deprecate the programmatical method to install an apk in test runtime. Change-Id: I059265ffcd9da7ac949506cf7467779b8503cf46
2016-07-13Merge "Update ShellUtilTest with new UiAutomator API."android-wear-n-preview-2Xin Chen
2016-07-13Update ShellUtilTest with new UiAutomator API.Xin Chen
UiDevice adds executeShellCommand method to run shell commands. We use the API instead of ShellUtil fixture in the framework, and ShellUtil fixture will be deprecated once we migrate all usage. Change-Id: Ib90afb585c482def56038e8d2a88d08b4bf2be80
2016-07-13Merge "Switch nyc system image branch to nyc-emu-dev from nyc-emu-release"Ying Chen
2016-07-12Switch nyc system image branch to nyc-emu-dev from nyc-emu-releaseYing Chen
Change-Id: Ic3de4a7c6cca402733b468897453b7ad6129d93b
2016-07-12Merge "Fix the ResId of Setting's list"Xin Chen
2016-07-12Increase the time limit of VpnTest to 90 secondsXin Chen
VpnTest takes a bit more than 60 seconds sometimes. Change-Id: I942bfc4e90c82487ae0f1c5d5449b08b54a3e492
2016-07-12Fix the ResId of Setting's listXin Chen
Change-Id: Iedd49c4f93b13df7ad59de91df38292d76581247
2016-07-11Create sdcard.img for AVDYing Chen
- sdcard.img is required to run ui tests Change-Id: Ia10ae0a9a1082331aa6013d31f05f1c47f2138b1
2016-07-08Change AddGoogleAccountTest to get API info from testing frameworkXin Chen
Change-Id: I5f35d8d68d6440eec620b6b6929f5b3c8027d008
2016-07-07Fix several bugs in framework fixtures and tests.Xin Chen
... based on buildbot results on x86, api23, default system images. Change-Id: I878903976db9aeb86ab8f8f4a962f456195f0ed0
2016-07-07Update UI test recipe to test emulator with host GPU only.Xin Chen
Change-Id: I7b4dd9f8e017582bae16bb298cc379d85421eb7b
2016-07-07Disable UI tests on Google APIs, API 24.Xin Chen
Change-Id: I1823d8fbfe0430863d0b9766eb5b79d742ffc0f7
2016-07-06Update Android build tool version to 23.0.3Xin Chen
Change-Id: I434def8ffba823c57960036faddd53279c85cde2
2016-07-06Merge "Change the Android buildtool version of UI tests."Xin Chen
2016-07-06Change the Android buildtool version of UI tests.Xin Chen
Change-Id: I4f28a8b82bccfaa279280e118f3bee93aaca4864
2016-07-06Define ANDROID_HOME for ui tests.Ying Chen
Change-Id: I08614b2a195e43a88520848b8d584ced25bde243
2016-07-06Fix a bug that UI tests failed to upload logs.Xin Chen
Change-Id: I020f55f71b14626f41fe568359af9ef97d720b69
2016-07-01Download system image builds before each stepYing Chen
- The purpose is to save disk space - Always clean up system image directory and only download before test step Change-Id: If01f9435a47ccab1d4bdbea7f6a94ed72217e7b4
2016-07-01Merge "Add a new category and builder for ui test"Ying Chen
2016-07-01Add system image UI tests into adt recipe.Xin Chen
Also refactor some string and remove dead code in test_ui. Change-Id: I88472223812c1090957a3ad1235f0851847f918d
2016-06-29Add a new category and builder for ui testYing Chen
- Create ui scheduler that only watches for emu-master-dev. - Add one linux builder for ui test. - This CL starts only adds single triggering branch and single linux builder, we will add more branches and builders when it's stable. Change-Id: I24db4b2a4c29b05d53863b0830a52e30b5b3ebfc
2016-06-28Add two Network IO system image UI tests to verify toggling of cellular data.David
Change-Id: I08eb8c5ff4fc637825dff7a22a98f1800c15a5b7
2016-06-28Fix for showing system apps list for APIs <= 22Pallavi Angampally
Change-Id: I47e14d521044da9859022c7e9c1be32d22a04fe7
2016-06-27Merge "Add two Date and Time system image UI tests for enabling time zone ↵David Yu
and 24-hour format."
2016-06-25Merge "Remove image files after unzip to save disk space"Ying Chen
2016-06-24Remove image files after unzip to save disk spaceYing Chen
Change-Id: I8fb6db737436fbf421e5552bc2737cf72dd4f1ca
2016-06-24Add two Date and Time system image UI tests for enabling time zone and ↵David
24-hour format. Change-Id: Ic303c092fb004d6dafe27f7c1c7f35ce5301b044
2016-06-24Manual modification to the results from tests run 2016-06-21_19-05-55Jim Kaye
Those results used only 3 runs of CTS tests, which is not enough data to conclude 'gotBroken' or 'gotFixed'. This CL changes those two designations to 'flaky', which is the best conclusion with such a small sample. Summary of the information in this file: 751 Bad 347 Flaky 0 gotBroken ------ 1098 Total bad 0 gotFixed 119881 Good ------ 120979 Tests that were run 6036 Not executed ------ 127015 Total tests Change-Id: I5def124e28d07d5f29913d5cbad551e508b6fa3e
2016-06-23Fix typo in CTS plan repository pathDave Detlefs
My previous CL https://android-review.googlesource.com/#/c/241410/ introduced code to exclude certain CTS tests. While I tested the translation locally before submitting, there was an error that was only revealed last night, when the code ran on the BuildBot machine: I got the directory in which the CTS plan files live wrong ("plan" --> "plans"). This should fix that. Change-Id: I09e8c0281fcdc8fc7320eb9691e57f640758b57d
2016-06-22Merge "Update flakiness results based on tests run 2016-06-21_19-05-55"Jim Kaye
2016-06-22Update flakiness results based on tests run 2016-06-21_19-05-55Jim Kaye
Until a few days ago, CTS tests were not running to completion. These results are based on the few tests that were run after that issue was resolved: 2016-06-19_10-46-23 build_0-rev_2988904 2016-06-20_17-31-08 build_3-rev_2991947 2016-06-21_19-05-57 build_6-rev_2997187 Summary of the information in this file: 751 Bad 127 Flaky 63 gotBroken ------ 941 Total bad 157 gotFixed 119881 Good ------ 120979 Tests that were run 6036 Not executed ------ 127015 Total tests I have changed the format of this text file. The changes should make it easier to compare revisions *in the future*. (But the changes make it very difficult to compare with versions prior to this one.) The format changes are: 1) Put the flakinessResult on the same line as the fullName. This helps keep 'diff' tools sync'd. 2) Alphabetize using sqlite3's sort order. (Rather than picking some arbitrary sort tool each time.) Change-Id: Id3507787183c4d640e51f55c295fd92adb823a33
2016-06-22Allow emulator CTS tests to be excludedDave Detlefs
Some CTS tests have been found to have persistent negative side-effects: they may either crash the emulator, or leave it in a state of diminished performance. In order to get useful information from the rest of the test run, this CL allows us to specify tests to exclude. The set of tests to exclude is expressed as a data structure in the test_cts.py file. The mechanism we have available is that a CTS test plan file specfies a set of test packages in the plan, but also allows you to specify individual tests to be excluded. The data structure is a dictionary mappin plan names (e.g., "CTS") to another dictionary, this one mapping test package names (e.g., "android.calllog") to a sequence of excluded tests in that package (e.g., "android.calllog.cts.CallLogBackupTest#testSingleCallBackup"). When we're asked to execute a test plan, we check whether that plan has exclusions in this data structure. If so, then we create a new test plan, derived from the .xml for the original. We assume the original plan has no exclusions. We copy line by line; if the line specifies a test package with exclusions, we insert those exclusions in the output. We direct cts-tradefed to execute the newly-created test plan. I've tested (via the test code now commented out in the file) that the translation code produces the desired output: for the current CTS.xml, and the exclusions given in the CL, it produces the desired output: a CTS-emu-stable.xml file that differs from the original only by the added exclusions. If there are suggestions on how to test further, I would welcome them. Note: I also eliminated what seemed like a redundant second call to create_test_case_for_avds(). If I'm missing a reason this might be necessary, let me know. Also, I'm not completely fluent in python, so feel free to critique coding style decisions. Change-Id: I57c0154439760fac4019adf5aeaa202031cb49e4
2016-06-21Merge "Edit enableSetDateAndSetTime to intilize environment UI settings ↵David Yu
prior to test running."
2016-06-20Edit enableSetDateAndSetTime to intilize environment UI settings prior to ↵David
test running. Change-Id: Ic91053485f7fdcb190c6b1fc7115baabcee55ba4
2016-06-20Increase timeout for SDK updateYing Chen
- Sometimes it takes longer than 30 minutes to complete update, especially when there are updates in system images. Change-Id: I679a3c5e7f4b78fae26dde906178425ffe8e28f8
2016-06-20Merge "Add systems UI test for Now Cards confirmation page."David Yu
2016-06-20Add systems UI test for Now Cards confirmation page.David
Change-Id: Ie071cf782bd7b5305c12fd4bf5dee89455f57703
2016-06-20Revert "Don't remove tmp files for Windows."Ying Chen
This reverts commit 81435e862497e36cc76ef17814764d97def66bd5.
2016-06-17Change system directory and sdk filter for nyc images,Ying Chen
use "android-24", "android-N" has become obsolete. Change-Id: Idcd682dcad8a002e6b4c5c1beef01fc32abdffc9
2016-06-17Reduce length of workdir to avoid "paht too long" problemYing Chen
on Windows builders Change-Id: I3fd1606e0db263fdec1e7723affc04a60448536c
2016-06-17Don't remove tmp files for Windows.Ying Chen
- Observed some weird issue after deleting tmp files, Need to revert this change after figure out the reason Change-Id: Ic4703ea5e824f84c7c51c6315a897e8002771e26
2016-06-17Calculate adb push/pull speed by scriptYing Chen
- Print out from adb changed since adb version 1.0.36 Calculate the speed by script, to remove dependency on adb Change-Id: Ib08d103a5f0c8672f1045ff3185576bfe2953886
2016-06-17Run CTS on Ubuntu 14.04 Quadro, run complete test configYing Chen
on Ubuntu 15.04 Change-Id: I802ef17d79e706e589f50191a5b4fe5616f802f5
2016-06-17Fix regex of CTS avd name so local copies will not return a matchYing Chen
- When running local debug, we sometimes need to make a copy of CTS AVD, and append suffix to the orignal name. this change avoids running CTS on such AVDs Change-Id: I887605cd93c710792c070263d0952791f5694a45
2016-06-17Replace emu-2.0-release with emu-2.2-release branchYing Chen
Change-Id: I25afaa7990f0f45b68b98b178fe6290b5dd64ac1
2016-06-17Add system image UI test to verify enabling Developer options link.David
Change-Id: I26d27a00e463b20d068e72f9540d1b225b4b0a5f
2016-06-17Refactor resource-id identifiers and reorganzie Res class.Xin Chen
Organzie Res class into three categories for better management. Refactor existing resource-id identifiers to force the rule that all resource-ids must be acquired from Res class. Change-Id: I4804f92f0f84ba70e3c25f066b743a56606c19d2
2016-06-17Add a systems image UI test for Set date and Set Time.David
Verifies Set date and Set time fields are editable. Change-Id: I9481f609bfbf16861a45e67eb2c083b55b8851e5
2016-06-17Merge "Adding a fixture to launch System app list."Pallavi Angampally
generated by cgit v1.2.3 (git 2.25.1) at 2024-05-26 04:05:20 +0000