Android 測試

Android測試隨著時間變化，很多工具被廢棄，也有許多新的工具發(fā)展。

AndroidTest.png

AndroidTest2.png

簡單地介紹以下其中幾種比較重要或者常用的android 測試工具:

• Mockito: Mock android 平臺相關(guān)依賴, 單元測試.

• Robolectric: Run android tests inside the JVM on your workstation. Android 程序是跑在 Dalvik 虛擬機(jī)上的， 直接實現(xiàn)了android.jar的api， 參考這里

• MonkeyRunner: Like appium, use hidden api. Chinese doc

• [UiAutomator1]: 給UI做黑盒測試， 進(jìn)程和和app進(jìn)程獨立, 寫測試用例打包成jar， push到android設(shè)備內(nèi)使用uiautomator runtest 你的jar.jar跑

• ATSL: Android Test Support Library

  • Espresso
  • UiAutomator2

Instrumentation test

Instrumentation test現(xiàn)在是android測試的標(biāo)準(zhǔn)。
UIAutomator1也遷移到Instrumentation based這個框架下。

那什么是 Instrumentation test呢？

文檔自然是最吼的， 但是如果不愿意看文檔， 可以繼續(xù)往下看本篇文章

What's instrumentation

• Official doc
• Chinese doc
• Chinese doc

App 創(chuàng)建時都會實例化一個Instrumentation對象, 在Activity之間共享實例， 控制Activity的生命周期, 使得Activity與控件的生命周期不需要被系統(tǒng)控制, 并且可以隨意控制activity的創(chuàng)建與銷毀。

如下面的代碼:

    // Start the main activity of the application under test
    mActivity = getActivity();

    // Get a handle to the Activity object's main UI widget, a Spinner
    mSpinner = (Spinner)mActivity.findViewById(com.android.example.spinner.R.id.Spinner01);

    // Set the Spinner to a known position
    mActivity.setSpinnerPosition(TEST_STATE_DESTROY_POSITION);

    // Stop the activity - The onDestroy() method should save the state of the Spinner
    mActivity.finish();

    // Re-start the Activity - the onResume() method should restore the state of the Spinner
    mActivity = getActivity();

    // Get the Spinner's current position
    int currentPosition = mActivity.getSpinnerPosition();

    // Assert that the current position is the same as the starting position
    assertEquals(TEST_STATE_DESTROY_POSITION, currentPosition);

幾句話概括以下：

Android instrumentation is a set of control methods or hooks in the Android system.

These hooks control an Android component independently of its normal lifecycle.

They also control how Android loads applications.

如何指定instrumentation?

在一個測試apk的 manifest中， 聲明 targetPackage 和 name， 然后打包編譯后使用
adb shell am instrument ...命令啟動測試apk， 然后android系統(tǒng)會做兩件事:

1. 將 targetPackage app進(jìn)程重啟
2. 然后實例化 name聲明的 instrumentation類， 創(chuàng)建instrumentation線程

然后測試代碼跑在instrumentation線程中， 并且可以控制、操作app進(jìn)程、app上下文來實現(xiàn)自動化測試， 這就是 Android instrumentation test.

<?xml version="1.0" encoding="utf-8"?>
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
      xmlns:tools="http://schemas.android.com/tools"
      package="com.example.android.testing.uiautomator.BasicSample.test"
      android:versionCode="1"
      android:versionName="1.0">
    <uses-sdk android:minSdkVersion="18" android:targetSdkVersion="28" />

    <instrumentation android:targetPackage="com.example.android.testing.uiautomator.BasicSample"
                     android:name="androidx.test.runner.AndroidJUnitRunner"/>

    <application tools:replace="label" android:label="BasicSampleTest" />
</manifest>

Espresso VS UiAutomator v2 VS UiAuotomator v1

Refrences:

1. Uiautomator1 vs espresso (Remeber the answers are talking about uiautomator v1, in 2016)

2. Uiautomator2 vs 1

3. Espresso

   Each time your test invokes onView(), Espresso waits to perform the corresponding UI action or assertion until the following synchronization conditions are met:
   
   The message queue is empty.
   There are no instances of AsyncTask currently executing a task.
   All developer-defined idling resources are idle.
   By performing these checks, Espresso substantially increases the likelihood that only one UI action or assertion can occur at any given time. This capability gives you more reliable and dependable test results.
   

   • Chinese doc

Below are some snippets:

1. A key benefit of using Espresso is that it provides automatic synchronization of test actions with the UI of the app you are testing. Espresso detects when the main thread is idle, so it is able to run your test commands at the appropriate time, improving the reliability of your tests. This capability also relieves you from having to add any timing workarounds, such as Thread.sleep() in your test code.
   The Espresso testing framework is an instrumentation-based API and works with the AndroidJUnitRunner test runner.
2. UiAutomator is powerful and has good external OS system integration e.g. can turn WiFi on and off and access other settings during test.
3. Based on uiAutomator
4. Espresso is targeted at developers, who believe that automated testing is an integral part of the development lifecycle. While it can be used for black-box testing, Espresso’s full power is unlocked by those who are familiar with the codebase under test.

UiAutomator1.0 use uiautomator runtest yourtest.jar cmd.

While uiautomator use adb shell am instrument -w -r -e debug false -e class ...

# special case pre-processing for 'runtest' command
if [ "${cmd}" == "runtest" ]; then
  # Print deprecation warning
  echo "Warning: This version of UI Automator is deprecated. New tests should be written using"
  echo "UI Automator 2.0 which is available as part of the Android Testing Support Library."
  echo "See https://developer.android.com/training/testing/ui-testing/uiautomator-testing.html"
  echo "for more details."

Appium vs ATSL(Android native test)

Appium is based on ATSL now. UiAutomator2 is widely used now. The support for espresso is released, but due to espresso is more suitable for white box test, so it's not

1. Speed:

   • Espresso is suitable to cover simple cases, unit tests. It's much faster than uiautomator, and appium. (About 7 times faster than robotium).
   • UiAutomator is faster and than appium, cause http requests take time.

2. Ability:

   • Appium has access to the server connecting android devices, so you can call cmds like adb install when running cases. While you can not install or uninstall apps when using uiautomator directly. With andro's helper, you can install/uninstall apps at the beggining of a case, or at the end of a case. While all adb shell commands are available in native test.
   • Appium is not suitable to write grey-box test, also you can access android methods through reflection, but without code base of android, it's hard. While with native-test, you're free to get the context of app, activities, and call special lifecycle methods in activities.
   • With appium, you can use almost all language to write your tests. While with the native test, you can only write with kotlin/java

3. Stability:

   • Appium use uiautomator-server apk and uiautomator-server-test apk. It should start a tcp port on the device, and recieve commands to controll the device. Compared to android test which is maintained by google team, appium is not as stable as native test, but acceptable.
   • Cases wrote in espresso don't need to poll the state of ui, so they are more stable. While comparing cases wrote in appium with cases wrote in uiautomator, they are almost the same except for that appium use network connection, which is not reliable.