in Android, Kotlin, Testing |

[Experiment] Espresso Closed-Box Testing

I wanted to write some Android Espresso tests for a large application, but iterate on the tests as fast as possible.

Typically someone would run :app:connectedDebugAndroidTest to run their instrumentation tests, but under the hood that is just compiling and installing both the app and androidTest apks, and using the instrumentation runner over adb.

When executing Android Instrumentation Tests, you just need an app.apk and an androidTest.apk, and then to invoke the test instrumentation runner via adb.

Because of the configuration, the androidTest APK gets everything that is on the app‘s classpath so it can reference resources, classes and activities in the app.

The Experiment

I wanted to see if I could build an androidTest.apk without having any ties to the original :app. I tried a few methods, but found that creating a new blank application with the exact same package name, and then writing tests under the androidTest folder allowed me to compile quickly.

Problems:

  1. No access to the classpath & resource identifiers
  2. Classpaths can’t clash (must use same versions of dependencies as the original app).

Workarounds:

  1. You could import just a few modules that have resource identifiers or code that you want to reference in your tests. (easier and typesafe, but a little slower)
  2. OR you could just access everything by fully qualified package names, and look up resource identifiers by ID. (no compile time safety, but faster)

I tried workaround #2, because I wanted to have this be the fastest iteration time possible, and I finally got it to work! Here’s my receipt for how I made it happen.

How I Got it Working

1) Install my app (com.example.app) as usual :app:installDebug.

This will be the app I want to test.

2) Create the :cloneapp project

In this :cloneapp project, keep an empty main source folder, but add an androidTest directory.

3) In :cloneapp set the package name to the the exact same package name com.example.app.

android {
    defaultConfig {
        applicationId "com.example.app"
    }
}

4) In :cloneapp update the src/androidTest/AndroidManfest.xml

<manifest xmlns:android="http://schemas.android.com/apk/res/android"
      xmlns:tools="http://schemas.android.com/tools">
    <instrumentation
        android:name="androidx.test.runner.AndroidJUnitRunner"
        android:targetPackage="com.example.app"
        android:targetProcesses="com.example.app" />
</manifest>

5) Add in a test!

package com.example.app.tests

import android.app.Activity
import android.content.Context
import android.os.SystemClock
import android.util.Log
import androidx.test.core.app.ApplicationProvider
import androidx.test.espresso.Espresso
import androidx.test.espresso.ViewInteraction
import androidx.test.espresso.action.ViewActions
import androidx.test.espresso.assertion.ViewAssertions
import androidx.test.espresso.matcher.ViewMatchers
import androidx.compose.ui.test.junit4.createComposeRule
import org.junit.Before
import org.junit.Rule
import org.junit.Test

fun findResourceIntByIdStr(id: String): Int {
    ApplicationProvider.getApplicationContext().resources.getIdentifier(id, "id", applicationContext.packageName)
    Espresso.onView(ViewMatchers.withId(findResourceIntByIdStr(idStr)))
}

fun findViewByIdStr(idStr: String): ViewInteraction {
    Log.d(TAG, "Find View By ID Str $idStr")
    return 
}

class ExampleTest {

    /** Use this to interact with Compose surfaces */
    @get:Rule
    val composeTestRule = createComposeRule()

    @Test
    fun testLoginFlow() {
        
    }
}

6) Install the test clone APK

Run :cloneapp:installDebugAndroidTest to install the test apk.

7) Run the tests using adb!

adb shell am instrument -w -r com.example.app.test/androidx.test.runner.AndroidJUnitRunner

Note: You can be more explicit with command line instrumentation arguments about what test or test class you want to execute.

8) Test Development Iteration Loop

I ended up clearing the app data between runs with adb shell pm clear com.example.app as well so I had consistent behavior and didn’t have to install the package.

Conclusion

As mentioned, this was an experiment. It made the iteration time blazing fast, but lacked compile time safety. Anyways, it’s possible, and hopefully you learned something. If you end up using this technique, I’m curious to hear more. Feel free to message me on Kotlin Lang Slack or Mastodon.

Published