Parameterized tests allow you to write a test once, but allow it to be called with multiple parameters. This means it is creating more methods than you’ve written, but without you having to write and maintain each one individually.

Here is a trivial example of writing a single test with TestParameterInjector, but having it run twice with true and false.

@RunWith(TestParameterInjector::class)
class MyTest {

    @Test
    fun test(@TestParameter isOwner: Boolean) {
        // Your test logic here
    }
}

This example results in the following tests being run:

• MyTest#test[isOwner=true]
• MyTest#test[isOwner=false]

TestParameterInjector has been the defacto way to do this on the JVM and is still a good solution for those projects. The original Burst 1.x project was archived and pointed users to use TestParameterInjector.

Burst 2.0 bursts onto the Scene!

Fast forward to October 2024 and Kotlin Multiplatform in full swing. There is no solution for parameterized tests for Kotlin Multiplatform, so in order to solve the problem, Burst 2.0 was created as a Kotlin Compiler Plugin.

It Works on Android! 🎉

While Kotlin Multiplatform support is fantastic, my biggest need at work is to use it for Parameterized Android Instrumentation tests, so I tried it out on my open source project ShoppingApp and here are my experiences.

It ran successfully via the ./gradlew :app:connectedDebugAndroidTest --info command.

Despite IDE support issues (red highlighting), the underlying library compiles and executes on the command line 😄 .

Executing the Android Tests with the IDE also causes failures for some reason even though it works.

The Problem of Massively Sharding Android Tests

TestParameterInjector has been a solution for Android Parameterized Tests but falls short when wanting to statically compute all tests methods for sharding purposes.

Cloud based device farms like Firebase Test Lab can run your tests on _ number of devices at once and combine the results. This is facilitated in many projects through Flank (Fladle – Flank Support in Gradle). The problem is that TestParameterInjector computes the test method names at runtime, and therefore these device farms can only get an entire class to run (which might end up having 100 permutations with parameterized tests).

Parameterized tests are powerful, reduce boilerplate and ensure exhaustive coverage. To unlock this capability and support massively sharding tests, I personally spent 2 weeks developing a solution that DOES work with TestParameterInjector to compute all test names via static analysis of the APK via this multi-step process described here:

This has been super helpful in allowing my team to run parameterized tests in a massively sharded device farm, but it is a complex solution (but runs in 3 seconds), and while there are tons of integration tests for the library to ensure it works, because it’s an involved set of steps, it still feels like it could fall over at some point.

I could open source this solution, but it has it’s limitations, and if we do have Burst 2.0 at our disposal now. I’m going to endorse their project and avoid having to maintain a project with a pretty equal feature set.

The big hurdle with TestParameterInjector and computing a deterministic list of tests statically (using limited features like enums, strings, etc) is hard because it’s determining the tests to run via the JUnit Runner itself at runtime, instead of when code is being compiled. This means the full list of methods are not in the final Test APK, but need to be loaded into JUnit and computed in a JVM runtime.

Burst 2.0 uses the same limitations as my solution (enums, booleans and list of strings), but does it at the Kotlin Compiler phase Because of this different method, computing all android tests to be run can be done via static analysis of the APK by tools like https://github.com/linkedin/dex-test-parser which computes a full list of methods from an APK. This also means that it will work in Firebase Test Lab and other sharding by method runners. Related Issue of TestParameterInjector -> https://github.com/google/TestParameterInjector/issues/27 to allow that functionality.

Kotlin Compiler Plugin

This what the resulting classes look like for a compiled class after getting processed by Burst.

Required Dependencies

Transitively brings in Kotlin 2.0.21 to your build classpath.

Conclusion

Burst 2.0 seems like the right solution for Android Developers to run parameterized tests and have it possible for sharded test runners to deterministically compute all test method names using static analysis.

If you are someone out there that really wants the implementation I have for TestParameterInjector, then reach out to me a threads.net/@handstandsam and I’ll see if I can share it in open source (but won’t maintain the library).

For now, to use Burst 2.0, you need Kotlin 2.0, but that’s already the case in the majority of projects since it was released as stable 6 months ago. You might have some IDE support issues for a bit, but I’m excited about Burst 2.0, and I’d encourage you to try it out on your Android projects!