in Updates |

LLM Based Android UI Testing – “Journeys with Gemini”

What Do We Know About “Journeys With Gemini”?

“Journeys with Gemini” was first (to my knowledge) discussed publicly at Droidcon London during a segment of #TheAndroidShow by Tor Norbye of the Android Studio team. We saw a sneak peek of a new way to write UI tests for Android using Google’s Gemini (AI Model) that takes a text based set of human readable steps and programmatically executes a UI test on your application.

We also heard a few more details about it at the VERY end (~31 mins in) of another Droidcon London talk “Scalable Testing Strategies”.

How and Where are Tests Defined?

  • Tests are defined in .journey.xml files in the tests appear in a view similar to the typical Android Test Runner and end up resolving in the androidTest source set.

File Format – What’s in a .journey.xml File?

It’s XML based file that contains the following elements to represent a test prompt.

  • Name
  • Description
  • Prompt Steps
    • Action
    • Assertion

How are Tests Run?

From what we see, it is all just IDE based tooling that looks similar to what we have today.

What’s the Editor Experience Like?

We see basic list behavior to add actions and assertions to the list. Additionally I spied a “drag” to reorder icon as well as a delete icon.

Journeys in Android Open Source (AOSP)

There is currently the shell of a Gradle Plugin and JUnit 5 runner. These have no implementation detail in them, so are API placeholders at this point. Link: https://cs.android.com/android-studio/platform/tools/base/+/mirror-goog-studio-main:journeys/

Here are the placeholder implementations at this point.

JourneysTestEngine.kt
JourneysGradlePlugin.kt

My Questions 🧠 (and Predictions) 🔮

When will the Journeys Feature be Available?

  • My Prediction: Early 2025.
    • Since they have already teased it publicly, and they’ll want feedback before any sort of release for Google I/O 2025 which will most likely be very AI/Gemini heavy like it was in 2024.
    • Historical release dates for recent Canaries of Android Studio:
      • “Meercat” (Nov 12th, 2024)
      • “Ladybug” (July 15, 2024)
      • “Koala” (May 14, 2024)
      • “Jellyfish” (December 28, 2023)

Do Running Journeys Require Connectivity to Gemini Cloud-based Services?

  • My Prediction: Yes.
    • This is still something evolving and because Gemini Nano only works on a limited set of hardware, I don’t know of any on-device Gemini solution for this. I would LOVE if there was one, but my current guess is that it’ll require connectivity

Why is there a Gradle Plugin?

  • My Prediction: Custom Execution using JUnit 5.
    • Currently Android UI tests only support JUnit 4 officially, and this is clearly using JUnit 5. Google would either have introduce official JUnit 5 support for Android (that would be pretty sweet), or have a different way of running these tests.

Will “Journeys” use Espresso or UI Automator?

  • My Prediction: Espresso.
    • Google’s been pushing Espresso a long time, and it’s great for “developer” tests. They could be using UI Automator instead, but I haven’t seen them pushing materials about UI Automator a lot. It would be cool to support both. I guess we’ll see.

What Will the Journeys New “Test Recorder” Look Like?

  • My Prediction: A lot like the old one.
    • Currently there is only “Action” “Assertion” prompts, and you’d probably click through the app to record it.
    • Bonus: I had a small “course” on Espresso Test Recorder on caster.io before it shut down. Luckily I was able to move the videos to YouTube at that time, so they are still available.
NOTE: This playlist has OLD Content (Most likely outdated)

Will this support iOS?

  • My Prediction: No.
    • While the Android Developers team is working hard to have Jetpack Libraries support Kotlin Multiplatform, but this feature looks like it’ll have a strong tie to Android Studio.

Will Journeys in Android Studio support non-Gemini LLMs?

  • My Prediction: No.
    • Android Studio is made by Google and so is Gemini. The tight integration gives them the ability to make a powerful toolkit that works well together. Based on previous observation (like Firebase Test Lab and Firebase Crashlytics integrations), they only support their own products natively.

Will it be possible to use Journeys with non-Gemini LLMs?

  • My Prediction: Yes. (but with non-Google Open Source)
    • This won’t be something Google officially supports, but it’s an XML based format with open sourced tooling. That means it is easy for others to add that support onto it.

Related work in this space

The idea of Prompt/LLM driven test execution isn’t totally brand new, but it’s still a very evolving space. Related content I’ve seen:

Conclusion (and Final Prediction)

It’s really exciting to see Google doing this. I think it’s where the industry is heading with LLMs and agents. I wish this product could support multiple LLM providers and iOS, but I’m pretty skeptical that it’ll be there based on the amount of investment that would be needed to build and support that. It just wouldn’t be Google’s best use of budget, but I do hope they build it in a way that others can extend it easily if they desire.

Find me on Bluesky Social and start a conversation on this.

Disclaimer: All the information about Journeys is publicly available through videos and open source. I have no insider knowledge on this product. I wrote this post because I saw it in a recording of Droidcon London and wanted to figure out as much as I could.

Published

Updated