YouYoung Seo
Android is not a single target. Hundreds of manufacturers, thousands of device models, multiple active OS versions, and custom UI layers that vary by brand. Add apps that combine native views, WebViews, and third-party SDKs in a single flow, and you have a testing environment that breaks assumptions fast.
Most Android test automation is built around one assumption: the app exposes a testable structure. When that structure is not there, or changes, there is usually no fallback.
A computer-use agent starts from a different assumption. The screen is enough.
How AskUI Runs on Android
AskUI deploys a computer-use agent that observes the Android screen, reasons about what it sees, and acts through OS-level input. The same loop a human tester runs, just automated.
The agent does not query the view hierarchy. It reads what is visible, which means it works on any Android surface: standard apps, React Native, WebView-heavy flows, system dialogs, and Android-based infotainment displays that expose no accessibility structure at all.
Tests are written in plain English as Markdown or CSV files. The agent reads them and executes directly on the device. No code translation required. It finds elements on screen the same way a tester would.
Where Android Testing Gets Complicated
Device and OS Fragmentation
Android runs on devices from hundreds of manufacturers, each with different screen sizes, hardware configurations, and custom UI layers on top of the base OS. Unlike iOS, Android updates are distributed by manufacturers and carriers independently, meaning active OS versions span years. A test suite that works on one device configuration is not guaranteed to work on another.
Algorithmic automation can only handle what it was programmed to expect. When the UI renders differently across devices, or a button label changes, or an unexpected popup appears, the test breaks. The agent reads the screen as-is, regardless of the device or OS version it is running on.
Android-Based Infotainment Systems
A growing number of vehicles run Android-based infotainment systems. These displays often render UI at the display level, outside the standard Android view hierarchy. No resource IDs. No accessibility tree.
Standard Android automation has no path into these surfaces. AskUI connects via AgentOS in Companion Mode: HDMI capture for the display, USB HID for input. The agent reads the screen and interacts through the same input path a finger or physical button takes.
A test for an infotainment system looks like this:
# Test: Verify route guidance starts correctly
## Preconditions
- System is on the home screen
- No active navigation session
## Steps
1. Tap the Maps tile
2. Search for Berlin Hauptbahnhof
3. Select the first result
4. Tap Start Navigation
## Postconditions
- Route preview is visible
- Estimated arrival time is displayedThe agent reads this the same way a tester would. It finds the Maps tile on screen and taps it. No resource ID needed.
Apps That Cross System Boundaries
Permission dialogs, system notifications, authentication handoffs, app-to-app transitions. These steps live outside the app's own context, and outside the reach of tools that instrument only the app under test.
The agent operates at the OS level. It sees the full screen regardless of which app or system component is in focus.
Mixed Architecture
Many Android apps combine native views, embedded WebViews, and third-party SDKs in a single flow. Each layer boundary is a potential failure point for script-based automation that requires every UI state to be defined in advance.
The agent does not distinguish between layers. It sees what is on screen and interacts with it.
What the Test Project Looks Like
Everything the agent needs lives in plain text files. The folder structure determines what runs and in what order.
├── prompts/
│ ├── device_information.md # Android device + OS details
│ ├── ui_information.md # app-specific concepts
│ └── report_format.md
├── procedures/
│ └── launch_app.md
├── plans/
│ └── regression.md
└── tests/
└── your_android_app/
├── setup.md
├── rules.md
└── login_flow.mddevice_information.md tells the agent what it is running on:
# Device Information
Target: Android 14, Samsung Galaxy A54
Display: 1080x2340, portrait
Input: touch
Connection: AgentOS host via ADBQA engineers, domain experts, and testers who know the application can write and maintain tests in plain text. No scripting or automation expertise required.
Performance
In AskUI internal evaluations on the public AndroidWorld benchmark (April 2026), AskUI with Claude Sonnet 4.6 reached 92% task completion. The human baseline on the same benchmark was 88%.
Source: AskUI internal benchmarks against the public AndroidWorld suite, April 2026.
Deployment
AgentOS connects the agent to the Android device. Two configurations:
Host mode connects AgentOS to the Android device via ADB from a machine on the same network. Standard for CI pipelines and device farms.
Companion mode runs AgentOS on a Raspberry Pi or mini-PC, connected to the Android device via USB HID and HDMI capture. Used for Android-based infotainment systems, locked-down devices, or environments where software installation on the target is not possible. The device stays untouched.
Same SDK, same tests, same files. Only the connection changes.
Common Questions About Android Agent Testing
What is Android agent testing?
Android agent testing is an approach to Android test automation where a computer-use agent, an AI system that observes the screen and acts through OS-level input, executes the tests. The agent works from what is visible on screen, not from the app's internal element structure. This makes it applicable to surfaces that traditional Android automation tools cannot reach.
Does AskUI require ADB access to test Android apps?
Not always. Host Mode uses ADB to communicate with the Android device. Companion Mode connects via USB HID and HDMI capture and requires no software installed on the Android device itself, making it suitable for locked-down or restricted environments.
Can AskUI test Android-based infotainment systems?
Yes. AskUI connects via Companion Mode, HDMI capture for the display and USB HID for input. The agent does not require the Android view hierarchy to be exposed, which makes it compatible with infotainment displays that render UI outside the standard accessibility tree.
Can AskUI handle Android apps that mix native views and WebViews?
Yes. The agent operates at the screen level and does not distinguish between rendering layers. It interacts with whatever is visible on screen, regardless of whether it is native Android, an embedded WebView, or a third-party SDK component.
How are Android tests written with AskUI?
Tests are plain Markdown or CSV files describing preconditions, numbered steps, and expected outcomes. No instrumentation setup required.
Can the same tests run across multiple Android devices in parallel?
Yes. Each device registers as a separate target in AgentOS. Tests can be distributed across devices and run in parallel from the same test repository.
How does AskUI handle Android device fragmentation?
The agent reads the screen directly rather than querying device-specific element structures. The same test file runs on a Samsung Galaxy, a Pixel, or an automotive display. The agent adapts to what it sees on screen.