Agent Device

agent-device gives an AI coding agent a feedback loop against the running app. One runtime is exposed three ways — a command-line tool, a stdio MCP server, and a typed Node.js client — so an agent can inspect, control, debug, and verify apps on iOS, Android, and HarmonyOS simulators, emulators, and physical devices, plus tvOS, Android TV, Amazon Vega OS, web, macOS, and Linux.

Agents read accessibility snapshots instead of reasoning over screenshots alone. A snapshot assigns stable refs such as @e2 to elements, and commands act through those refs or through semantic selectors matching text, label, or role. Commands run with --settle return a diff of what changed, so a fresh snapshot is only needed when the diff omits something. Screenshots, video, logs, traces, network data, performance samples, crash details, and React profiles are captured as evidence for review.

Device state is kept in sessions scoped to the caller's git worktree, and host-local device claims stop parallel agents from taking each other's simulators and emulators; ownership is inspected without a daemon and stale owners can be released. A working run can be saved as an .ad replay script for local use or CI, or exported as strict Maestro YAML, and the same commands drive hosted devices through a remote proxy or a device cloud such as BrowserStack, AWS Device Farm, or Limrun.

Underneath, commands are sent to XCTest on iOS and tvOS, ADB and a snapshot helper on Android, HDC and ArkUI uitest on HarmonyOS, the Vega CLI on the Vega Virtual Device, a local helper on macOS, and AT-SPI on Linux; support depth varies by target, and a capabilities command reports what a given platform supports. Web support runs agent-browser inside the same session and replay system. Node.js 22.12 or newer is required, and web automation needs Node.js 24 or newer.

Features

• Accessibility snapshots: the app's accessibility tree is exposed with stable refs, in place of raw screenshots
• Semantic actions: press, fill, scroll, gesture, wait, assert, and alert handling, addressed by ref or selector
• Settle diffs: a command can wait for the interface to settle and return only what changed
• Evidence capture: screenshots, video, logs, traces, network data, performance samples, crashes, and React profiles
• Replay scripts: an exploratory run is saved as an .ad script to rerun locally or in CI, or exported as Maestro YAML
• Session and device ownership: sessions are scoped per git worktree and device claims prevent parallel agents colliding
• Three entry points: the CLI, an official stdio MCP server, and a typed Node.js client over the same runtime
• Broad target list: iOS, Android, HarmonyOS, tvOS, Android TV, Vega OS, web, macOS, and Linux, including React Native, Expo, and Flutter apps
• Remote devices: a remote proxy and device-cloud connections route the same commands to hosted devices