AgentGUI is a desktop interface for observing and steering a team of AI agents, drawn as a scrollable pixel-art office. Each task is a desk with an agent sitting at it. Opening a desk shows its live activity feed, file tree, terminal, and full debug messages, and you can redirect or reassign agents mid-task without leaving the interface. It works with Hermes agents, importing existing Hermes profiles, with experimental support for agents built on the Claude Agent SDK.
Each desk has a task description that an autonomous manager uses to audit the agent's trace while it runs, marking progress and steering the agent back when it drifts. For review, a run breakdown shows reasoning, tool calls, and idle stretches over time, a debug terminal shows the sandbox shell input and output, and saved trajectories can be reloaded and replayed later.
Agent terminal actions run inside Docker sandbox containers, while workspace files, task descriptions, conversation history, and memories stay on the host. You can customize each agent's persona, model, and run-time tool scope, and share files across a team. Inference can use the Gemini or Claude APIs or run fully local through Ollama or vLLM. The project is described in a research preprint accepted to EMNLP 2026.
Features
- Office view: each task is a desk in a pixel-art office showing the agents working in parallel
- Desk inspection: live activity feed, file tree, terminal, and full debug messages per agent
- Steering: redirect, interrupt, or reassign agents mid-task from the interface
- Autonomous manager: audits each agent's trace against its task description and steers it back on track
- Run breakdown: a time-resolved view of reasoning, tool calls, and idle stretches
- Saved trajectories: reload and replay a past session for review
- Sandboxed execution: agent shell commands run in Docker containers, with work files kept on the host
- Team setup: per-agent personas, models, run-time tool access, and shared team files
- Hermes import: existing Hermes agent profiles appear automatically
- Local or cloud inference: Gemini and Claude APIs, or local models through Ollama and vLLM
- Subagent visualization: subagents and research task templates in the interface
