E2BOpen Computer Use runs a Linux desktop in an E2B Desktop Sandbox and lets an open-source language model operate it through keyboard, mouse, and shell commands. The sandbox display is streamed live to the client machine, and you can pause the agent and give it a new instruction at any point. Ubuntu is the image used, though the design is not tied to it.
The agent splits the work across three models named in config.py: a grounding model that locates elements on screen, a vision model that reads the display, and an action model that decides what to do. Providers are swappable, and the ones included cover Fireworks, OpenRouter, and the Llama API for Llama 3.2 and 3.3, Groq, DeepSeek, Google Gemini 2.0 Flash, OpenAI GPT-4o and GPT-4o mini, Anthropic Claude, Moonshot, Mistral's Pixtral and Mistral Large, and the OS-Atlas and ShowUI grounding models hosted on Hugging Face Spaces.
Running it needs Python 3.10 or later, an E2B API key, and a key for whichever model providers the config selects; Hugging Face Spaces models need only a token to avoid Gradio rate limits. Starting the program opens a web interface that asks for a first instruction, or takes one on the command line, and the display stream appears a few seconds later.
Features
- Sandboxed desktop: the computer the agent drives is an E2B Desktop Sandbox rather than the host machine
- Direct control: the agent works through keyboard, mouse, and shell commands, not a scripted click path
- Three-model split: separate grounding, vision, and action models, each configured independently
- Swappable providers: more than ten models across Fireworks, OpenRouter, Llama API, Groq, DeepSeek, Google, OpenAI, Anthropic, Moonshot, and Mistral, with new providers added in one file
- Live display: the sandbox screen is streamed to the client while the agent works
- Interruptible: you can pause the run and prompt the agent mid-task
- Web interface: started with a Poetry script, optionally seeded with an initial prompt
