macOS-use lets AI agents operate a MacBook by launching and interacting with any app and UI component on the machine. You describe a task in natural language, such as opening the calculator and computing a result, or signing in to a website through Google auth, and the agent carries it out and calls done when finished.
The project is a port of Browser Use's agent approach to the desktop and is published on PyPI as mlx-use. It runs against OpenAI, Anthropic, or Gemini models through an API key. The stated long-term goal is an agent for Apple's MLX framework that runs local, private inference through mlx and mlx-vlm on any Apple device, including iPhone and iPad.
The project is at an early stage: it can use stored credentials and auth services to complete tasks, does not stop at CAPTCHAs or other bot checks, and is not intended to run unsupervised. Success rates vary by task prompt.
Features
- Cross-app control: launches and drives any macOS app and UI element from a natural-language task
- Model providers: OpenAI, Anthropic, and Gemini APIs
- Installed-app discovery: checks which apps exist on the machine and resolves actual app names when a first attempt fails (for example, Calendar versus iCal)
- Example tasks: calculator arithmetic, web login through Google auth, and looking up information online
- Roadmap: improved self-correction, an action for asking the user for input, local inference with a small fine-tuned model, and iPhone/iPad support
