OpenLive

OpenLive is a desktop application that wraps a speech and vision loop around an AI model you supply. It handles voice activity detection, deciding when a speaker has finished, streaming speech-to-text, the model turn, streaming text-to-speech, and barge-in so a reply can be interrupted mid-word. Camera and screen frames ride along with each turn, and a look tool grabs a high-resolution frame on demand.

Everything except the model runs on the machine, in the renderer over WebGPU. Audio is not uploaded; what leaves the device is the transcript, plus camera or screen frames when those are enabled. The brain can be a provider key (Anthropic, OpenAI, Google, xAI, DeepSeek, Groq, Ollama and others) or a coding agent already installed locally — Claude Code, Codex, Cursor, OpenCode or Hermes — driven over the Agent Client Protocol as JSON-RPC on stdio.

With a coding agent as the brain, sessions stay in the agent's own store, so a call with Claude Code lands where claude --resume finds it and existing CLI sessions appear in the app's history. Permission requests are spoken aloud and answered by voice or tap, plan steps can be narrated while the agent works, and a context and cost chip tracks the session.

The pipeline is cascaded — speech to text to model to speech — rather than a single speech-to-speech model, which is what allows any model or agent to be swapped in behind it. The on-device stack is Silero for voice activity detection, Whisper for transcription, Smart-Turn for end-of-turn detection, and either Kokoro (28 voices) or Supertonic (10 voices at 44.1 kHz) for speech. Zero-shot voice cloning with ZipVoice records 5 to 30 seconds and is an optional install of roughly 208 MB.

Features

• On-device voice loop: voice activity detection, transcription, end-of-turn detection and speech synthesis all run locally on WebGPU
• Bring your own brain: a provider API key or a locally installed coding agent drives the conversation
• Agent Client Protocol: coding agents run as child processes over JSON-RPC on stdio, with model and mode switchable mid-call
• Shared sessions: calls land in the agent's own session store and resume from either the app or the CLI
• Permission relay: the agent's command and edit approvals are spoken and answered by voice or tap
• Vision: camera or screen frames accompany each turn, and a text-only model can borrow a separate vision model
• Barge-in: speech stops mid-word when you interrupt
• Voice cloning: zero-shot cloning from a 5-to-30-second recording, with profiles that preview, rename and move between machines
• Mini mode: an always-on-top pill keeps listening while you work, with a tray icon and notifications
• Transcript export: replies render as markdown with copy buttons on code blocks, and the conversation exports to a file
• Key storage: API keys are encrypted at rest with AES-256-GCM and displayed only as their last four digits