Project AIRI is a self-hosted container for AI companions: virtual characters you own and run yourself, on the web, on macOS and Windows desktops, or on a phone. It takes the AI VTuber idea past chat, aiming at a character that plays games, sees what you are doing, and talks with you in real time rather than answering in a text box.
It was built on web technologies from the start, including WebGPU, WebAudio, Web Workers, WebAssembly, and WebSockets, so the browser build runs on modern devices and installs as a PWA on mobile. The desktop build is not limited to the browser sandbox: it uses NVIDIA CUDA and Apple Metal through the candle runtime, and it can hold TCP connections for things the web cannot do, such as joining a Discord voice channel or playing Minecraft and Factorio.
Character presentation covers both VRM and Live2D models, with model control and animations including auto-blink, auto-look-at, and idle eye movement. Voice input comes from the browser or from Discord, with client-side speech recognition and talking detection; voice output is multi-provider, covering ElevenLabs, Microsoft and Azure Speech, OpenAI-compatible text-to-speech, Alibaba Cloud Model Studio, and local Kokoro TTS. Memory currently runs in-browser on DuckDB WASM or pglite.
Model access goes through the project's own xsai client, which covers a wide provider list including OpenAI and Azure OpenAI, Anthropic, Google Gemini, DeepSeek, Qwen, Mistral, xAI, Groq, Moonshot AI, Minimax, Zhipu, and aggregators and local runtimes such as OpenRouter, vLLM, SGLang, and Ollama. A separate organization hosts sub-projects spun out of the app, covering retrieval, memory, embedded databases, and Live2D utilities.
Features
- Game playing: Minecraft and Kerbal Space Program are working, with Factorio at proof-of-concept stage
- Chat channels: Telegram and Discord, including audio input from a Discord voice channel
- Speech: client-side speech recognition and talking detection in the browser
- Voice synthesis: ElevenLabs, Microsoft and Azure Speech, OpenAI-compatible TTS, Alibaba Cloud Model Studio, and local Kokoro TTS
- Character rendering: VRM and Live2D models with auto-blink, auto-look-at, and idle eye movement
- Local acceleration: the desktop build uses CUDA and Metal through candle without extra dependency management
- In-browser memory: DuckDB WASM and pglite back the current database layer, with in-browser WebGPU inference planned
- Platform reach: browser, macOS and Windows desktop builds, and an experimental mobile stage
- Broad provider support: hosted APIs, aggregators, and local runtimes selected through the xsai client
