Moss is a semantic search runtime that embeds inside your application process rather than running as a separate database. You create an index of documents through Moss Cloud, load it into the runtime, and query it locally; because the loaded index and the embedding model sit in-process, there is no network hop on the query path. It is aimed at conversational and voice agents, copilots, and other systems where retrieval latency shows up in the user experience.
The architecture has three parts: Moss Cloud handles ingestion, embedding, storage, and distribution; an index packages documents and vectors as one artifact; and the runtime pulls indexes over HTTPS, holds them in memory, serves queries, and stays in sync as documents change. The runtime ships as server-side SDKs for Python (3.10+) and Node.js (20+), plus Elixir and C (libmoss), and as a WebAssembly build (@moss-dev/moss-web) that runs queries entirely in the browser.
On a benchmark of 100,000 documents with 750 queries at top_k=5 on an M4 Pro MacBook, end-to-end latency including embedding measured:
| System | P50 | P95 | P99 | Mean |
|---|---|---|---|---|
| Moss | 3.1 ms | 4.3 ms | 5.4 ms | 3.3 ms |
| Pinecone | 432.6 ms | 732.1 ms | 934.2 ms | 485.8 ms |
| Qdrant | 597.6 ms | 682.0 ms | 771.4 ms | 596.5 ms |
| ChromaDB | 351.8 ms | 423.5 ms | 538.5 ms | 358.0 ms |
The comparison systems used an external embedding service, and Pinecone and Qdrant ran as cloud search.
Features
- Hybrid search: semantic and keyword retrieval in a single query
- Built-in embeddings: bundled embedding models, with the option to bring your own
- Metadata filtering: $eq, $and, $in, and $near operators
- Browser runtime: a WebAssembly SDK for client-side search on static sites, extensions, and offline-first apps
- Database connectors: ingest directly from SQLite, MongoDB, MySQL, and Supabase
- CLI: manage indexes and run queries from the terminal
- Agent framework integrations: LangChain, DSPy, LlamaIndex, CrewAI, AutoGen, Haystack, Mastra, Pydantic AI, Strands Agents, Langflow, and Vercel AI SDK
- Voice stack integrations: Pipecat, LiveKit, Vapi, ElevenLabs, Agora, and TEN Framework, including an MCP server for Agora
- Web and automation: Next.js, VitePress plugin, and n8n nodes
- Local voice example: an Ollama, Moss, and Pipecat stack that runs LLM inference and retrieval on one machine
Integrates with
AutoGen
Microsoft multi-agent framework now in maintenance mode, succeeded by Microsoft Agent Framework
CrewAI
Python multi-agent framework pairing autonomous role-based Crews with event-driven Flows for precise orchestration
DSPy
Compose LM pipelines as declarative Python modules, then optimize their prompts and weights algorithmically
-
ElevenLabs
Expressive text-to-speech with word-level timing plus file-based transcription.