Voice Lab is a testing and evaluation framework for LLM-powered agents, with a focus on voice agents. It addresses the routine work of maintaining such agents: reviewing call logs by hand, refining prompts without metrics, and checking whether a model switch will degrade behavior. The current release covers the text side of a voice agent, that is, the underlying language model and prompt, rather than audio.
You define test scenarios in a testdetails.json file, where each scenario carries a system prompt, an initial message, tool-call definitions such as an endconversation function with structured termination evidence, success criteria, and a simulated user persona with role, traits, mood, and response style. Custom evaluation metrics are defined in JSON and scored by an LLM acting as judge. A web-based configuration editor generates the JSON files. The framework runs on an OpenAI API key.
Features
- JSON-defined metrics: custom metrics scored with LLM-as-a-judge and tracked across configurations
- Model comparison: run the same scenarios across models to evaluate cost, latency, and behavior before switching
- Prompt variation testing: test multiple prompt versions against the same scenarios
- Persona simulation: simulated users with configurable role, traits, mood, and response style
- Tool-call scenarios: scenarios declare function tools, including a structured end_conversation call with termination evidence
- Success criteria: required confirmations per scenario determine pass or fail
- Configuration editor: a hosted UI generates scenario and metric JSON files
- Additional context: scenarios can pass agents extra data such as prices or account details via JSON