Twilio is a communications platform whose voice, messaging, and email APIs give agents a way to reach people on the phone and in chat. For voice agents the relevant pieces are Media Streams, which pipes raw call audio to your server over a WebSocket, and ConversationRelay, which runs speech-to-text and text-to-speech on Twilio's side and exchanges text with your application.
A unidirectional stream starts with TwiML or the REST API and can deliver inbound and outbound tracks, up to four tracks per call shared with SIPREC, transcription, and answering-machine detection. A bidirectional stream starts with , blocks further TwiML until it ends, delivers only the inbound track, and lets your server send audio back; one bidirectional stream is allowed per call. Twilio sends connected, start, media, dtmf, stop, and mark messages; your server sends media, mark, and clear. Audio is base64 mu-law at 8 kHz, mono, with no file headers, and each media message carries an inbound or outbound track label. DTMF arrives on bidirectional streams only, and requests carry an X-Twilio-Signature header for validation.
ConversationRelay uses to open a WebSocket that carries transcribed caller text to your app and streams your text replies back as speech. Speech providers include Deepgram Nova-2 for recognition and Google, Amazon Polly, and ElevenLabs Flash v2 and v2.5 for synthesis. Related products cover SMS, WhatsApp, RCS, email, and conversation memory, orchestration, and intelligence services.
Features
- Media Streams: WebSocket delivery of live call audio for transcription, analytics, or custom agents
- Bidirectional streaming: lets your server play audio into the call
- Message protocol: connected, start, media, dtmf, stop, mark, and clear events
- Audio format: base64 mu-law, 8 kHz, single channel, with inbound and outbound track labels
- Stream limits: four unidirectional tracks or one bidirectional stream per call
- ConversationRelay: managed STT and TTS with a text WebSocket to your application
- Speech providers: Deepgram, Google, Amazon Polly, and ElevenLabs
- Request signing: X-Twilio-Signature validation on WebSocket connections
- Messaging channels: SMS, WhatsApp, RCS, and email APIs alongside voice
Integrated by
-
Async
Text-to-speech via Async's WebSocket and HTTP APIs.
Dograh
Self-hostable open-source voice AI platform for building inbound and outbound phone agents with a drag-and-drop workflow builder
GBrain
Local knowledge brain that indexes a vault into Postgres and exposes retrieval, synthesis, and a graph over MCP