TwelveLabs is a video-understanding platform reached through an API and language SDKs. Videos are uploaded as assets, then searched, analyzed, or turned into embeddings; the platform processes visuals, sounds, spoken words, and on-screen text together rather than treating a video as a transcript.
Search works over an index. An index is created with a Marengo model and the modalities it should analyze, uploaded assets are added to it, and indexing runs asynchronously until the asset reports ready. Queries can be natural-language text, up to ten images, or a composed query that combines images with text to narrow results — for example an image of a car plus "red color". Each result identifies the matching video and the start and end time of the clip in seconds, ranked by relevance.
Analysis uses a Pegasus model against an asset directly, with no index required. A custom prompt generates summaries, extracts insights, answers questions, or produces structured output, and responses can stream as text fragments or return complete in one call. Embeddings are created from video, audio, images, documents, and text for similarity search, classification, clustering, recommendation, or retrieval-augmented generation, with query embeddings returned synchronously and bulk content embedded as asynchronous per-file tasks.
| Constraint | Value |
|---|---|
| Python version | 3.7 or newer |
| Guide upload size | up to 4 GB per file |
| Synchronous processing | files up to 200 MB; larger files are polled until ready |
| Search query text | up to 500 tokens |
| Analysis prompt | up to 2,000 tokens |
| Composed query images | up to 10 |
SDK exceptions map to HTTP status codes, so a bad request, authentication failure, permission denial, or missing resource each raises a distinct error type.
Features
- Asset upload: publicly accessible URLs or direct local file uploads, with status polling for large files
- Indexed search: Marengo-backed indexes configured per modality across visual, audio, and transcription
- Query modes: natural-language text, image queries of up to ten images, and composed image-plus-text queries
- Clip-level results: matching video identifier, relevance rank, and start and end times in seconds
- Video analysis: Pegasus prompts of up to 2,000 tokens producing summaries, insights, answers, or structured output
- Streaming or complete responses: text fragments delivered live, or the full result in a single call
- Embeddings: vectors from video, audio, image, document, and text input for search, clustering, and retrieval-augmented generation
- Typed errors: SDK exception classes mapped to HTTP status codes
