Cerebras Inference serves generative models from Cerebras CS-3 systems, which are built around the wafer-scale WSE-3 processor, and is reachable through the Cerebras Cloud or an on-premise deployment. The API is OpenAI-compatible, so an existing OpenAI client reaches it by changing the base URL and key, and the model catalog covers open models such as gpt-oss-120b.
The Python library wraps the REST API. It carries type definitions for every request parameter and response field, offers synchronous and asynchronous clients built on httpx, and returns responses as Pydantic models with helpers for JSON and dictionary conversion. Chat completions and text completions are both available, and either can stream over server-sent events, with usage and timing information arriving in the final chunk.
On construction the client sends a few warming requests to reduce time to first token. That can be disabled, and reusing one client instance rather than rebuilding it is documented as the supported pattern. Requests time out after one minute by default and connection errors, 408, 409, 429, and 5xx responses are retried twice with exponential backoff. Both are configurable per client or per request. Failures raise typed exceptions mapped from the status code, and standard-library logging can be turned on at info or debug level.
Access starts with an API key from the Cerebras Cloud console, with a free tier that includes an initial credit, self-serve pay-per-token usage with higher rate limits, and an enterprise tier offering dedicated queue priority, custom model weights, and uptime guarantees.
Features
- OpenAI-compatible API: existing OpenAI client code works after changing the base URL and key
- Typed Python client: request parameters as TypedDicts and responses as Pydantic models, with editor autocompletion and validation
- Sync and async clients: Cerebras and AsyncCerebras share one interface, with an optional aiohttp backend for higher concurrency
- Streaming: server-sent events for both chat and text completions, with usage and timing in the final chunk
- TCP warming: connection warm-up requests at construction reduce time to first token, and can be turned off
- Retries and timeouts: two automatic retries with backoff and a one-minute default timeout, configurable globally or per request
- Typed errors: distinct exception classes per status code, all inheriting from a common API error
- Raw response access: .with_raw_response. on any method exposes headers and the underlying response
- Wafer-scale serving: inference runs on CS-3 systems powered by the WSE-3 processor, available in the cloud or on-premise
- Tiered access: a free tier with starting credit, self-serve pay-per-token, and an enterprise tier with dedicated priority and custom weights
