Replicate is a hosted platform for running machine learning models through an API. You call a model by name from Node, Python, or plain HTTP, pass it an input, and receive the output; the catalogue covers image generation and editing, speech, music, image restoration, image-to-video, image captioning, and large language models. You can also fine-tune models and deploy your own custom models on the platform.
Custom models are packaged with Cog, the platform's companion tool for putting a model in a production container. You describe the environment in a cog.yaml file (GPU use, system packages, Python version, requirements) and implement a Runner class in Python whose setup method loads weights and whose run method declares typed inputs and outputs. Cog generates the Docker image, selects compatible CUDA, cuDNN, PyTorch, TensorFlow, and Python combinations, derives an OpenAPI schema from the type annotations, validates requests against it, and serves predictions over a REST endpoint from a Rust/Axum server. The resulting image runs on any Docker host or on Replicate.
Cog runs on macOS, Linux, and Windows 11 through WSL 2, and needs Docker. Beyond run and build, it includes a training interface for adding a fine-tuning API to a model and a serve command that builds and runs in one step.
Features
- Model API: run hosted models from Node, Python, or HTTP with a single call, passing inputs and reading outputs
- Model catalogue: image generation and editing, speech, music, image restoration, image-to-video, captioning, and language models from providers such as Google, OpenAI, Black Forest Labs, and ByteDance
- Fine-tuning and custom deployment: train on your data and deploy your own models to the platform
- Cog packaging: a cog.yaml environment definition plus a Python Runner class produce a production Docker image
- CUDA compatibility resolution: Cog picks working CUDA, cuDNN, PyTorch, TensorFlow, and Python combinations for the image
- Typed inputs and outputs: Python type annotations become an OpenAPI schema that validates requests and responses
- HTTP inference server: each container exposes a /predictions REST endpoint generated from the model's types
- Portable images: built containers run on your own infrastructure or on Replicate
- Training interface: adds a fine-tuning API alongside prediction in the same package
