Diffusers is a Python library from Hugging Face for working with pretrained diffusion models that generate images, audio, and 3D molecular structures. It serves both inference and training, and its design prioritises usability over performance, simple over easy, and customisability over abstractions.
The library has three building blocks: pipelines that run a full diffusion task in a few lines of code, interchangeable noise schedulers that trade speed against output quality, and pretrained model components (such as UNets and VAEs) that you combine with a scheduler to assemble your own end-to-end diffusion system. DiffusionPipeline.from_pretrained loads any of the 30,000+ Diffusers-format checkpoints on the Hugging Face Hub. Modular Diffusers lets you compose pipelines from modular components, and guides cover loading, memory and speed optimisation, and training with different techniques.
Task coverage includes:
Features
- Unconditional generation: DDPM
- Text-to-image: Stable Diffusion, unCLIP, DeepFloyd IF, and Kandinsky
- Text-guided image-to-image: ControlNet, InstructPix2Pix, and Stable Diffusion img2img
- Inpainting: Stable Diffusion inpainting
- Image variation and super-resolution: Stable Diffusion image variation and the x4 upscaler
- Schedulers: swap DDPM, DDIM, and other samplers without changing the model
- Training: scripts and guides for training and fine-tuning diffusion models
- Optimisation: fp16 and other memory and speed guides, including Apple Silicon (MPS) support
- Agent conventions: a .ai/ directory of project conventions installable as a Claude plugin or as individual skills
