Pruna is a Python model-optimisation framework that makes trained models faster, smaller, and cheaper to run. It bundles caching, quantization, pruning, distillation, and compilation techniques behind a single smash function: you load a pre-trained model, describe the algorithms you want in a SmashConfig, and use the optimised model exactly as you used the original.
It targets several model families, including large language models, diffusion and flow-matching models, vision transformers, and speech-recognition models. Optimisation algorithms can be combined, and an evaluation interface measures the result - you define a task, attach a data module, and run an evaluation agent against the optimised model.
Pruna runs on Linux, macOS, and Windows with Python 3.9 or later, and can use a CUDA toolkit for GPU support; individual algorithms may restrict which operating systems they support. The algorithm groups and their documented effect on speed, memory, and output quality are listed below.
Features
| Technique | What it does | Speed | Memory | Quality |
|---|---|---|---|---|
| batcher | Groups multiple inputs to be processed together | improves | worsens | same |
| cacher | Stores intermediate results to speed up later operations | improves | same | same |
| compiler | Optimises the model with instructions for specific hardware | improves | same | same |
| distiller | Trains a smaller model to mimic a larger one | improves | improves | worsens |
| quantizer | Reduces the precision of weights and activations | improves | improves | worsens |
| pruner | Removes redundant connections and neurons | improves | improves | worsens |
| recoverer | Restores model performance after compression | same | same | improves |
| factorizer | Fuses several small matrix multiplications into one | improves | same | same |
| enhancer | Post-processes output, for example denoising or upscaling | worsens | same | improves |
| distributer | Spreads inference, the model, or parts of the computation across devices | improves | worsens | same |
| kernel | Uses specialised GPU routines for parts of the computation | improves | same | same |
