Captum is a model interpretability library for PyTorch. It implements attribution algorithms such as Integrated Gradients, DeepLift, GradientSHAP, saliency maps, SmoothGrad, and VarGrad that show which input features, neurons, layers, training examples, or concepts contribute to a model's predictions, and it works with models built on libraries such as torchvision and torchtext with little modification.
Attribution methods are classes in captum.attr that wrap a model and expose an attribute method. Input-level methods (for example IntegratedGradients, DeepLift, GradientShap, Occlusion, FeaturePermutation, ShapleyValueSampling) score input elements against a baseline; layer and neuron variants (LayerConductance, NeuronConductance, LayerIntegratedGradients, and others) score internal units; NoiseTunnel wraps any method with SmoothGrad, SmoothGrad-squared, or VarGrad smoothing. Methods that approximate an integral also return a convergence delta that serves as an approximation error.
Beyond attribution, the library provides Testing with Concept Activation Vectors (TCAV), TracIn influence functions and similarity-based influence for tracing predictions back to training examples, stochastic gates, infidelity and sensitivity metrics, and adversarial-attack and minimal-perturbation utilities for counterfactual explanations. Captum Insights is a web interface, also available as a Jupyter widget, for visualizing attributions. Requirements are Python 3.8 or later and PyTorch 1.10 or later; the project is marked as beta.
Features
- Feature attribution: Integrated Gradients, DeepLift, DeepLiftSHAP, GradientSHAP, Input X Gradient, Saliency, Guided Backpropagation, Deconvolution, GradCAM, Occlusion, Feature Permutation, and Shapley Value Sampling
- Layer and neuron attribution: conductance, integrated gradients, DeepLift, GradientSHAP, and internal influence variants for hidden units
- Noise smoothing: NoiseTunnel applies SmoothGrad, SmoothGrad-squared, or VarGrad to any attribution method
- Concept and influence methods: TCAV, TracIn (CP, fast, and random-projection variants), and similarity influence
- Evaluation metrics: infidelity and sensitivity scores for explanations
- Perturbation tools: adversarial attacks and minimal input perturbation for counterfactuals
- Captum Insights: a web app and Jupyter widget for interactive attribution visualization
- Multi-modal coverage: vision, text, and other modalities across most PyTorch model types
