Axolotl is a free and open-source framework for post-training and fine-tuning large language models. A single YAML file carries the whole pipeline — dataset preprocessing, training, evaluation, quantization, and inference — and the axolotl command fetches example configurations, then trains from one of them.
Training methods cover full fine-tuning, LoRA, QLoRA, GPTQ, quantization-aware training (int8, int4, FP8, NVFP4, MXFP4), FP8 mixed precision, NVFP4 and MXFP4 mixture-of-experts LoRA, preference tuning (DPO, IPO, KTO, ORPO), reinforcement learning (GRPO, GDPO), and reward or process-reward modelling. Multimodal fine-tuning covers vision-language models and audio models with image, video, and audio inputs.
Scaling is handled through composable parallelism: context parallelism, tensor parallelism, expert parallelism, and fully sharded data parallelism can be combined within a node and across nodes, with multi-GPU support through FSDP1, FSDP2, and DeepSpeed and multi-node runs through Torchrun or Ray. Performance work includes multipacking, Flash Attention 2, 3, and 4, Xformers, Flex Attention, SageAttention, Liger Kernel, Cut Cross Entropy, and ScatterMoE kernels.
Running Axolotl needs Python 3.11 or newer (3.12 recommended), PyTorch 2.11.0 or newer, and an NVIDIA GPU (Ampere or newer for bf16 and Flash Attention) or an AMD GPU. Datasets load from local paths, Hugging Face, and S3, Azure, GCP, or OCI cloud storage. The project also ships Docker images and a PyPI package, and bundles agent-oriented reference docs that the axolotl agent-docs command prints per topic.
Features
- Single-config pipeline: one YAML file drives preprocessing, training, evaluation, quantization, and inference
- Training methods: full fine-tuning, LoRA, QLoRA, GPTQ, QAT, FP8 mixed precision, preference tuning, RL, and reward modelling
- Multimodal training: vision-language and audio models with image, video, and audio inputs
- ND parallelism: context, tensor, expert, and fully sharded data parallelism composed within and across nodes
- Multi-GPU and multi-node: FSDP1, FSDP2, and DeepSpeed locally; Torchrun and Ray across machines
- Kernel optimizations: multipacking, Flash Attention 2/3/4, Xformers, Flex Attention, SageAttention, Liger Kernel, Cut Cross Entropy, and ScatterMoE
- MoE support: expert quantization and LoRA fine-tuning directly on mixture-of-experts weights, including NVFP4 4-bit training
- Dataset sources: local files, Hugging Face, and S3, Azure, GCP, or OCI storage
- Deployment artifacts: Docker images and a PyPI package for cloud platforms and local hardware
- Agent docs: axolotl agent-docs prints topic-specific references bundled with the package for AI coding agents
