Axolotl is a Python framework for post-training and fine-tuning large language models. One YAML configuration file drives the whole run: dataset preprocessing, training, evaluation, quantization, and inference. It trains models from the Hugging Face Hub on NVIDIA or AMD GPUs, from a single GPU up to multi-node clusters.
Training methods cover full fine-tuning, LoRA and QLoRA, quantization-aware training, preference tuning, reinforcement learning, and reward modeling. Multimodal training covers vision-language and audio models with image, video, and audio inputs. Supported model families include GPT-OSS, Llama, Mistral and Mixtral, Qwen, Gemma, GLM, Granite, Kimi-Linear, InternVL, and Voxtral, with new architectures added as they are released.
For scale, Axolotl composes context, tensor, expert, and fully sharded data parallelism within and across nodes, and it trains mixture-of-experts (MoE) models with expert parallelism and quantized expert weights. Trained models export to GGUF, with optional quantization, for deployment with llama.cpp and Ollama. Plugin interfaces launch training jobs on cloud providers, and remote training runs through Tinker-compatible APIs.
Features
- Training methods: full fine-tuning, LoRA, QLoRA, GPTQ, QAT (int8, int4, FP8, NVFP4, MXFP4), FP8 mixed precision, and NVFP4 or MXFP4 MoE LoRA
- Preference and RL tuning: DPO, IPO, KTO, ORPO, GRPO (including async GRPO), GDPO, and reward or process reward modeling
- Multimodal training: vision-language models such as LLaMA-Vision, Qwen2-VL, Pixtral, LLaVA, and InternVL 3.5, plus audio models such as Voxtral
- Single YAML config: one file covers preprocessing, training, evaluation, quantization, and inference
- Attention and kernels: multipacking, Flash Attention 2, 3, and 4, Xformers, Flex Attention, SageAttention, Liger Kernel, Cut Cross Entropy, and ScatterMoE
- N-D parallelism: composes FSDP2, HSDP, tensor, context, sequence, and expert parallelism, with multi-GPU training through FSDP2 or DeepSpeed and multi-node training through Torchrun or Ray
- MoE training: expert parallelism over NVLink or PCIe, MoE expert quantization to cut VRAM, and LoRA directly on expert weights
- Memory controls: CPU-staged QLoRA loading for models larger than one GPU and selective activation checkpointing with optional CPU offload
- Dataset sources: local files, the Hugging Face Hub, and cloud storage on S3, Azure, GCP, and OCI
- GGUF export: exports trained models with optional quantization for llama.cpp and Ollama
- Other objectives: text diffusion training, BitNet 1.58-bit fine-tuning, and Dynamic Fine-Tuning that weights the SFT loss by token probability
