LlamaFactory trains and fine-tunes large language models without writing training code. Over 100 model families are configured through one command-line interface or LLaMA Board, a Gradio web interface, so a run is described in YAML or a form rather than in a script. It runs locally, in Docker, or on hosted notebooks and cloud GPU platforms.
Training approaches cover continued pre-training, supervised fine-tuning including multimodal instruction data, reward modelling, and preference optimisation with PPO, DPO, KTO, and ORPO. Each approach is available with full-parameter tuning, freeze tuning, LoRA, QLoRA, OFT, and QOFT. Precision options run from 16-bit full tuning down to 2-, 3-, 4-, 5-, 6-, and 8-bit quantized tuning through AQLM, AWQ, GPTQ, LLM.int8, HQQ, and EETQ.
Memory and speed work is drawn from published methods: GaLore, BAdam, APOLLO, Adam-mini, Muon, OFT, DoRA, LongLoRA, LLaMA Pro, Mixture-of-Depths, LoRA+, LoftQ, and PiSSA on the optimisation side, and FlashAttention-2, Unsloth, Liger Kernel, KTransformers, RoPE scaling, NEFTune, and rsLoRA on the kernel and throughput side. A Megatron-core training backend is available through an adapter.
Supported architectures include Llama through Llama 4, Llama 3.2 Vision, Qwen3 and Qwen3-VL, DeepSeek and its R1 distillations, Gemma 2, 3 and 3n, GLM-4 and GLM-4.5, Mistral and Mixtral-MoE, Phi, InternLM and InternVL, Kimi-VL, Granite, Falcon, Hunyuan, ERNIE-4.5, BLOOM, GPT-2, and GPT-OSS, with several families supported on their release day. Weights download from the Hugging Face Hub, ModelScope, or Modelers.
Tasks extend past plain text to multi-turn dialogue, tool use, image understanding, visual grounding, video recognition, and audio understanding. After training, a model serves behind an OpenAI-style API, the Gradio interface, or the CLI, with vLLM or SGLang as the inference worker, and experiment metrics go to LlamaBoard, TensorBoard, Weights & Biases, MLflow, or SwanLab.
Features
- Zero-code interfaces: a CLI and the Gradio-based LLaMA Board cover configuration, training, evaluation, and chat
- Training approaches: continued pre-training, supervised fine-tuning, reward modelling, PPO, DPO, KTO, and ORPO
- Tuning modes: full-parameter, freeze, LoRA, QLoRA, OFT, and QOFT across every approach
- Quantization: 2- to 8-bit QLoRA through AQLM, AWQ, GPTQ, LLM.int8, HQQ, and EETQ
- Optimisation algorithms: GaLore, BAdam, APOLLO, Adam-mini, Muon, DoRA, LongLoRA, LLaMA Pro, Mixture-of-Depths, LoRA+, LoftQ, and PiSSA
- Throughput: FlashAttention-2, Unsloth, Liger Kernel, KTransformers, RoPE scaling, NEFTune, and rsLoRA
- Model coverage: more than 100 families, including day-0 support for Qwen3, Qwen2.5-VL, Gemma 3, GLM-4.1V, InternLM 3, and MiniCPM-o-2.6
- Multimodal training: image understanding, visual grounding, video recognition, and audio understanding alongside text
- Model hubs: downloads from the Hugging Face Hub, ModelScope, or Modelers
- Serving: an OpenAI-style API with a vLLM or SGLang worker, plus Gradio and CLI chat
- Experiment tracking: LlamaBoard, TensorBoard, Weights & Biases, MLflow, and SwanLab
- Hardware backends: NVIDIA and AMD GPUs, Ascend NPU, and Megatron-core through an adapter
