Soup

Soup is a command-line tool for fine-tuning and post-training language models from a single YAML configuration. soup init writes a config from a template, soup train runs the job, and batch size, GPU detection, and quantization are selected for you. Supervised fine-tuning and the preference methods (DPO, ORPO, SimPO, IPO, KTO, GRPO, PPO) run through the same command, as do tool-calling, distillation, classification, and vision or audio tasks.

An opt-in layer-streaming mode (stream_layers: true) keeps the frozen base model out of GPU memory and feeds it to the card one decoder layer at a time, which is what lets an 8B model train on a 4 GB GPU. On an RTX 3050 Laptop 4 GB, Llama-3.1-8B-Instruct with NF4 quantization was measured at 119.6 tokens per second and a 3.32 GB peak, bit-exact against a resident run, and reproduced on an H100 at a median 113.00 tokens per second in the same 3.32 GB. The mode is beta: in NF4 above roughly 165 MiB per decoder layer the backward pass produced wrong gradients while the forward pass stayed bit-exact, a defect since repaired and gated at 32B and above. Preference losses reuse the same streamed base with its adapters switched off instead of loading a second reference model.

Any text-generation model that loads with AutoModelForCausalLM works without config changes, and families including Llama 3.x/4, Qwen 2.5/3, Gemma 3, Mistral, Mixtral, DeepSeek R1/V3, and Phi-4 ship as ready-made recipes. Training data in Alpaca, ShareGPT, ChatML, preference-pair, vision, audio, ASR, plaintext, and embedding layouts is auto-detected from JSONL, JSON, CSV, Parquet, or TXT files. Finished adapters can be merged into the base, exported to GGUF, ONNX, TensorRT, AWQ, GPTQ, or BitNet, and served behind an OpenAI-compatible API. Python 3.10 to 3.12 is required.

VRAM	Max model (QLoRA 4-bit)	Example
8 GB	~7B	Llama-3.1-8B, Mistral-7B
16 GB	~14B	Phi-4-14B, Qwen2.5-14B
24 GB	~34B	CodeLlama-34B, Yi-1.5-34B
48 GB	~70B	Llama-3.3-70B
80 GB+	70B+ (full) or MoE	Mixtral-8x22B, DeepSeek-V3

Features

• One config, one command: a single soup.yaml drives training, with batch size, quantization, and device detection resolved automatically
• Layer streaming: the frozen base streams from host RAM or NVMe one decoder layer at a time, so only the adapter occupies GPU memory
• Preference training: DPO, ORPO, SimPO, IPO, KTO, BCO, GRPO, and PPO alongside supervised fine-tuning, with the reference model served by the same streamed base
• PEFT options: DoRA, LoRA+, rsLoRA, VeRA, OLoRA, NEFTune, PiSSA, ReLoRA, GaLore, YaRN and LongLoRA for long context, plus full fine-tuning as lora.r 0
• Ship gate: soup ship scores a tune against eight bundled offline suites (multiple choice, arithmetic, tool calling, JSON validity, safety and refusal) and returns a ship or don't-ship verdict with an emittable evidence record
• Reward synthesis: soup reward synth infers a deterministic verifier from reference outputs, writes it as a Python reward function, and refuses to emit one that cannot separate references from bad answers
• Export and serving: merge adapters, export to GGUF, ONNX, TensorRT, AWQ, GPTQ, or BitNet, and run an OpenAI-compatible or Anthropic Messages endpoint
• MCP server: soup mcp serve --allow-execute runs a planned training or export behind a single-use, server-generated confirmation token with the config snapshotted at plan time
• Diagnostics: soup doctor reports GPU, dependency, and environment state and prints the install command for a matching CUDA wheel
• Backends: Unsloth and MLX backends, DeepSpeed ZeRO-3 with CPU offload, FSDP, and multi-GPU launch