TuFT lets several users fine-tune large language models on one shared pool of GPUs through a single API. Clients connect with the Tinker SDK or a compatible client, ask the server which base models it offers, create a LoRA training client for one of them, run forward and backward passes with a loss function, apply optimizer steps, save checkpoints and sampler weights, then sample tokens from the trained weights.
The server is a FastAPI service started with tuft launch, configured by a YAML file that lists the base models it serves along with authentication, persistence, and telemetry settings. It holds several LoRA adapters per base model, tracks sessions and their training runs, and manages checkpoint storage; optional Redis-backed persistence lets server state survive a crash, and OpenTelemetry covers traces, metrics, and logs.
Installation resolves GPU wheels before downloading: the installer inspects the NVIDIA driver, selects the validated CUDA 13.0 wheel variant for the pinned torch and vLLM stack, and runs import and CUDA smoke tests, failing with guidance rather than installing a broken environment when the driver is too old. You can override the backend for CPU-only or custom-wheel builds, and a prebuilt Docker image is available.
Deployment helpers wrap the same tuft launch server for rented GPUs, so training can be driven from a laptop with no local GPU: Modal for serverless GPUs with scale-to-zero and per-second billing, and Lambda Cloud for an on-demand GPU virtual machine billed per minute. Documented workflows include supervised fine-tuning on chat data with assistant-only loss masking, GRPO-style reinforcement learning on verifiable tasks, on-policy distillation using per-token reverse KL, and client-defined objectives through forwardbackwardcustom on both the Hugging Face and FSDP backends.
Features
- Multi-tenant API: several users share one deployment, each with their own sessions, training runs, and checkpoints
- Tinker-compatible clients: the Tinker SDK or any compatible client talks to the server over REST
- LoRA training loop: forwardbackward, optimstep, savestate, and saveweightsforsampler cover training and checkpointing
- Sampling from trained weights: a sampling client loads saved sampler weights and generates tokens
- Custom losses: forwardbackwardcustom supports client-defined objectives such as composite DPO plus NLL
- Training recipes: chat supervised fine-tuning, GRPO-style reinforcement learning, and on-policy distillation
- GPU wheel preflight: driver inspection, CUDA 13.0 wheel selection, and post-install smoke tests, with an environment-check script for connectivity problems
- Cloud deployment: Modal and Lambda Cloud helpers for renting GPUs on demand
- Persistence and telemetry: optional Redis state persistence and OpenTelemetry tracing, metrics, and logs
- Console: a dashboard for monitoring training runs, checkpoints, and a sampling playground
