Cavegemma is a LoRA fine-tune of Gemma 4 31B that bakes the "caveman" compressed-output style directly into the model's weights, so it answers technical questions in a terse, article-free register without needing a system prompt, skill file, or runtime toggle. The style rules come from the author's caveman skill for coding agents: drop articles, filler, and pleasantries while keeping code blocks and error strings byte-exact. The aim is cheaper inference and a compression style that survives across hosts and agents, including setups where no system prompt is available.
It ships in two forms: a full bf16 merged model and a lightweight LoRA adapter that stacks on the base Gemma 4 31B instruct model. Training used QLoRA (rank 16) on roughly 1,750 synthesized verbose-to-caveman pairs drawn from six permissively licensed datasets (dialogue, debugging, code review, refactoring, Q&A), with the caveman side generated via CLI coding agents following the canonical ruleset.
On a 193-pair holdout the model preserves 96-100% of code fences byte-exact and 91-98% semantic similarity while cutting article density to 0.5-2% (versus an English baseline around 8%). The README is candid about limitations: compression is weaker than the gold pairs (0.6-0.9x versus 0.3-0.5x), the code-review category is under-represented in eval, and Gemma 4's vision/audio paths are untested since the fine-tune was text-only. The repo includes the full reproduction pipeline: corpus building, synthesis, filtering, Unsloth/TRL training, and eval scripts.
