Tokenjuice

Tokenjuice is a deterministic output compactor for terminal-heavy agent workflows. When an agent or harness runs a noisy command such as git status, pnpm test, docker build, rg, or pnpm --help, tokenjuice leaves the command itself untouched, observes the output after execution, and returns a smaller payload built by rule-driven reducers instead of passing the full terminal text back into context.

Reduction is rule-driven rather than model-driven. Built-in JSON rules ship in src/rules, user overrides live in ~/.config/tokenjuice/rules, and project overrides in .tokenjuice/rules; later layers override earlier ones by rule ID. Rules classify command output, normalize lines, keep or drop patterns, count facts, and retain deterministic head and tail slices. A safe-inventory policy keeps exact file-content reads raw, allows standalone repository inventory commands to compact, and leaves mixed command sequences untouched. The --raw and --full flags bypass reduction when the task needs the original bytes, and opt-in artifact storage keeps the full output retrievable by ID.

The CLI exposes three surfaces: reduce compacts existing text, wrap runs a command and compacts its observed output, and reduce-json provides a stable JSON protocol for host adapters. Host integrations are thin wrappers that install a hook, extension, rule, or guidance file and call the shared compactor. Supported hosts include Claude Code, CodeBuddy, Codex CLI, Cursor, Droid, GitHub Copilot CLI, OpenClaw (as a bundled plugin, OpenClaw 2026.4.22 or newer), OpenCode, pi, and VS Code Copilot Chat, with a further list of beta integrations covering Aider, Amp, Cline, Continue, Gemini CLI, Goose, OpenHands, Windsurf, Zed, and many others.

Features

• Rule-driven reducers: inspectable JSON rules that classify, normalize, keep or drop, count, and slice command output
• Layered rule overrides: built-in, user (~/.config/tokenjuice/rules), and project (.tokenjuice/rules) layers merged by rule ID
• Three CLI surfaces: reduce for existing text, wrap -- for live execution, and reduce-json for adapters
• Explicit bypass: --raw and --full return untouched output; --store keeps full artifacts retrievable via ls and cat
• Safe-inventory policy: file-content reads stay raw while standalone inventory commands compact
• Host installers: tokenjuice install wires hooks or rule files for 10 supported and many beta coding agents
• Diagnostics: doctor hooks and doctor check installed wiring; verify --fixtures and discover exercise the rule set
• Usage stats: tokenjuice stats reports reduction activity, with a --timezone option