distil

We pointed it at the providers. Anthropic's default context-editing policy (keep=3) changed the agent's next action in 95–100% of cases, against a 2.5% A/A noise floor. Keeping the 3 most recent tool uses didn't lower the change rate at all — it turned stalling into acting on missing facts. OpenAI's compaction changed 12.5–20%. Pre-registered, replicated, n=40 per run.

Read the study → · rerun it on your own config

• It proves decision-equivalence per request — and can say no. Shadow mode replays a sampled request three times: twice on the original context and once on the compressed one, then reports 1{A=B} − 1{A=A'} — a paired difference against the model's own self-agreement, with a bootstrap 95% CI, unclipped, so it is allowed to be negative. One reporting floor (50 A/B + 30 A/A) gates every surface; below it, every surface says below reporting floor instead of a number. The current live sample cleared that floor on 2026-09-15 and reads 97.5% [95.5, 99.5] over n=398 A/B — under 99%, so the status line flags it ⚠ rather than ✓.
• What it folds, it can give back byte-exact. A digest is a marker plus a handle into a local content-addressed store, and the agent gets a distil_expand tool to recover the original mid-task. The gateway ships Tier-0 only rather than emit a stub it cannot restore.
• It will not digest a line your agent has to quote back. An Edit(old_string=…) is a literal match. Reading exact-quote provenance from the shell command, not just the tool name, took byte-exact quote loss from 39.3% → 16.2% on real coding traffic — and it costs real savings, which we price rather than hide.
• It does not break your prompt cache. Compression is suffix-only and cache-monotonic by construction: a later turn may never rewrite bytes the provider has already cached. We shipped that bug once, measured it at 2× the cost of compressing nothing, and made the invariant enforced. The cache contract →
• It has been pointed at a hostile input, not just a hard one. distil validate --adversarial runs a COMA-class battery through the same path the proxy uses, and we publish the two cases that do not come back clean. Threat model →
• Every rung of the dial is measured, not just the default. distil bench --curve traces savings against fact recall across the whole ladder, offline and free. The curve →

What it does

• Wrap your agent — 11 presets: distil wrap -- claude · codex · gemini · aider · opencode · qwen · goose · grok · openhands · copilot · kimi. Zero config, no code change.
• Run a proxy — point any base_url client at it. Python, TypeScript, any language, any framework. Sync proxy, async proxy, and a standalone gateway, with the same provider coverage in each: Anthropic Messages, OpenAI Chat Completions and the Responses API, Azure OpenAI, and Gemini generateContent.
• Call it as a library — from distil import compress_messages in your own agent loop.
• Give your agent a recall tool — MCP server: it compresses its own output and gets the exact bytes back on demand.
• Framework hooks — LangChain · LangGraph · LiteLLM · Agno · Strands · AutoGen · LlamaIndex, in-process, no network hop — plus an ASGI middleware for any Starlette/FastAPI app that hosts its own LLM endpoint, and the npm package for the Vercel AI SDK.
• On a subscription — distil hook --install: Claude Code compresses its own tool output through the documented PostToolUse extension point. No proxy, no credentials touched. distil quota shows the rate-limit window it buys back. Details →
• See what it did — live status line, session dissect, per-request headers, OTel spans, Prometheus metrics.

pipx install distil-llm && distil onboard    # detects your agent + billing, wires everything

Not sure which of those you want? Two questions pick your mode → — plain language, honest savings ranges, no jargon.

Will it save you money? On metered billing (an API key), yes — directly, off the bill. On a flat-rate Pro/Max subscription there is no per-token bill to cut, but there is a rate-limit window, and spending fewer tokens per turn leaves more of it for the next task. distil quota shows that window live. Savings come from large, repetitive tool output: verbose JSON and duplicated log runs compress 25–99%, while prose and unique-line output compress ~0% — a short session that never reads a big file showing near 0% is the tool working correctly, not failing. Why →

---

🧩 Use it as a library

Building the agent yourself? Compress the message list where it lives — no proxy, no network hop:

from distil import compress_messages, expand_handle

result = compress_messages(messages)          # OpenAI/Anthropic-style dicts
print(f"{result.saved_pct:.1f}% smaller")
response = client.messages.create(model=..., messages=result.messages)

original = expand_handle(result.handles[0])   # byte-exact, any time, any process

Tool results get the reversible digest; user and system text get lossless transforms only; the model's own turns are never rewritten. Handles resolve across processes and restarts, so a digest made by the proxy expands here and vice versa. verbatim=True disables digests entirely.

Named compress_messages/expand_handle rather than compress/expand because distil.compress and distil.expand are modules — a top-level export sharing those names would resolve to the function or the module depending on unrelated import order.

TypeScript too — compress(messages) from the npm package, byte-identical to the Python engine. Full reference: Library API → · runnable examples: python_library.py · js_library.ts.

Maintain a framework? docs/INTEGRATING.md is the ~20 lines and the four rules — we would rather the integration live in your repo than ours.

---

Property	Distil	Headroom 0.37.0 (2026-09-04)
Per-request behavioural check	Paired A/A′/B replay, unclipped difference, bootstrap CI, one reporting floor	No shadow or dual-send path in the codebase; accuracy_guard="strict" is echoed on /healthz and /stats but nothing branches on it
Recovery of what was folded	Content-addressed store + agent-facing distil_expand, byte-exact, verified by a gate	A TTL cache (SQLite, 1800s, 1000-entry FIFO), no integrity or round-trip check
Lossy paths with no recovery	None — the gateway ships Tier-0 only rather than emit a stub it cannot restore	Four: OpenAI chat streaming, Responses under ChatGPT auth, Gemini streaming, Bedrock
Savings number	Counted, then calibrated against the provider's billed usage	Falls back to chars/3.5
Exact-quote guarantee for coding agents	Provenance read from the shell command, not just the tool name; quote loss 39.3% → 16.2%	Not a property the tool has
Cache contract	Suffix-only, cache-monotonic, enforced as an invariant	Genuinely strong prompt-cache replay (overlay_cached_prefix) — real engineering
Adversarial gate	COMA-class battery in CI; the two cases that don't come back clean are published	None shipped
Degradation curve	Every ladder rung measured, offline and free	Point configuration only
Shipped default	Compresses	Mode cache — a full bypass on Bedrock, freeze-only on OpenAI

---

🚀 Use it now

One command sets you up and tells you what to do next:

pipx install distil-llm
distil onboard      # detects your agent + billing, wires the status line, prints a guided tour

It detects your environment (Claude Code · Codex · Gemini CLI; metered vs subscription) and hands you the exact commands. Or wrap your agent directly — no config, no code change:

# Claude Code on a metered API key — saves real $$:
distil wrap --expand -- claude

# Claude Code on a Pro/Max subscription — flat-rate, ToS-safe (trims context, not $):
distil wrap --lossless-only -- claude

# Codex, Gemini CLI, aider — same pattern; env var auto-selected per agent:
distil wrap --expand -- codex     # → OPENAI_BASE_URL
distil wrap --expand -- gemini    # → GOOGLE_GEMINI_BASE_URL
distil wrap --expand -- aider     # → OPENAI_BASE_URL

# Headless too — print mode, CI, and Agent SDK scripts route the same way:
distil wrap -- claude -p "summarise this diff"
distil wrap -- python my_agent_sdk_script.py

Using Cursor, Cline, or Windsurf? They are IDE extensions — no argv to wrap and no documented env var, so distil wrap cannot reach them. Run a proxy and point the editor's base-URL setting at it: docs/IDE-AGENTS.md. (GitHub Copilot is not redirectable at all, and that page says so rather than wasting your afternoon. The Continue CLI — as opposed to its VS Code extension — routes only through a config file, and distil wrap -- cn manages that file for you; see the same page.)

Each recognized agent (claude / codex / gemini / aider / opencode / qwen / goose) auto-selects the right env var and upstream — no --env-var or --upstream flag needed. Prints preset: <agent> detected → <VAR> on start. Explicit flags always win.

Tired of typing distil wrap every time? Make it the default — once:

distil default            # adds a managed shell alias so `claude` always routes through distil
distil default --undo     # remove it anytime (backed up before any change)

It detects your shell (zsh / bash / fish / PowerShell) and billing mode, writes the right line to the rc file your shell actually reads, and tells you what it detected. Want every SDK covered (not just the agent you type)? distil default --always-on runs a persistent proxy service — powerful, but it pins ANTHROPIC_BASE_URL, so every client on the machine goes through one local process.

That pin used to be a single point of failure: a proxy that was down for one second meant sessions failing with ConnectionRefused, an error that names the provider rather than distil. It no longer is. The service supervisor (launchd/systemd) owns the listening socket, so a crash or a restart leaves connections queued in the kernel backlog instead of refused — the client waits about a second rather than dying. distil default --always-on also verifies the service is genuinely registered and serving before it wires anything, and refuses to wire at all if it isn't.

If you ever need out and distil is already uninstalled, sh ~/.distil/uninstall.sh removes the pin, the service, and the shell block using nothing but sh.

Then watch genuine savings from your traffic — measured, not estimated:

distil leaderboard          # cumulative tokens + $ saved, from the local ledger
distil dashboard            # live terminal TUI — token-trim + decision-equiv bars, Ctrl-C to exit
distil dissect             # per-session deep-dive: savings, digest inventory, anomalies (--html/--serve)

Validate it on your traffic. --shadow runs a fraction of requests twice (compressed and full) and compares the agent's chosen next action:

distil wrap --shadow 0.1 -- claude   # wrap + shadow 10% of requests
distil shadow-stats                  # live decision-equivalence rate

Honest scope: that's next-action equivalence — a proxy, not task success (E7 shows it doesn't fully transfer under aggressive lossy compression). Distil fails safe to full context.

Will it save money? On metered billing (API key) — fewer tokens, fewer dollars, directly. On a flat-rate subscription there is no per-token bill, so the saving is rate-limit headroom: fewer tokens per turn means more turns before you hit the window (distil quota shows it live). Coding agents: short sessions ~7%, big wins on long, many-turn sessions the model never re-reads.

---

💡 Why Distil is different

You don't need byte-equivalence — you need decision-equivalence: your agent taking the same actions with compressed context. That's measurable and certifiable.