tokensave

---

Why tokensave?

AI coding agents waste tokens exploring codebases. Every grep, glob, and file read costs money. On complex tasks, agents spawn multiple Explore sub-agents that scan hundreds of files just to build context.

tokensave gives agents a pre-indexed semantic knowledge graph. Instead of scanning files, the agent queries the graph and gets instant, structured answers -- the right symbols, their relationships, and source code, in one call.

How It Works

┌──────────────────────────────────────────────────────────────┐
│  AI Coding Agent (Claude Code, Codex, Gemini, Cursor, ...)   │
│                                                              │
│  "Implement user authentication"                             │
│        │                                                     │
│        ▼                                                     │
│  ┌─────────────────┐       ┌─────────────────┐               │
│  │  Sub-agent      │ ───── │  Sub-agent      │               │
│  └────────┬────────┘       └─────────┬───────┘               │
└───────────┼──────────────────────────┼───────────────────────┘
            │                          │
            ▼                          ▼
┌──────────────────────────────────────────────────────────────┐
│  tokensave MCP Server                                        │
│  ┌─────────────┐  ┌─────────────┐  ┌─────────────┐           │
│  │   Search    │  │   Callers   │  │   Context   │           │
│  │   "auth"    │  │  "login()"  │  │   for task  │           │
│  └──────┬──────┘  └──────┬──────┘  └──────┬──────┘           │
│         └────────────────┼────────────────┘                  │
│                          ▼                                   │
│              ┌───────────────────────┐                       │
│              │   libSQL Graph DB     │                       │
│              │   • Instant lookups   │                       │
│              │   • FTS5 search       │                       │
│              └───────────────────────┘                       │
└──────────────────────────────────────────────────────────────┘

Without tokensave: Agents use grep, glob, and Read to scan files -- many API calls, high token usage.

With tokensave: Agents query the graph via MCP tools -- instant results, local processing, fewer tokens.

---

Key Features

Smart Context Building	Semantic Search	Impact Analysis
One tool call returns everything the agent needs -- entry points, related symbols, and code snippets.	Find code by meaning, not just text. Search for "authentication" and find login, validateToken, AuthService.	Know exactly what breaks before you change it. Trace callers, callees, and the full impact radius of any symbol.
80+ MCP Tools	50+ Languages	12+ Agent Integrations
From call graph traversal to dead code detection, atomic edit primitives, code-health metrics, test mapping, and complexity analysis.	Rust, Go, Java, Python, TypeScript, C, C++, Swift, Svelte, Astro, and 43 more including WGSL/HLSL/Metal shaders, CUDA/HIP, and Markdown. Three tiers (lite/medium/full) control binary size.	Claude Code, Codex CLI, Gemini CLI, Qwen Code, Kiro, Cursor, OpenCode, Copilot, Cline, Roo Code, Zed, Antigravity, Kilo CLI, Kimi CLI, Mistral Vibe, Grok Build, Factory Droid, OMP, Pi, Plank.
Multi-Branch Indexing (opt-in)	100% Local	Always Fresh
Optional per-branch databases. Cross-branch diff and search without switching your checkout.	No data leaves your machine. No API keys. No external services. Everything runs on a local libSQL database.	On-demand staleness check on every MCP call (30 s cooldown) plus catch-up sync when the server connects. Multi-agent work is expected to use git worktrees — each agent gets its own checkout and the index diverges are merged by git, not by a file watcher.
Subprocess-Isolated Extraction	Code-Health Analytics	Atomic Edit Primitives
A native crash in any tree-sitter grammar (abort, segfault, anything) kills only the worker; the pool respawns it and sync continues. Sync never dies on a malformed file.	Composite health score (0-10000), Gini inequality, file-DAG depth, design-structure matrix, risk-weighted test gaps, and session deltas.	Edit files without regex or shell-quoting hazards: unique-anchor str_replace, atomic multi-replace, AST-rewrite, anchored insert. Auto re-indexes after writes.

---

Quick Start

1. Install

Homebrew (macOS):

brew install aovestdipaperino/tap/tokensave

Scoop (Windows):

scoop bucket add tokensave https://github.com/aovestdipaperino/scoop-bucket
scoop install tokensave

Cargo / cargo-binstall (any platform):

# Fast install prebuilt binary without compiling:
cargo binstall tokensave

# Or compile from source:
cargo install tokensave                          # full (50+ languages, default)
cargo install tokensave --features medium        # medium tier
cargo install tokensave --no-default-features    # lite (smallest binary)

Prebuilt binaries (Linux, Windows, macOS):

Download from the latest release and place the binary in your PATH.

Platform	Archive
macOS (Apple Silicon)	tokensave-vX.Y.Z-aarch64-macos.tar.gz
Linux (x86_64)	tokensave-vX.Y.Z-x86_64-linux.tar.gz
Linux (ARM64)	tokensave-vX.Y.Z-aarch64-linux.tar.gz
Windows (x86_64)	tokensave-vX.Y.Z-x86_64-windows.zip

2. Configure your agent

tokensave install                         # auto-detects installed agents
tokensave install --agent antigravity     # Google Antigravity (formerly Windsurf)
tokensave install --agent auggie          # AugmentCode
tokensave install --agent claude          # Claude Code
tokensave install --agent cline           # Cline
tokensave install --agent codex           # OpenAI Codex CLI
tokensave install --agent copilot         # GitHub Copilot
tokensave install --agent cursor          # Cursor
tokensave install --agent droid           # Factory Droid
tokensave install --agent gemini          # Gemini CLI
tokensave install --agent kilo            # Kilo CLI
tokensave install --agent kiro            # AWS Kiro
tokensave install --agent kimi            # Moonshot Kimi CLI
tokensave install --agent omp             # Oh My Pi (OMP)
tokensave install --agent opencode        # OpenCode
tokensave install --agent pi              # Pi (pi.dev)
tokensave install --agent plank           # Plank (macOS only)
tokensave install --agent qwen            # Qwen Code
tokensave install --agent roo-code        # Roo Code
tokensave install --agent vibe            # Mistral Vibe
tokensave install --agent zed             # Zed
tokensave install --agent grok            # Grok Build (xAI)
tokensave install --git-hook yes           # auto-install the global post-commit and post-checkout hooks (no prompt)
tokensave install --git-hook no            # skip the post-commit and post-checkout hooks (no prompt)
tokensave githooks                         # show which global git hooks tokensave owns
tokensave githooks off                     # remove them, leaving any hook content you wrote

Each agent gets its MCP server registered in the native config format. Claude Code additionally gets a PreToolUse hook (blocks wasteful Explore agents), a UserPromptSubmit hook, a Stop hook, prompt rules in CLAUDE.md, and auto-allowed tool permissions. Kiro gets global MCP config, tokensave.md steering loaded as a resource, and a tokensave-managed default agent with permissive built-in/tokensave tool approval, delegation guardrail hooks, and post-write sync; user-managed Kiro agents are preserved.

Global OMP installs target the profile reported by bare omp config path, writing <resolved-agent-dir>/mcp.json and <resolved-agent-dir>/rules/tokensave.md. Export OMP_PROFILE or OMP's compatible PI_PROFILE when installing into a named profile; OMP's resolver also honors PI_CONFIG_DIR and PI_CODING_AGENT_DIR. Tokensave trusts that native resolver rather than duplicating OMP's profile logic. Tokensave installs MCP and advisory rules for OMP; it does not install OMP hook enforcement.

All changes are idempotent -- safe to run again after upgrading. After agent setup, you'll be offered global git post-commit and post-checkout hooks. tokensave uninstall removes those hooks along with the agent integrations; pass --keep-git-hooks to leave them, or manage them on their own with tokensave githooks.

Project-local install

By default tokensave install registers the MCP server in your global agent config (e.g. ~/.claude.json). To register tokensave for just the current project instead, add --local:

tokensave install --local --agent claude
tokensave install --local --agent omp

This writes project-scoped config you can commit and share with your team. For Claude that's ./.mcp.json, ./.claude/settings.json, and ./CLAUDE.md; OMP uses ./.omp/mcp.json and ./.omp/rules/tokensave.md without invoking the OMP CLI. Supported agents: claude, cursor, droid, gemini, zed, opencode, roo-code, kiro, auggie, omp, plank (each writes its own project file, e.g. .cursor/mcp.json, .factory/mcp.json, .gemini/settings.json, .zed/settings.json, opencode.json, .roo/mcp.json, .kiro/settings/mcp.json, .augment/settings.json, .omp/mcp.json, .mcp.json for plank). Other agents have no project-scoped config and report an error with --local.

Remove a project-local install with tokensave uninstall --local.

3. Index your project

cd /path/to/your/project
tokensave init

This creates a .tokensave/ directory with the knowledge graph database. Initialization and sync are separate commands: init is a one-time opt-in per project, while sync only updates projects that were already initialized. This prevents the global git hooks from silently creating databases in repos you never intended to index. After init, use tokensave sync to incrementally update -- only changed files are re-indexed.

MCP server

{
  "mcpServers": {
    "tokensave": {
      "command": "/path/to/tokensave",
      "args": ["serve"]
    }
  }
}

PreToolUse hook

The hook runs tokensave hook-pre-tool-use -- a native Rust command (no bash or jq required). It intercepts Agent, Grep, Glob, and Bash tool calls: Explore agents are blocked outright, symbol-shaped grep/rg/ag invocations (plain identifiers, alternations, \b-wrapped names) are redirected to the matching tokensave MCP tool, and path-shaped discovery (Glob, find -name, fd --extension) over code extensions is redirected to tokensave_files. Regex patterns, git grep, piped commands, non-code extensions, search roots outside the index, and find predicates that change what the command does (-exec, -delete, -mtime) all pass through untouched; set TOKENSAVE_DISABLE_GREP_HOOK=1 to opt out per shell.

Filters are read most-specific-first: an explicit type is authoritative, then an explicit file glob, then the search path. A documentation search such as path: "." with glob: "**/*.md" therefore passes through rather than being treated as a code search on the broad path, while a code-only glob (**/*.rs) still redirects even under a non-code path. Mixed globs (**/*.{rs,md}) pass through, since they can return documentation.

Headless / subagent dispatch (claude -p). Child processes dispatched by an orchestrating session inherit its ~/.claude/settings.json, including this hook. To let a child run raw searches, set TOKENSAVE_DISABLE_GREP_HOOK=1 in the child's environment -- the native binary honors it and passes every path (Grep, Glob, Bash, Agent) through, so there is no need for the blunt --settings '{"hooks": {}}' that strips all hooks. The guardrail is stateless: it never consults citation history, so it only ever redirects the symbol-shaped searches described above and steers untyped research fan-out; ordinary commands are unaffected whether the session is interactive or headless.

CLAUDE.md rules

Appends instructions to ~/.claude/CLAUDE.md that tell Claude to use tokensave tools before reaching for Explore agents or raw file reads.

---

Crash-Resilient Sync

Tree-sitter grammars are compiled C/C++ code. They occasionally hit an internal assertion or otherwise terminate the process by paths that Rust panic handling cannot intercept. As of v4.3.0, every file is parsed inside a short-lived worker subprocess: if a grammar segfaults, calls abort(), or hits a stack overflow, only the worker dies. The pool respawns it, the offending file is logged and skipped, and sync keeps going.

The worker is a hidden extract-worker subcommand authenticated against the parent via a 256-bit per-spawn token, required as both a TOKENSAVE_WORKER_TOKEN env var and as the first 32 bytes received on stdin. Direct invocation by users fails. Defaults to available_parallelism() workers; opt out with TOKENSAVE_DISABLE_SUBPROCESS=1.

Edit primitives (tokensave_str_replace, tokensave_insert_at, etc.) still run in-process: they target one file at a time where subprocess overhead would dominate, and an extractor crash there is immediately visible to the agent.

---

Multi-Branch Indexing (Optional)

tokensave can optionally maintain a separate code graph per git branch. When enabled, switching branches never gives you stale results and never re-indexes files you already parsed on another branch. Multi-branch tracking is opt-in -- without it, tokensave uses a single database for all branches.

How it works

When you track a branch, tokensave copies the nearest ancestor DB and syncs only the files that differ. This means tracking a feature branch off main is nearly instant -- it only parses the files you've changed.

CLI commands

tokensave branch add              # track the current branch
tokensave branch list             # see tracked branches and DB sizes
tokensave branch remove <name>    # stop tracking a branch
tokensave branch removeall        # remove all tracked branches except default
tokensave branch gc               # clean up branches deleted from git

Cross-branch MCP tools

Three MCP tools enable cross-branch queries without switching your checkout:

• tokensave_branch_search -- search symbols in another branch's graph
• tokensave_branch_diff -- compare code graphs between two branches: symbols added, removed, and changed (signature differs). Supports file and kind filters.
• tokensave_branch_list -- list tracked branches with DB sizes, parent branch, and sync times

Branch fallback

When the MCP server can't find a database for the current branch, it serves from the nearest ancestor branch's DB and includes a warning in every tool response suggesting you run tokensave branch add.

Automatic branch tracking (v7.3.0)

Once multi-branch mode is bootstrapped (a first manual tokensave branch add created the branch metadata), new branches can be tracked automatically instead of falling back to the ancestor DB. Two independent mechanisms cover this; projects in single-DB mode are never affected, and neither mechanism ever touches the default branch's database.

Git hook (on branch checkout). The post-checkout hook that tokensave install sets up recognizes a branch checkout (as opposed to a file checkout) and runs tokensave branch add in the background. That command is a no-op when the branch is already tracked or is the default branch, so ordinary switching between known branches costs nothing. The initial checkout of a fresh git clone and of a new git worktree add is a branch checkout too, and it can land on a branch that is not the default one (git clone -b feature, git worktree add -b feature); there the hook runs tokensave init first and tokensave branch add after it, in that order. A hook written by an earlier version keeps the body it was installed with — the installer never rewrites an existing one — so on those installs a fresh worktree still needs auto_track below, or a manual tokensave branch add.

Open-time auto-track (opt-in). When TokenSave::open runs — CLI command or MCP server start — and the active branch is untracked, tokensave can track it on the spot by copying the nearest tracked ancestor's DB and recording it in the branch metadata. This is gated by the auto_track config field (default false) or the TOKENSAVE_AUTO_TRACK environment variable, which overrides the config per-run (any value enables it except 0, false, no, off, or empty). The copy is the same near-instant ancestor-DB copy a manual branch add performs; no sync runs at that moment — the post-commit hook keeps the new branch DB fresh as you commit, or run tokensave sync to refresh immediately. Auto-tracking is strictly best-effort: any failure is reported as a warning and open() proceeds with the usual ancestor fallback, so it can never break a tool call.

In short: with the hook installed, checking out a new feature branch — including the branch a fresh clone or worktree starts on — transparently gives it its own per-branch graph; with auto_track enabled, even a branch created outside a checkout is picked up the first time tokensave opens the project on it.

See docs/BRANCHING-USER-GUIDE.md for the full guide.

---

Cross-Session Memory

Three MCP tools persist decisions and code-area context across sessions, stored in the per-project .tokensave/tokensave.db.

Tool	Purpose
tokensave_record_decision	Save a design/architecture decision with optional reason, files, and tags
tokensave_record_code_area	Mark a path the agent has worked in (touch counter + last_touched_at)
tokensave_session_recall	FTS5 query over saved decisions; pair with the two write tools

Use these so the agent doesn't have to re-explain architecture choices session-to-session.

---

Savings Ledger

Every MCP call writes an append-only row to ~/.tokensave/global.db (savings_ledger table). Inspect with tokensave gain:

tokensave gain                    # current project, last 30 days
tokensave gain --all              # all projects
tokensave gain --history --range 7d
tokensave gain --json

Dollar estimates use the existing pricing module (Sonnet input pricing, refreshed daily via LiteLLM).

---

Reproducible Benchmark

tokensave bench runs a fixed query set through tokensave_context and reports retrieval savings vs a full-file baseline (mirrors the CCE methodology):

tokensave bench                                    # ships with 10 default queries
tokensave bench --queries my-queries.toml --json
tokensave bench --max-nodes 5

Measured against this repo (tokensave itself) using the shipped generic query set:

#	Query	Baseline	Context	Savings	Files	Nodes
1	How is configuration loaded at startup?	45.3k	454	99%	4	5
2	Where are command-line arguments parsed and dispatched?	948	402	58%	3	3
3	How is the main entry point organized?	6.1k	251	96%	3	8
4	How are errors defined, wrapped, and propagated?	3.5k	819	77%	2	3
5	Where is logging or diagnostic output emitted?	8.6k	514	94%	6	14
6	How are tests organized and what test harness is used?	3.5k	818	77%	2	3
7	How is data persisted to disk or to a database?	11.9k	330	97%	3	6
8	How are async tasks or background work spawned?	29.4k	364	99%	2	3
9	How does the build wire up dependencies and initialize state?	10.9k	1.4k	88%	4	5
10	How are public API surfaces exposed (HTTP endpoints, library exports, or CLI commands)?	22.5k	235	99%	4	5

Aggregate: 88% mean retrieval savings (142.8k → 5.5k tokens across 10 queries).

The default query set targets patterns present in most application codebases (CLIs, daemons, services). Run it on your own project with tokensave bench to see your numbers, or write a tailored query file (--queries my.toml) for tighter recall.

Criterion bench against large real-world repos

benches/large_repos.rs is a criterion micro-benchmark that exercises the MCP tools end-to-end against four large open-source codebases pinned at constant refs. Each tool is driven by at least 5 queries with arguments (node ids, qualified names, file globs, …) sampled from the indexed graph once per repo, so timings are reproducible across runs.

Repos and pinned refs (defined in benches/repos.rs):

Repo	URL	Ref
polkadot-sdk	https://github.com/paritytech/polkadot-sdk	polkadot-stable2412
emacs	https://github.com/emacs-mirror/emacs	emacs-30.1
scipy	https://github.com/scipy/scipy	v1.14.1
node	https://github.com/nodejs/node	v22.11.0

Each repo is shallow-cloned (git init + git fetch --progress --depth 1 origin <ref> + checkout FETCH_HEAD) on first use and cached locally; subsequent runs reuse the checkout. Git output is streamed to the terminal so the multi-GB fetch shows real-time progress.

Tools covered (5 queries each). Read tools — search, context, callers, callees, node, by_qualified_name, signature, impact, body, files, complexity, doc_coverage, largest, hotspots, god_class, module_api, derives, dead_code, rank, coupling, circular. Write tools — str_replace, multi_str_replace, insert_at, and (if ast-grep is on PATH) ast_grep_rewrite.