llm-cli-gateway

"Without consultation, plans are frustrated, but with many counselors they succeed." — Proverbs 15:22 (LSB)

Secure local control plane for AI coding agents.

llm-cli-gateway lets supported MCP clients operate Claude Code, Codex, Gemini/Antigravity, Grok Build, Mistral Vibe, Cognition Devin, Cursor Agent, and configured HTTP API providers through one user-owned gateway while preserving native CLI sessions, local credentials, durable async jobs, validation receipts, and review workflows.

Why developers try it: use the client you are already in to delegate work to local coding agents, scope remote execution to registered workspaces, gate risky actions, survive disconnects, and collect auditable review evidence without turning those agents into a generic chat proxy.

Current signals: CI and security workflows pass on main, OpenSSF Scorecard is published, OpenSSF Best Practices is passing, releases use Sigstore signing, and the package is MIT licensed.

Quick Start

npm install -g llm-cli-gateway

Or use directly with npx from an MCP client:

{
  "mcpServers": {
    "llm-gateway": {
      "command": "npx",
      "args": ["-y", "llm-cli-gateway"]
    }
  }
}

What It Provides Today

llm-cli-gateway is a single-user MCP control plane for operating AI coding agents from supported local or remote clients. It is more than a thin CLI wrapper:

• Runs registered provider CLIs and configured HTTP API providers through consistent sync and async MCP tools.
• Persists long-running jobs, supports restart-safe result collection, deduplication, cancellation, and sync-to-async deferral.
• Tracks sessions, real CLI resume paths, structured response metadata, and cache telemetry.
• Supports cache-aware promptParts, including explicit Claude cache_control when opted in.
• Can run supported provider requests inside gateway-managed git worktrees for isolated multi-agent review and implementation loops with either the local file-backed or PostgreSQL session manager. Session-bound reuse and cleanup remain limited to the host that owns the filesystem artifact, even when PostgreSQL shares the session row across hosts. After session deletion, session_clear_all, or file-backed TTL eviction, both session managers retain a failed cleanup for retry by the host that owns the worktree. Deletion processed from another host removes no worktree and leaves that record intact. The database-side cleanup_expired_sessions function stages the same record instead of deleting a worktree-bearing session; it invokes no gateway observer and so attempts no removal itself. Grok, Devin, and Mistral require an explicit provider-native sessionId for a gateway worktree; fresh, createNewSession, and resumeLatest-only worktree requests fail closed because they cannot durably reselect it. Materialization suppresses repository, system, and global Git hooks, configured clean, smudge, and process checkout filters, sparse checkout, and lazy object fetching. Filter-dependent content such as Git LFS remains in its repository representation instead of executing host commands.
• Ships personal-appliance setup surfaces: HTTP transport with bearer-token auth, doctor --json, setup UI artifacts, provider setup snippets, Docker fallback, and checked release bundles.
• Remote web connectors use MCP OAuth discovery and authorization-code setup with static client or shared-secret gates. Client secrets are generated locally, stored only as hashes, and printed only by explicit copy-once commands.
• Provider CLI requests can select registered workspaces by alias via workspace; every HTTP/tunnel request must use a registered alias, session workspace, or [workspaces].default before provider execution. Local unrestricted filesystem access is the stdio transport.

Workflow Assets

The repo ships agent-ready workflow skills under .agents/skills for async orchestration, session continuity, multi-LLM review, implement-review-fix loops, retrospective evidence walks, secure approval-gated dispatch, and Personal Agent Config Kit operations. Nine caller-facing skills are bundled in the published npm package: async-job-orchestration, multi-llm-review, session-workflow, secure-orchestration, implement-review-fix, retrospective-walk, public-demo-session, least-cost-routing, and personal-agent-config-kit. Machine-readable DAG-TOML plans live under docs/plans and setup/install-plan.dag.toml for workflows that need deterministic sequencing and verification gates.

Skill packs can be updated outside the core npm release by placing skill directories in local, operator-controlled paths. The gateway loads bundled skills first, then [skills].paths, then LLM_GATEWAY_SKILLS_PATH, then ~/.llm-cli-gateway/skills when it exists; later roots override earlier skills by name. Each skill is a directory containing SKILL.md. A root may also carry skill-pack.json to pin expected SKILL.md hashes:

[skills]
paths = ["/opt/llm-cli-gateway/skills"]

export LLM_GATEWAY_SKILLS_PATH="/opt/team-skill-pack:/opt/incident-skill-pack"

{
  "name": "team-pack",
  "version": "1.0.0",
  "skills": [
    {
      "name": "incident-retrospective",
      "sha256": "<sha256 of incident-retrospective/SKILL.md>"
    }
  ]
}

The loader is intentionally local-only: it never fetches remote Markdown at startup. To update a pack, install or replace files through your package manager or deployment system, then restart the gateway so the advertised skills://... resources refresh.

The next documentation focus is provider-specific skill and DAG-TOML pairs for each outbound CLI and API-provider family: Claude, Codex, Gemini/Antigravity, Grok, Mistral Vibe, Devin, Cursor Agent, OpenAI-compatible endpoints, Anthropic Messages, and xAI Responses. The implementation plan is tracked in docs/plans/provider-workflow-assets.dag.toml, with each provider asset expected to cover install/login checks or token-env checks, session behavior, approval modes, cache/telemetry surfaces, failure modes, and a smoke-test gate.

Trust & Supply Chain

• CI runs build, lint, format, tests, package checks, and npm audit.
• Security CI runs actionlint, zizmor, shellcheck, typos, osv-scanner, gitleaks, and lychee.
• GitHub release installer artifacts are checksummed and signed with Sigstore keyless signing.
• npm releases use a generated prod-only shrinkwrap and release security audit; GitHub Actions Trusted Publishing exchanges the job's OIDC identity for short-lived npm publish credentials.
• The npm package intentionally ships a generated, prod-only npm-shrinkwrap.json so registry installs resolve the audited release tree. Release gates regenerate it from package-lock.json, compare for parity, and run a registry-fidelity consumer install before publishing.
• Socket behavioural alerts are documented in socket.yml and under "Security Considerations" below. shellAccess and shrinkwrap are reviewed package capabilities/configuration for this CLI appliance, not hidden install behaviour.

Personal MCP Appliance

The personal-appliance contract keeps that surface intentionally narrow: one trusted user runs the gateway on a machine or volume they own, connects one MCP endpoint, and lets supported clients operate local coding agents through workspace-scoped, approval-gated, auditable requests.

The product contract is documented in docs/personal-mcp/PRODUCT_CONTRACT.md. It defines the single-user scope, security posture, target support matrix, and provider-support verification gates. Public setup guides must not claim ChatGPT, Claude web, Claude Desktop, Codex, Gemini CLI, Gemini web, or Grok inbound support until the corresponding provider/client path has been verified.

This project does not provide hosted multi-tenant credential custody. Provider credentials stay on the user's machine or user-owned deployment volume.

For a single developer who works on several workstations and repositories, the Personal Agent Config Kit guide explains the Git-synchronised personal baseline, repository overlays, immutable context stamps, and workstation-safe provider continuity model. The baseline directory is confined to a non-symlinked descendant of the local home directory, and every publish or sync revalidates its configured origin fetch and push URLs against the HTTPS/SSH-only policy. The Kit is intentionally local-caller-only; Kit provider execution and recovery of an unadmitted attempt require healthy durable SQLite or PostgreSQL async-job admission. Kit mode supports Claude, Codex, and Mistral; every other provider fails closed while it is enabled. Kit scope never inherits the gateway process cwd: Claude and Mistral require a registered workspace selection or configured default, while Codex can also use an explicit absolute workingDir. Relative Kit workingDir values are rejected before filesystem or Git inspection. It disables cross-model validation tools while enabled and is not an HTTP/OAuth configuration or remote execution feature.

For a retention-pinned non-Kit Claude MCP request configuration, follow the local-only same-host recovery procedure. It accepts no SQL or arbitrary-path override. A valid recovery invocation emits JSON and exits 0 after success or 2 for a safe refusal; invalid usage or unavailable durable storage exits 1.

Release-readiness history is tracked in docs/personal-mcp/RELEASE_READINESS.md. Dogfooding evidence (which target LLMs guided setup, what unsafe suggestions were captured, and which findings were deferred from the initial personal-appliance rollout) is in docs/personal-mcp/DOGFOODING_RESULTS.md.

Current personal-appliance artifacts include:

• Streamable HTTP startup: LLM_GATEWAY_AUTH_TOKEN=<token> npm run start:http
• Machine-readable diagnostics: npm run doctor
• Go bootstrapper: installer/ with setup, doctor --json, start, stop, status, repair, upgrade, uninstall, print-client-config, and verified bundle download commands.
• Release packaging: on the public mirror, the release workflow builds Linux binaries on GitHub-hosted ubuntu-latest and builds Windows/macOS binaries on their GitHub-hosted platform runners. The private upstream uses its internal self-hosted Linux runner. Each release publishes checksummed platform bundles with the gateway, production dependencies, and a managed Node runtime; see installer/packaging/README.md.
• Docker Compose fallback: docker/personal.compose.yml + docker/Dockerfile.personal for users who already manage containers.
• Local setup UI artifact: setup/ui/index.html
• Provider setup snippets: setup/providers/
• Cross-validation tools: review_changes, validate_with_models, second_opinion, compare_answers, red_team_review, consensus_check, ask_model, synthesize_validation, job_status, job_result, and validation_receipt (plus the validation-receipt://{validationId} resource). review_changes and durable receipts require a SQLite or PostgreSQL validation-run store and are absent in Personal Agent Config Kit mode.

Install / Upgrade / Uninstall (single binary)

Windows PowerShell:

$Version = '<version>'
$Base = "https://github.com/verivus-oss/llm-cli-gateway/releases/download/v$Version"
$InstallDir = Join-Path (Join-Path $env:LOCALAPPDATA 'Programs') 'llm-cli-gateway'
$ExeName = "llm-cli-gateway-$Version-windows-amd64.exe"
$BundleName = "llm-cli-gateway-bundle-$Version-windows-amd64.tar.gz"
$Exe = Join-Path $InstallDir 'llm-cli-gateway.exe'
$Checksums = Join-Path $InstallDir 'SHA256SUMS'
$ChecksumBundle = Join-Path $InstallDir 'SHA256SUMS.sigstore.json'
New-Item -ItemType Directory -Force $InstallDir | Out-Null
Invoke-WebRequest -UseBasicParsing "$Base/$ExeName" -OutFile $Exe
Invoke-WebRequest -UseBasicParsing "$Base/SHA256SUMS" -OutFile $Checksums
Invoke-WebRequest -UseBasicParsing "$Base/SHA256SUMS.sigstore.json" -OutFile $ChecksumBundle
cosign verify-blob $Checksums --bundle $ChecksumBundle --certificate-identity "https://github.com/verivus-oss/llm-cli-gateway/.github/workflows/release-installer.yml@refs/tags/v$Version" --certificate-oidc-issuer "https://token.actions.githubusercontent.com"
if ($LASTEXITCODE -ne 0) { throw "Sigstore verification failed for SHA256SUMS" }
function Get-ReleaseSha256($Name) {
  $line = Select-String -Path $Checksums -Pattern "^[a-fA-F0-9]{64}\s+$([regex]::Escape($Name))$" | Select-Object -First 1
  if (-not $line) { throw "No SHA256SUMS entry found for $Name" }
  return (($line.Line -split "\s+")[0]).ToLowerInvariant()
}
if ((Get-FileHash $Exe -Algorithm SHA256).Hash.ToLowerInvariant() -ne (Get-ReleaseSha256 $ExeName)) { throw "Checksum mismatch for $ExeName" }
$env:RVWR_GATEWAY_BUNDLE_URL = "$Base/$BundleName"
$env:RVWR_GATEWAY_BUNDLE_SHA256 = Get-ReleaseSha256 $BundleName
& $Exe setup
& $Exe stop
& $Exe install-bundle
& $Exe start
& $Exe status
& $Exe doctor

The Windows installer keeps a stable llm-cli-gateway.exe command in %LOCALAPPDATA%\Programs\llm-cli-gateway and adds that directory to the user PATH. Do not script against release-versioned exe names after install.

# After downloading the binary that matches your OS/arch from a release:
cosign verify-blob SHA256SUMS --bundle SHA256SUMS.sigstore.json \
  --certificate-identity "https://github.com/verivus-oss/llm-cli-gateway/.github/workflows/release-installer.yml@refs/tags/v<version>" \
  --certificate-oidc-issuer "https://token.actions.githubusercontent.com"
sha256sum --check SHA256SUMS            # verify before run (or `shasum -a 256 --check` on macOS)
chmod +x llm-cli-gateway-<ver>-<os>-<arch>
./llm-cli-gateway-<ver>-<os>-<arch> setup
./llm-cli-gateway-<ver>-<os>-<arch> install-bundle    # uses the platform bundle URL/SHA256
./llm-cli-gateway-<ver>-<os>-<arch> start
./llm-cli-gateway-<ver>-<os>-<arch> doctor

# Upgrade: replace the binary, set the new bundle env vars, run upgrade.
./llm-cli-gateway-<new>-<os>-<arch> upgrade

# Uninstall: dry-run first, then run with --yes.
./llm-cli-gateway-<ver>-<os>-<arch> uninstall
./llm-cli-gateway-<ver>-<os>-<arch> uninstall --yes

Docker fallback:

LLM_GATEWAY_AUTH_TOKEN=$(openssl rand -hex 32) \
  docker compose -f docker/personal.compose.yml up -d
docker compose -f docker/personal.compose.yml run --rm doctor

Features

Core Capabilities

• Multi-LLM Orchestration: Unified interface for Claude Code, Codex, Gemini, Grok, Mistral (Vibe), Devin, and Cursor Agent CLIs
• Session Management: Track gateway session metadata and provider-specific continuity with persistent storage
• Gateway-owned worktrees: Run supported sync or async provider requests inside a managed git worktree with either the local file-backed or PostgreSQL session manager. Same-session reuse requires same-host durable ownership plus a matching live Git registration and gateway branch; manager-level named path collisions fail closed. A shared PostgreSQL session row does not transfer ownership of its filesystem-local worktree to another host. Grok, Devin, and Mistral require an explicit provider-native sessionId; fresh, createNewSession, and resumeLatest-only worktree requests fail closed. A worktree requires a registered workspace selected explicitly, through caller-owned session metadata, or by the configured default; it never inherits process cwd or combines with workingDir, addDir, or includeDirs. Materialization suppresses repository, system, and global Git hooks, configured clean, smudge, and process checkout filters, sparse checkout, and lazy object fetching. Filter-dependent content such as Git LFS remains in its repository representation instead of executing host commands. Session deletion, session_clear_all, and file-backed TTL eviction hide a durably owned worktree session while cleanup runs, under either session manager. If Git removal fails, the manager retains a durable cleanup-pending tombstone, blocks reuse, and retries cleanup when a manager is registered on the owning host. The record is finalized only once Git no longer registers the worktree, read back rather than inferred from the recorded path being absent: a worktree that was moved, or whose owner marker cannot be read, is not a removal. Deletion processed by a different host removes no worktree and leaves the owning host's record intact. The database-side cleanup_expired_sessions function stages the same tombstone instead of deleting a worktree-bearing session; it invokes no gateway observer and so attempts no removal itself. A tombstone is not bounded by retention.
• Token Optimization: opt-in prompt/response compaction via optimizePrompt / optimizeResponse, both defaulting to false. The 44% / 37% figures are v1.0.0 measurements on a specific corpus, not a guarantee: src/optimizer.ts is a deterministic phrase-stripper, and nothing measures a ratio at runtime.
• Correlation ID Tracking: Full request tracing across all LLM interactions
• Cross-Tool Collaboration: LLMs can use each other via MCP (validated through dogfooding)

Observability

• Durable Flight Recorder: Provider requests and responses follow [persistence].backend: SQLite writes to ~/.llm-cli-gateway/logs.db by default, while PostgreSQL writes them to the configured database with the other durable subsystems. A PostgreSQL failure leaves the recorder unavailable or degraded and is reported generically on health surfaces; it never falls back to SQLite. Existing rows are not migrated or dual-read when the backend changes. Records include correlation IDs, duration, and token usage where the provider emits it (today: claude on stream-json/json, codex, and the API providers; grok, gemini, mistral, devin and cursor emit no usage on their CLI wire). Two exclusions are worth knowing: cross-LLM validation seats write no flight-recorder row at all, so llm_request_result cannot read one back by correlation ID, and repository-review seats are additionally excluded by design so review evidence is not retained in a non-expiring table. The retry_count and circuit_breaker_state columns are constants on the async path, which is the production path; retry and circuit breaking apply only to the direct-execute fallback. Read history through llm_request_list and llm_request_result so per-principal ownership checks remain in force. For human browsing of a SQLite deployment: datasette ~/.llm-cli-gateway/logs.db
• Structured Metadata: Tool responses include machine-readable structuredContent (model, cli, correlationId, sessionId, durationMs, token counts)
• Cache observability resources: cache-state://global, cache-state://session/{id}, and cache-state://prefix/{hash} MCP resources return aggregate cache hit/miss/savings — tokens and hashes only, no prompt text. session_get includes a cacheState block when the session has prior requests.
• Provider capability inventory: provider_tool_capabilities and provider-tools://catalog expose the gateway request fields, supported/degraded provider controls, local skill/tool discovery, and safe config-surface hints for Claude Code, Codex CLI, Gemini/Antigravity, Grok CLI/API, Mistral Vibe, Cognition Devin, and Cursor Agent. doctor --json includes a compact provider_capabilities summary for setup assistants.

Cache-aware operation

Every *_request and *_request_async tool except devin_request / devin_request_async and cursor_request / cursor_request_async accepts an optional promptParts field that structures the prompt for better cache hit rates (the Devin and Cursor headless paths take a plain prompt only). The gateway concatenates the parts in canonical order (system → tools → context → task) so that the stable prefix bytes precede the volatile task tail unchanged across calls, letting each provider's automatic prompt-caching land on the same content hash each time.

{
  "promptParts": {
    "system": "You are a helpful code reviewer.",
    "tools": "You have access to Read, Grep, Bash.",
    "context": "<long stable context block — file dumps, etc.>",
    "task": "Review the changes in src/foo.ts for security issues."
  }
}

prompt and promptParts are mutually exclusive — pass exactly one.

Per-CLI capability matrix (prefix discipline is automatic via promptParts for all providers except Devin and Cursor, which have no promptParts surface; explicit levers are provider-specific):

CLI	Prefix discipline	Explicit lever(s)
claude	yes	promptParts.cacheControl + outputFormat: "stream-json" (Anthropic cache_control breakpoints on stable blocks; ttl="1h" forced)
codex	yes	none (OpenAI implicit)
gemini	yes	none (implicit server-side)
grok	yes	compactionMode / compactionDetail (context compaction: `summary
mistral	yes	none (implicit)
devin	no	plain prompt only
cursor	no	plain prompt only

Claude example (explicit cacheControl)

claude_request({
  promptParts: {
    system: "You are a helpful code reviewer.",
    context: "<long stable file dump>",
    task: "Review the diff.",
    cacheControl: { system: true, context: true }, // task is never marked
  },
  outputFormat: "stream-json",
});