OrcaReplay

English · 简体中文 · 日本語 · 한국어 · Deutsch · Français · Español · العربية

Your agent broke something at 2am. Replay it at 9am — exactly, offline, as many times as you like.

Record any coding agent. Reproduce the run byte-for-byte with no model called. Fork it from any step onto a different model and see who gets it right.

Built by the team behind OrcaRouter — one API key and one endpoint for Claude, GPT, Gemini, Grok, DeepSeek, Qwen and the rest. It is what orca setup points at by default, and what makes orca compare a single command instead of four provider accounts.

Find us: OrcaRouter All model APIs

Github Repos: OrcaCode Review · OrcaRouter Lite

Connect: X · Discord · Hugging Face · Ollama

Real output from one session — a Claude Code run recorded, replayed against the recording, then forked at checkpoint 4 onto two models and graded by npx tsc --noEmit. Nothing here is mocked up.

Try it in three commands

orca record claude              # your agent, unmodified, doing whatever it does
orca replay last                # the same run again — no network, no tokens, no charge
orca replay last --from 4 --model claude-haiku-4-5 --ui

The third line is the one people stay for: same files, same conversation prefix, different model from step 4 onward. The model is the only variable, which is what makes the answer mean anything.

npm i -g orcareplay

The three commands at the top need an agent installed, a key, a network and real tokens. If you have none of those yet, one command brings its own:

orca quickstart

It writes a small project with a genuine bug in it and a recording of an agent fixing that bug, then replays the recording against the project with no model called: two failing tests before, four passing after, three turns served from the trace and nothing spent. --full prints the whole timeline and the replay as it happened.

Read your agent's own system prompt

A proxy that sees the whole loop also sees the prompt the harness assembled before it sent anything. One command captures it, scrubs the machine out of it, and files it by model:

node capture/capture.mjs claude --model claude-opus-5

Interactive prompts and -p prompts are not the same prompt, and neither is the same across models. See capture/README.md for the measured differences, the pitfalls, and the sanitising rules.

Why this exists

Agent debugging today is archaeology. You scroll a terminal, you re-run and get a different failure, you add print statements to someone else's harness. The tools that exist are observability tools: they tell you a run cost $4.12 and used 61k tokens, which is not the question you have. The question you have is why did it delete my migration file.

OrcaReplay answers that by giving you the run back.

	Observability tools	OrcaReplay
Tells you what a run cost	✅	✅
Tells you which tool call deleted the file	sometimes	✅
Runs the agent again and gets the same answer	❌	✅ from the recording, byte-for-byte
Lets you change the model and re-run from step 4	❌	✅
Needs you to modify your agent	usually an SDK wrapper	❌ two env vars
Works after you close the terminal	❌	✅ it is a file
Sees past the model API — shell exit codes, file writes	❌	✅ every turn
Records an agent with no API endpoint to redirect	❌	✅ opt-in --tls-intercept

The last two rows are the ones an SDK wrapper structurally cannot reach. Capture happens below the agent — at the process and socket boundary — so it does not matter whether the agent is yours, whether you can edit it, or whether it even holds an API key: a Codex CLI signed in with a ChatGPT subscription talks to its own backend over TLS and has no base URL to point anywhere, and orca can still record it. See when the harness will not be redirected.

How it works

Model APIs are stateless, so on every turn an agent resends the entire conversation — including the previous turn's tool results. A proxy in front of the model therefore sees the whole loop: each request, each streamed response, every tool call the model emitted, and every tool result the harness produced. That one property is what the tool is built on, and it is why OrcaReplay does not patch your agent — it stands up a local proxy, sets two environment variables, and gets out of the way.

What "egress blocked" means, exactly. On replay the proxy refuses to forward anything it cannot serve from the trace, so no model is called and no tokens are spent — that run prints egress=blocked. --loose lifts it deliberately: an unmatched request is then answered by the provider and recorded as a major divergence, and the same line reads egress=live-on-unmatched. A replay still executes the recorded tool calls for real, and a tool that opens its own socket — a shell command running curl, an MCP server fetching something — is outside the guarantee either way. By default it never reaches the proxy and goes to the network as usual. Under --tls-intercept it does reach the proxy, because that sets HTTPS_PROXY for the whole child: a host off the intercept list is tunnelled through untouched, though its hostname, port and byte counts still land in the trace, and a host on the list is refused there like any other unmatched call. Replay is not a sandbox; if you need one, run it inside one.

Three more layers catch what the protocol cannot see: an exit code, a real duration, which stream a byte came out of, a file written without telling anyone. A fifth exists for the agents that read no base-URL variable at all — see which agents.

%%{init: {'theme':'neutral'}}%%
flowchart LR
    A["<b>your agent</b><br/><i>unmodified</i>"]

    subgraph orca["orca · five capture layers"]
        direction TB
        P["<b>proxy</b><br/>base-URL env var"]
        SH["<b>PATH shim</b><br/>exit code · timing · streams"]
        MC["<b>JSON-RPC tee</b><br/>MCP config rewrite"]
        FS["<b>shadow git index</b><br/>workspace per turn"]
        FH["<b>fetch hook</b><br/>for a hardcoded origin"]
    end

    A --> P & SH & MC & FS & FH
    P -->|"forwarded, auth intact"| U["<b>the model API</b><br/><i>or OrcaRouter · any gateway</i>"]
    orca ==> T[("<b>one trace</b><br/>.orca/runs/run_a1b2c3")]

They all land in the same timeline, ordered by when they actually happened rather than when orca got around to reading them.

Exact, fork and compare are one thing

They are not three subsystems. They are the same proxy with a cursor — the position in the recorded stream where it stops answering from disk and starts answering from the network.

%%{init: {'theme':'neutral'}}%%
flowchart LR
    subgraph disk["from disk · byte-for-byte · network blocked"]
        direction LR
        T1["turn 1"] --> T2["turn 2"] --> T3["turn 3"] --> T4["turn 4"]
    end
    T4 ==> CUR{{"<b>cursor</b>"}}
    CUR ==> T5
    subgraph net["from the network · any model you name"]
        direction LR
        T5["turn 5"] --> T6["turn 6"] --> T7["…"]
    end

command	where the cursor sits	what you get
orca replay last	at the end	the whole run again, network blocked — no tokens, no charge, no variance
orca replay last --from 4 --model X	at checkpoint 4	turns up to 4 identical, then a different model takes over
orca compare last --from 4 --models a,b	at checkpoint 4, several times	one table, one variable — the model

A checkpoint is not recorded; it is derived — any point where the conversation prefix is complete and the workspace was snapshotted. Every fork therefore starts from a state that provably existed.

What a bug hunt actually looks like

Your agent was supposed to fix a failing auth test. It exited 0 and the test still fails. Start with what it actually did:

$ orca show last
run_6473f858b59e  generic-openai@0.1.0  14 events  exit 0

SEQ  KIND   WHAT                                            DETAIL
0    RUN    run started                                     generic-openai
1    SNAP   tree 919d32ba037537b43814c83779963b2cc3023db7   0 changed
2    MODEL  claude-opus-5                                   1 messages
3    MODEL  claude-opus-5                                   stop: tool_use · 100 in · 20 out
4    TOOL   edit_file                                       {"path":"auth.ts",…}
5    SNAP   tree c6af62b75c0c8b8938bd6087328b5148f3dcd534   1 changed
6    FILE   auth.ts                                         modified +1 −3
7    TOOL   edit_file                                       ok
8    MODEL  claude-opus-5                                   3 messages
9    MODEL  claude-opus-5                                   stop: end_turn · 101 in · 5 out
10   SNAP   tree c6af62b75c0c8b8938bd6087328b5148f3dcd534   0 changed
11   SHELL  ["sh","-c","node --check nonexistent-file.ts"]  /tmp/hunt
12   SHELL  shell result                                    exit 1 · 43ms
13   RUN    run ended                                       exit 0

info usage input=201 output=25 cost=$0.004890

Three facts the model's own transcript could not have told you, and the run's exit code hid: the file really changed (seq 6, +1 −3), the check the agent ran failed (seq 12, exit 1), and it finished anyway. The run exited 0 because the agent exited 0.

That last fact is the one worth a command of its own. orca show gives you the order things happened in; orca graph gives you what produced what:

$ orca graph last
FROM              TO               KIND      WHY
3 model.response  4 tool.call      recorded  tool_use block in the response
4 tool.call       6 fs.change      inferred  changed path appears in tool input, same or previous turn
4 tool.call       7 tool.result    recorded  tool result answers its call
7 tool.result     8 model.request  recorded  tool_result block in the request
11 shell.exec     12 shell.result  recorded  shell result answers its exec

  1 inferred — derived from this trace, not recorded in it

Two kinds of edge, and the difference matters. A recorded edge was written when the run happened, because a tool_use block is physically inside the response that emitted it. An inferred edge was worked out just now by the rule it names — a filesystem snapshot is taken once per turn rather than once per tool call, so attributing a file change to a particular call is a good guess and not a fact. Inferred edges are never written back into the trace, the same way checkpoints are derived and never recorded, so a field a third-party reader trusts never contains something orca made up.

--graph-card draws the whole run that way — time left to right, kind of thing top to bottom, with the chain that produced the failure lit against everything else:

The shape is the point. A run is one motif repeated — request, response, call, effect, result — so anything that breaks it is worth a look, and an event with no edge leaving it is an absence a list cannot show at all.

orca export last --card bug.svg draws just that chain, which is the version that fits in an issue or a message:

Nothing picked the subject by hand — --to was not passed. The card carries its own legend because a dashed line travelling without its trace would otherwise launder a guess into a fact, and it prints the command that reproduces it.

SVG renders in a GitHub issue and almost nowhere else that matters — X will not take it as an upload, and Slack and Discord give it no preview — so name the file .png and you get one, or .gif and the chain builds a hop at a time. That path needs a browser, and orca does not depend on one: docs/media/README.md keeps the render toolchain out of package.json so nobody running npm ci pays for a Chromium download, and a picture command is not a reason to reverse that. Ask for a raster without it and orca says the one line that fixes it; orca doctor reports it either way, and .svg never needs anything.

orca export last --card bug.png       # the chain, ready to post
orca export last --card bug.gif       # the same chain, one hop per frame
npm i --no-save playwright-core pngjs gifenc   # only needed for the two above

Now reproduce it as often as you like, for nothing:

$ orca replay last
info replay.done reused=2/2 exact=2 divergences=0 unmatched=0 exit=0

No network, no tokens, no variance. Then ask the question you actually have — would a different model have got this right?

$ orca compare last --from 5 --models claude-opus-5,claude-haiku-4-5 --verify "npm test"
MODEL             VERDICT  TOKENS  COST       WALL  RUN
claude-opus-5     pass     201/25  $0.004890  0.3s  run_1457b35062ba
claude-haiku-4-5  pass     201/25  $0.000326  0.3s  run_b8ee08479fb6

Both pass. One costs 15× less. Same files, same conversation prefix, same checkpoint — the model is the only thing that changed, which is the only reason that number means anything.

The timeline

orca replay last --ui (or orca ui) opens the run as one self-contained HTML file — no server to keep running, no network, nothing to install. Filter it, step it, or press space and watch the run play back at the pace it actually happened.

Every layer lands in the same timeline, so you can read the run as one story rather than four: the model turns and their token counts, each tool call with its arguments and result, the shell commands with their exit codes and timing, and the filesystem changes with the tree they produced.

orca export last -o bug.html writes exactly that page to a single file you can attach to an issue. It carries no external reference of any kind — CI asserts that — so it renders from a download folder, on a plane, in five years.

Same task, different model

orca compare forks one recorded run onto several models from the same checkpoint, with the same files and the same conversation prefix, and grades each one with a command you choose. The model is the only variable, which is what makes the answer mean anything.

orca compare last --from 4 \
  --models claude-sonnet-5,claude-haiku-4-5 \
  --verify "npm test" \
  --share verdict.svg          # the card above, ready to paste into an issue

Pointing it at several models

Comparing models means reaching several providers, and doing that by hand means knowing that --upstream-anthropic and --upstream-openai exist, that one gateway can serve both wire formats, and where the key goes. All of that is real and none of it is discoverable, so there is a command that asks instead:

$ orca setup
Gateway URL (serves the model APIs) [https://api.orcarouter.ai]:
  get a key at https://www.orcarouter.ai/console/token — OrcaRouter keys start sk-orca-
API key (stored 0600; leave blank for none):
  info config.saved path=~/.config/orca/config.json mode=0600 gateway=https://api.orcarouter.ai auth=stored

  6 models available:
    anthropic/claude-opus-5
    anthropic/claude-haiku-4-5
    openai/gpt-5.2
    ...

$ orca models
MODEL                      $/MTOK IN  $/MTOK OUT
anthropic/claude-opus-5    15         75
anthropic/claude-haiku-4-5 1          5
openai/gpt-5.2             1.25       10
some-local-model           —          —

orca setup asks the gateway what it actually serves rather than just writing the file, so a wrong URL or a dead key is an answer now instead of a 401 in the middle of a comparison. It also stores the models you picked, so after that orca compare last --verify "npm test" needs no model list and no upstream flags at all. orca models prices what it recognises and shows a dash for what it does not, because inventing a number for an unknown model is how a comparison table ends up quoting a cost that was never real.

OrcaRouter is the default answer to that first question — press Enter and you have one origin and one key serving Claude, GPT, Gemini, Grok, DeepSeek, Qwen and the rest, which is exactly the shape orca compare wants. Its model ids are namespaced by provider (anthropic/claude-sonnet-4.6, openai/gpt-4o-mini), which orca handles: the namespace picks the wire format and is stripped before pricing.

It is a default, not a destination: type over it, or pass --gateway <url>, and anything that speaks the OpenAI-compatible /v1/models and chat endpoints works just as well — another hosted gateway, or something you run yourself.

It is also only ever a default for traffic you asked to send somewhere. With no gateway configured, orca record proxies your agent's own calls straight to whatever provider it was already talking to, on the agent's own key. Orca does not reroute a recording you never configured: that would post your source code to a third party as a side effect of pressing record.

Non-interactive: orca setup --key <k> takes the default, orca setup --gateway <url> --key <k> names another, and --key-env <VAR> reads the key from the environment rather than keeping a credential on disk.

The key never reaches a trace. It is attached to the outbound request only, while what gets recorded is built from the incoming request with auth stripped — so it is invisible to the recording by construction, not by a rule someone has to remember. It is withheld entirely if a flag sends that traffic somewhere other than the gateway that issued it.

Which agents

Two things decide whether a harness can be recorded: whether it can be pointed at the proxy, and whether orca understands the wire format it speaks once it arrives.

Agent	How it is captured	State
Claude Code	ANTHROPIC_BASE_URL	works — validated against a real bug fix, in detail
Codex CLI (API key)	OPENAI_BASE_URL → Responses API	works
Codex CLI (ChatGPT login)	--tls-intercept → Responses API	works, with a decision to make
OpenAI Agents SDK	OPENAI_BASE_URL → Responses API	works — the SDK itself, on the Responses API it defaults to, records and replays at exact=1, in CI; its own tracing is a second egress
Vercel AI SDK	fetch hook — orca record node -- node app.mjs	works — an agent posting to an origin compiled into its source records and replays at exact=1, in CI
grok-cli (and its Telegram bot)	orca record grok — GROK_BASE_URL, plus the hook for its sub-agents	works
OpenClaw	orca record openclaw — the hook for the gateway, inherited variables for the agents it spawns	works
opencode	orca record opencode	adapter shipped, both origins redirected
goose (Block)	orca record goose — OPENAI_HOST and OPENAI_BASE_URL, ANTHROPIC_HOST → Responses API	works — driven end to end against goose 1.49.0, what is different about it
LangGraph / LangChain	OPENAI_BASE_URL, ANTHROPIC_BASE_URL	works — a two-node graph, streaming and with a tool, records and replays at exact=2 and forks live, in CI
OpenHands	orca record generic-openai -- python your_agent.py — its SDK wraps LiteLLM and reads OPENAI_API_BASE	works — the SDK's own LLM layer records and replays at exact=1, in CI
CrewAI	orca record generic-openai -- python your_crew.py — since 1.x its own provider, reading OPENAI_API_BASE and OPENAI_BASE_URL	works — a real Agent, Task and Crew records and replays at exact=1, in CI; what 1.x changed
Aider	orca record generic-openai -- python your_agent.py — routes through LiteLLM, which reads OPENAI_API_BASE	works — the LiteLLM layer records and replays at exact=1, in CI
browser-use	orca record generic-openai -- python your_task.py — its ChatOpenAI passes an unset base_url straight through	works — records and replays at exact=1, in CI; LLM layer only, the browser is not driven
Hermes (Nous Research)	ORCA_BASE_URL_VARS=… orca record generic-openai -- hermes …	should work — it overrides per provider; name the variable
Codex-in-the-IDE	orca record exec --tls-intercept -- code .	works — the extension spawns the agent, and it inherits the capture
a bot with a hardcoded origin	orca record exec --tls-intercept -- <cmd>	works — a Grok bot posting to a URL in its own source, in detail
an agent in a sandbox or on another machine	orca attach	works — orca is reachable and prints what to export, in detail
anything else	orca record generic-openai -- <cmd>	works if it reads a base-URL variable; orca record node -- <cmd> if it does not

Claude Code, Hermes and goose have been driven end to end against the real harness, and each of them broke something. Claude Code broke four things; Hermes found a streaming exchange the proxy was dropping entirely; goose broke two more, and neither was in the adapter — one was the replay matcher, one was a run reporting success over a trace of nothing but errors. The rest are held to the adapter contract and to fixtures that record the exact variables each one sets, so a harness that renames the variable it reads turns a check red instead of producing an empty trace.

The harness that reads different variables