Experimental local compression proxy

The daemon can proxy Claude Code's Anthropic Messages requests through a listener on 127.0.0.1 with an ephemeral port and a random capability for each session. It is off by default. Enable compression.proxy: on globally, for a group, or for a session using the same compression settings precedence. Start or resume the session to route it through the proxy:

compression:
  proxy: on
  proxyTerse: false
  minTokens: 300
  ttlHours: 24

nid compression <group> proxy on and /compression proxy on set the group flag. Turning it off stops new tool-result compression for already proxied sessions; previously compressed results keep their exact view until restart or memo expiry. restart/resume removes their local routing. Proxy and injected prompt guidance have their own flags, independent of enabled for nid's own traffic.

Credentials and transport#

Only the built-in Claude driver with direct Anthropic routing is enabled. Existing ANTHROPIC_BASE_URL, Bedrock, Vertex, and Foundry routing is preserved, with a notice that the proxy was skipped. nid changes only the child's base URL. It does not obtain credentials, replace subscription login with an API key, or log request bodies, authorization headers, or capability URLs. OAuth bearer values, API-key values, anthropic-beta, and other end-to-end headers are forwarded. Transport headers such as Host, framing, and hop-by-hop headers are adjusted for the upstream connection. Anthropic documents that setting only a base URL retains subscription credentials, and that gateways must preserve the OAuth beta header. Claude Code gateway authentication.

Responses, including SSE bytes and unknown event types, stream through immediately. Only POST /v1/messages JSON requests are eligible for rewriting. Other paths and encoded bodies pass through. Bodies exceeding 8 MiB switch to unchanged streaming. Within that transport bound, synchronous work limits leave bodies over 2 MiB of UTF-8 completely unchanged (including shaping). Smaller requests cap new compression at 32 text blocks/512 KiB of UTF-8 combined; eligible memo replay retains its existing bytes without spending that new-work budget. Invalid UTF-8, invalid or ambiguous JSON, unsupported structures, and transformation errors preserve the original body. Disabling compression leaves new results alone. If the proxy cannot start, the agent launches with its original direct connection.

Upstream/network failures return a transport error. Once an inference request has been sent, the proxy does not automatically replay it: that could duplicate work and billing. Fail-open guarantees cover local request transformations, not recovery from a disconnected upstream or a dead daemon. Client retry behavior stays with Claude Code. Browser requests carrying Origin are refused; session capabilities are isolated and revoked when stopped or replaced.

A custom base URL disables Claude Code Remote Control, including for a subscription login. Doctor and the session-start notice call this out. Claude Code Remote Control requirements.

What changes in a request#

The engine examines only tool results in the final user message, paired with tool calls in the immediately preceding assistant message. It currently allows known test/build/check commands, gh pr checks and gh run view, and selected nid coordination tools. gh api is not compressed: its JSON (review comments, issues, changed files) is data the agent acts on item by item, and a sampled array would drop items from the middle. Unknown tools, file reads/edits, explicit source-viewing commands, and expansion-tool results are protected. Images and other non-text parts remain intact. The engine safety gates also apply, including source detection, minimum size, no re-compression and no-gain.

New compression splices only eligible JSON string spans. On subsequent requests, a per-session memo replays the exact previously sent content for a matching tool_use_id, error state, and SHA-256 of the original content bytes. It never re-summarizes historical results. All other whitespace, escaping, tool definitions, system blocks, thinking/signature blocks, and unrelated fields retain their original bytes. There is no history deletion or persisted transcript rewrite. Each compressed tool result keeps its original expansion ID, and savings are recorded under source proxy in /compression, the dashboard, and status reports.

The memo preserves the content the provider cached when the client resends the original result as history, including its exact marker and expansion ID. Claude moves cache_control breakpoints between requests, so the key covers content, not wrapper metadata; the client's current wrapper is preserved. The memo uses at most 128 entries/8 MiB per session, expires with the original's TTL, and is discarded on stop/restart. Missing, changed, expired, or evicted content passes through unchanged; this can cause a cache miss. Memo replay is counted in proxy traffic savings on each request because those bytes were actually reduced again. Real provider usage remains the authority for cached-token and net savings.

Optional stable terseness note#

compression.proxyTerse: true appends this constant text block to the end of the system prompt on every eligible Messages request, including the first request:

Be concise. Avoid repeating tool output; report conclusions, changes, and necessary next steps briefly.

It defaults to false and is captured at session start. Existing array blocks and their cache controls retain their exact bytes; a string system becomes an array with its original string as the first text block. Existing system content is never edited. The added system is identical across requests with the same client system. Restart/resume to change this flag. Turning only proxy off during an already proxied session leaves the captured note in place until restart, preserving its prefix. Enabling this feature for an existing conversation starts a different system prefix; enable it from session start for stable caching. The note adds input tokens; tool-result savings counters do not subtract that overhead or assume shorter model output. Use provider usage to assess its net cost.

Why thinking reduction is not implemented#

The proxy leaves thinking, budget_tokens, and effort settings unchanged, even on tool-only continuations. Anthropic explicitly documents that changing those settings invalidates message caches and may invalidate tool/system caches as well. A generic per-turn reduction would conflict with the cache-preservation goal and could reduce correctness on a tool result that needs substantial reasoning. Some models offer separate per-message controls, but this proxy has no verified cross-model compatibility contract for them. Anthropic thinking/cache troubleshooting, prompt cache hierarchy.

Why Codex proxying is not enabled#

Codex remains on its original transport. Its provider configuration supports Responses and optional WebSocket transport; its ChatGPT service/auth configuration is a separate concern and is not a documented universal inference redirect. Official Codex configuration reference.

Responses WebSockets can continue server-side state using previous_response_id, with connection-local caches and multiple conversation streams. A correct proxy would need a tested WebSocket state/retry implementation as well as HTTP support; disabling sockets or replacing provider/auth configuration just to intercept calls would change the client's behavior. We have no captured subscription-backend compatibility proof for that path. This is an implementation safety decision, not a claim that Codex proxying is impossible. Official Responses WebSocket documentation.

Verification#

test/compress-proxy.test.ts uses isolated original stores and a fake upstream. It checks exact history prefixes, existing system bytes, mixed text/image results, OAuth/beta/API-key headers, immediate SSE delivery, unchanged malformed/oversized/ encoded bodies, injected failures, live disable, capability revocation, stable system shaping, the Remote Control doctor notice, and a fake Claude child using the launch environment. It makes no live model calls and does not claim an agent task-outcome improvement; the real on/off team run is separate.

Live verification#

On 2026-10-09 UTC (2026-10-08 Pacific), Claude Code 2.1.294, model alias haiku, completed two turns through a separate source daemon using an existing first-party subscription login. Repeated with proxyTerse enabled. Each session ran pnpm test in a fresh temporary directory, producing 1,500 log lines with one planted ERROR QZ742: fixture checksum mismatch, then answered a follow-up without tools. All four answers identified the exact error correctly. The first measurements below were before the history-replay fix and exposed its need.

¹ nid's Claude input count includes uncached, cache-read, and cache-write tokens, summed across the model calls in each turn. These are provider usage counts, distinct from the local tokenizer's measurements. ² CLI-reported API-equivalent estimates on subscription billing, not charges. The runs shared warmed provider caches: the cost difference is not evidence that terseness saves money, nor an on/off compression benchmark.

Both sessions used one Bash tool call and three Messages requests. Local proxy counters recorded 8,040 → 190 tokens (terse off) and 8,040 → 189 (on), about 97.6% smaller for that tool result, including its expansion marker. In all six requests, hash/assertion receipts verified the compressor preserved earlier messages and the raw serialized prefix before the newest message. Each session's original system hash stayed constant. Optional system shaping is separately covered by the byte-preservation tests above. Responses streamed through the real client successfully; the unit suite verifies immediate SSE chunk delivery. Stopping each session made its capability URL return 404.

The original implementation compressed only the newest result and restored its full original when it became history. That caused roughly 21,000 cache writes on a tiny follow-up. The current memo fixes that by replaying the previously sent content. Whole-block hashing initially missed because Claude moves cache metadata; exact content hashing preserves that metadata while matching the result.

A second pre-fix execution also passed all four answers and revocation/prefix checks: tool-result counts were 8,042 → 189/190. In its terse session Claude additionally called expand, retrieved the original, and answered correctly. That session used 175,629 input/669 output tokens in the first turn, then 75,637/29 in the follow-up (75,273 cache reads), with a total estimated cost of $0.01017471. The ordinary session used 100,271/362 then 62,661/27, totaling $0.00825718. This confirms expansion works with the live client and illustrates why a compressed-block counter alone cannot establish net savings: agents may request the full output again.

After the memo fix, the controlled rerun required exactly one Bash call, with the answer taken directly from the displayed result, to keep expansion out of the cache comparison. All four answers, provider-visible tool-content continuity checks, and capability revocation checks passed:

Follow-up cache writes fell from 20,977/20,985 before replay to 264/133. Each run sent the 8,040 → 188-token result twice (initial compression plus one exact replay), so its proxy counter was 16,080 → 376 across two requests. Three requests per session shared one system hash. Unit tests prove exact raw prefix continuity with stable wrapper metadata and equivalent prefixes with client-managed cache breakpoints removed. The live receipts check the replayed tool content against its previously forwarded value; they also record client metadata/representation changes separately, without treating them as proxy edits. These are small smoke runs, not a statistically controlled task-outcome study.

Rerun explicitly from a source checkout:

node --import tsx scripts/proxy-smoke.ts

The script requires claude auth status to report first-party subscription authentication and refuses custom provider routing. It removes API-key variables, sets subscription billing and disables rate-limit API fallback. It creates its own temporary NID_HOME, uses free daemon/dashboard ports, ignores NID_BIN, and shuts down only that daemon in finally. Each group has a $0.10 budget including estimates; PROXY_SMOKE_MODEL can override haiku. This is an explicit live check, excluded from the ordinary test suite. It asserts correct answers, a single initial tool call, real proxy savings, identical replayed tool content, follow-up cache reads and fewer than 2,000 cache writes, plus revocation on stop. This controlled prompt avoids the expansion confound observed and reported above; it does not suppress expansion in normal use.

The printed temporary directory contains report.json (counts/outcomes), requests.jsonl (hashes/booleans only), and the isolated daemon's normal event database. No request bodies or credentials are added to debug receipts. capabilities.jsonl is local test state, never printed or included in the report; its URLs are revoked at teardown. cat/file-read tools are deliberately excluded from compression, so the fixture uses a test command instead.