Cut LLM spend. Keep the proof.
Trace every dollar, test cheaper models in shadow, and route only after quality holds. One base-URL swap gives your team an auditable path from visibility to savings.
- OpenAI + Anthropic
- Apache-2.0
- Self-hosted
- Air-gap safe
See the route before you take it
Live routing simulationWhy honesty-first
TokenTriage would rather say “I can’t prove that” than show you a pretty lie.
Competitors compete on provider count and throughput. We compete on trust — provenance, quality-proven routing, and a dashboard where every number survives an audit.
- One base-URL swapNothing else in your app changes.
- Ledger firstRouting is opt-in, and earned.
- Every dollar has provenanceStamped, and auditable.
- Self-host anythingSingle binary, no sidecar.
The problem
LLM spend is exploding — and opaque. Cutting it blind risks quality.
If more than one of these is true, there is almost certainly a number worth finding.
The bill outran the usage
Traffic grows 20%; the inference bill grows 90%. Nobody can point at the line that did it, because the invoice is one number per provider.
Finance asks a question engineering can’t answer
Which feature, which agent, which route, which token type? The data exists in logs and billing exports, but no one has joined the two.
The cheaper-model experiment got rolled back
Someone tried a smaller model for a week, quality complaints arrived, and it was reverted. Now nothing moves, because nothing can be proven safe.
The insight
Only cut cost when you can prove quality holds — auditably.
Cheaper models are safe to use far more often than teams dare, and dangerous exactly when no one is measuring. TokenTriage measures first: it costs every request, computes the cheaper counterfactual in shadow, and enforces a route only once the evidence clears the bar. When it can’t prove a win, it says so — and shows you why.
How it works
The trust ladder: ledger → shadow → route
You earn enforcement one rung at a time. Climb only when the evidence says the next rung is safe.
Ledger
See every dollar
Observe and cost every request — per model, per token type, per trace, per agent, per tool. Zero request mutation. This is where you start.
Shadow
Prove the savings are safe
Compute the would-route decision and its counterfactual cost, so the report shows the savings you’d get — without touching a single response.
Route
Enforce, only once earned
Rewrite the model, pick the cheaper upstream, honor cache pins — but only after the evidence says the next rung is safe. You never route blind.
The drop-in
One line changes. Then you can see it.
TokenTriage speaks the OpenAI (/v1/chat/completions) and Anthropic (/v1/messages) dialects. Point your base URL at it and every request is costed and logged.
from openai import OpenAI
# The only change: point the base URL at TokenTriage.
# Everything else in your app stays the same.
client = OpenAI(base_url="http://localhost:8787/v1")
resp = client.chat.completions.create(
model="gpt-5",
messages=[{"role": "user", "content": "Summarize this ticket."}],
)The trust dashboard
A dashboard where every number survives an audit
The overview: spend to date, projected period-end, and savings vs. baseline — each stamped with its pricing snapshot, EST flags where estimated, and UNPRICED rows counted, never hidden.
What you get
Three surfaces, one source of truth
Cost analytics & optimization
Every dollar attributed by model, token type, trace, agent, and MCP tool — with recommendations graded ESTIMATED or UNPROVABLE, and realized savings PROVEN only once measured.
Learn more →Quality-proven routing
A typed policy language and a shadow judge that answers one question honestly: does routing beat the zero-router baseline at its own cost — or can’t it be shown yet?
Learn more →The trust dashboard & provenance
Every number traces to one JSONL decision log. Estimates wear an EST flag; UNPRICED spend is an excluded slice, never rendered as $0.00.
Learn more →
Proof, not adjectives
Measured house benchmarks
Numbers from a run on our box (Intel Core Ultra 9 285K, Go 1.25.x, windows/amd64, CGO-free). Every figure is one an actual run produced — never extrapolated or invented.
≈0.065 ms
Median proxy overhead
Loopback micro-benchmark, three-arm. The <2 ms p99 figure is our CI budget, not a wire latency.
≈10.7 MB
Single static binary
linux/amd64, CGO-free. Container image FROM scratch ≈11 MB (estimated).
bytes.Equal
Byte-faithful passthrough
Proxied bytes equal going direct — streaming included. Errors fail open to verbatim passthrough.
Apache-2.0
Open source, self-hosted
One binary, air-gap-safe, UI embedded. No data files to ship.
The bigger picture
Toward the decision plane
The endgame isn’t a cheaper router — it’s the decision plane: the cheapest LLM execution plan that can be shown to meet each workload’s quality, latency, cost, and policy requirements. Cost reduction with bounded risk and defensible evidence.
Open core, honest gates
The ledger, verdicts, provenance, the shadow report, self-hosting, and exports are free forever — never crippled to upsell. Where paid value comes later (SSO, fleet management, long-retention hosted analytics) it is additive, never a fork. The honesty invariants apply at every tier.
See where your LLM spend goes
One binary. One base-URL swap. Five minutes to your first costed request — and not a single number you can’t trace.