OmniRoute's 89% Token Savings Is a Multiplication Problem, Not a Benchmark
The trending AI gateway's headline compression number comes from multiplying two other projects' marketing claims together. Its own docs show the formula, and a user's real analytics show 3%.
Open the compression documentation for OmniRoute, one of the fastest-climbing AI gateways on GitHub this summer, scroll to the stacked pipeline section, and you find the source of its most-repeated number sitting right there in a code fence:
combined = 1 - (1 - RTK savings) * (1 - Caveman input savings)
average = 1 - (1 - 0.80) * (1 - 0.46) = 89.2%
range = 1 - (1 - 0.60..0.90) * (1 - 0.46) = 78.4-94.6%
That is not a measurement. It is a formula, and the two numbers going into it are other projects' self-reported figures. The repository is admirably transparent about this, printing the arithmetic where anyone can check it. The problem is what happens to that number downstream, where "15-95% token savings, ~89% average" appears in the repository title, in aggregator write-ups, and in every social post about the project, stripped of the qualifier that makes it meaningful.
I want to walk through why this matters, because the underlying project is good and the number is doing it no favors.
What OmniRoute is, and why people care
OmniRoute is an MIT-licensed TypeScript gateway that exposes a single endpoint across a large number of model providers, with quota-aware automatic fallback, MCP and A2A support, and a local dashboard. Point Claude Code, Codex, Cursor, Cline, or Copilot at it and you route across providers, including free tiers, without changing your client. It first hit #1 on GitHub Trending on June 30, 2026, and has stayed near the top of the daily boards since.
The pitch is provider neutrality, and it is a real pitch. Every major cloud shipped a first-party Claude control plane this month. A community gateway that treats all providers as interchangeable is the natural counterweight, and that alone would justify the attention.
The compression stack is the second pitch, and it is the one I want to take apart.
The mechanism, which is genuinely clever
Compression in OmniRoute is a registry of composable engines rather than one algorithm (COMPRESSION_ENGINES.md, version 3.8.40). Two do the heavy lifting.
RTK targets command and tool output. It detects output classes like git status, Vitest, Cargo tests, Webpack builds, ESLint, Docker logs, and stack traces, then applies 49 JSON filters that strip ANSI sequences, progress noise, and repeated lines while preserving failures, warnings, and changed files. The upstream project reports 60 to 90 percent savings, and its README example takes a 30-minute Claude Code session from about 118,000 tokens to about 23,900.
Caveman condenses prose. It removes filler, hedging, and repeated context while preserving code blocks, URLs, JSON, and paths byte-perfect. Upstream reports roughly 46 percent on the input-compression side.
Stacked mode runs rtk -> caveman in order: squeeze the noisy tool logs first, then compress the remaining natural language. There are also structural engines that are genuinely interesting, including content-addressed deduplication across turns in a session, lossless columnar compaction of homogeneous JSON arrays, and an accessibility-tree filter that collapses repeated structural lines in browser-automation tool results.
The architecture is sound. Separating structural, lossless work from semantic, lossy work and running them in a defined order is the right design.
Where the number breaks
Three things go wrong between that architecture and the headline.
The inputs are borrowed marketing. The formula uses 0.80 for RTK, but RTK's own reported range is 60 to 90 percent. Eighty is the upper-middle of someone else's self-reported band, not a midpoint and not an independent measurement. Caveman's 0.46 is likewise upstream's figure. Compounding two optimistic self-reports produces a number more optimistic than either.
"Eligible tokens" is carrying enormous weight. The savings apply to the portion of your payload the engines can act on. If your session is mostly short prompts and model output rather than sprawling terminal logs, the eligible fraction is small, and 89 percent of a small fraction is a small number. The docs are honest about this scoping. The repository title is not.
Their own telemetry cannot verify it. Under "Known limitations," the docs say a stacked-pipeline step whose engine ran but produced zero percent savings returns null stats and does not appear in the engine breakdown, which makes it "indistinguishable from a step that was skipped." The instrument that would tell you whether stacked compression worked cannot distinguish "ran and saved nothing" from "never ran."
The receipt
That known limitation is not theoretical. Issue #4268, opened June 19 and still open, is a careful bug report from a user who set the default mode to stacked, set the auto-trigger threshold to 1 to remove any ambiguity, ran a heavy read-only inspection command through a coding agent, and watched the compression analytics not move at all while provider metrics confirmed the request was routed.
His baseline analytics over seven days are the most useful numbers published about this project anywhere:
Total requests: 475
Total tokens saved: 909,618
stacked: count 3, saved 18,918, avg 32%
ultra: count 472, saved 890,700, avg 3%
Four hundred seventy-two requests through ultra mode, averaging three percent. Three requests through stacked, averaging thirty-two percent. The headline says roughly eighty-nine.
Those are not directly comparable, and I want to be fair about that. Ultra is a different mode from stacked, this is one user's workload, and 909,618 tokens saved is a real and meaningful amount of money. But a prospective user reading "saves 15-95% tokens" has no way to arrive at "expect single digits on a typical mixed session unless your traffic is dominated by tool output."
There is a related trap worth knowing. The LLMLingua-2 semantic pruning engine depends on optional packages totaling roughly 800 MB, and the npm-published package, the standalone bundle, and the Docker image all ship without them by design. When they are absent, the engine fail-opens silently: text is returned unchanged, no error is logged. Selecting LLMLingua in most install paths is a no-op that looks like a feature.
Put this into practice
If you want the routing, run it, and measure the compression yourself rather than inheriting a number.
Install it, point one client at it, and leave compression off for a week. That gives you a real token baseline for your own workload, which no published figure can give you. Then turn on rtk alone, not stacked, because RTK is the engine with the strongest independent evidence and the clearest target. Coding agents that shell out constantly are exactly the traffic it was built for.
Use the /api/compression/preview endpoint before committing to a mode. It lets you run any mode against a representative payload and see the result, which is the honest way to answer "what will this do to my traffic" in about ten minutes.
Watch /api/context/analytics broken out by mode rather than trusting the total. Given the telemetry limitation, treat a flat stacked count as evidence of nothing rather than evidence of failure, and compare against your billing dashboard, which is the only number that cannot fail-open.
If you enable LLMLingua, verify it is actually running by confirming real prose shrinks and that the model downloaded into your data directory on first call. Silence means it is off.
Honest limitations
The star and contributor figures for this project are a mess, and I could not resolve them. GitHub's own repository pages returned 6.7k and 8k stars on two fetches during the same research session. Trendshift's snapshot lists 21.6k stars and 261 contributors. The repository title claims "500+ contributors." One aggregator described it as a 9.3k-star project. Provider counts drift the same way, from 160+ on a predecessor repository to 231+ on the Trendshift snapshot to 268+ in the current title. I report the range because I cannot verify a point value, and the spread itself tells you something about how carefully the project's numbers are maintained.
I have not run OmniRoute against my own traffic, and this piece is a documentation and issue-tracker analysis, not a benchmark. Someone who runs a genuinely tool-output-heavy workload through stacked mode may well land near the advertised range, and I would find that result more interesting than my skepticism.
The bug report is one user on one version, v3.8.28, on one provider path. It has no maintainer response and no label as of this writing, which I read as an unanswered question rather than a confirmed defect.
None of this makes OmniRoute a bad project. The engine registry design is better than most commercial equivalents, the docs disclose limitations that many projects would bury, and MIT-licensed provider neutrality is worth something on its own. My argument is narrow: the compression headline is a projection presented as a result, and the project would be more credible with a smaller, measured number.
What to do with this
The generalizable habit here has nothing to do with this repository. When a project publishes a compound performance claim, find the arithmetic. If the number was produced by combining figures from upstream dependencies rather than by measuring the combined system, treat it as a ceiling under favorable conditions rather than an expectation.
OmniRoute deserves credit for printing its formula where you can find it. Most projects would have published 89 percent and let you assume someone measured it. Go read the code fence, then go measure your own traffic, and let the difference between those two numbers teach you what your workload actually looks like.
Sources: OmniRoute repository, COMPRESSION_ENGINES.md, Issue #4268, Trendshift repository stats.