The Harness Effect: Writer Froze Six Models and Cut Agent Cost 41% by Changing Only the Orchestration Layer

A paper from Writer's co-founder and CTO puts a number on the orchestration layer, and it is larger than the spread across the entire model menu. Here is what the experiment actually measured, and how to run a version of it on your own stack.

Every argument about agent costs this year has been an argument about models. Which one is cheapest per token. Which one just halved its price. Which one finishes the task in four turns instead of nine. The layer sitting between the model and the task, the thing that assembles context, decides which tools to expose, sequences turns, and decides what to do when a call fails, has stayed conveniently unmeasured. That layer is where each company's real engineering lives, and showing your work there means showing your competitors what you figured out.

Writer showed theirs. On July 8 the company posted The Harness Effect to arXiv, a controlled swap with 22 locked evaluation tasks and six foundation models, changing exactly one thing: the orchestration layer. Everything else stayed frozen. On August 13 they shipped the product side of it, Palmyra X6 and a rebuilt agent, which is when most people noticed the paper existed.

The headline result is the one worth arguing with. Holding the models constant, the harness cut blended cost per task 41 percent, from $0.21 to $0.12. Median wall-clock time fell 44 percent, from 48 seconds to 27. Tokens per task fell 38 percent, from 14.2k to 8.8k. Task-completion quality landed at 0.78 to 0.81, which the authors themselves call directional at this sample size rather than an improvement.

Then there is the sentence that should stop you: on this workload, the orchestration layer moved cost per task more than the full spread of the model menu did.

Why this is the number that matters

Most teams are optimizing the wrong variable, and the reason is structural. Model choice is a decision you make in a meeting. Orchestration is a decision you make in a codebase, over weeks, with no vendor to buy it from and no procurement line to charge it to. So the meeting wins.

The paper opens with a term for what that produces: token maxing. You buy capability with tokens. Longer reasoning traces, more turns, wider tool payloads, bigger replayed contexts. Tokens per task grow faster than the value of the task. Falling per-token prices hide the pattern beautifully, because your unit price keeps dropping while your invoice keeps climbing, and it feels like adoption instead of waste.

Writer's co-founder and CTO Waseem AlShikh framed the customer version of this to SiliconANGLE: enterprises want token consumption to explode, because "it means adoption is happening," but they need costs to flatten.

Two findings inside the ablation are more useful than the headline.

First, the efficiency gain held across every model. Not one of the six got more expensive. The range was 33 to 61 percent cheaper. That is the part that transfers, because it says the savings are not a property of some model's quirks, they are a property of the loop.

Second, the quality gain did not transfer. It tracked baseline model strength almost exactly, a correlation of 0.99 across six points. Better harnesses make weak models cheaper and strong models better. If you are running a small model and hoping orchestration work will buy you accuracy, the paper says it will buy you a smaller bill instead.

There is also a metric buried in the abstract that I think is more legible than any percentage: task-completions per million tokens went from 54.9 to 92.0. That is a number a finance person and an engineer can both read without translation, and it does not move when a vendor changes its price sheet.

What the harness was actually doing

The paper groups the effect into six mechanism families, spanning what the authors call cache-shape discipline through failure-spend governance. That phrasing is doing a lot of compression, so here is the practical version, drawn from the paper's framing and the product notes Writer published alongside it.

Cache-shape discipline means keeping the front of your prompt byte-stable so prompt caching actually hits. The paper formalizes effective input price under caching, which matters because a cache miss on a 40k-token prefix is not a rounding error, it is the whole invoice. Most loops sabotage their own cache by injecting a timestamp, a session id, or a reordered tool list into the prefix.

Failure-spend governance means deciding in advance what a failed call costs you. An agent that retries three times on a malformed tool response, each retry replaying the full history, can spend more on the failure than on the success. Nobody budgets for this because nobody measures it separately.

The rest is recognizable to anyone who has run agents in production: adaptive effort by task complexity so simple questions get a direct answer instead of a plan, batching and sub-agent delegation instead of serial turns, carrying context through long sessions without replaying it wholesale, and recovering from interruptions rather than restarting.

None of that is exotic. That is the point. The paper's claim is not that Writer invented a new technique, it is that assembling the known ones carefully beats swapping the model.

How to run this on your own stack

You do not need Writer's product to test the claim. You need discipline about the measurement, which is the part most teams skip.

Start by instrumenting cost and wall-clock per completed task, not per call and not per token. Per-call metrics hide the exact behavior you are hunting, which is a loop burning six cheap calls where one careful call would have worked. Pick 15 to 25 tasks that represent real work, lock them, and never edit them again once you start. Writer locked 22. The locking matters more than the count.

Then freeze the model. Whatever you are running today, leave it alone for the duration. Every time you change the model mid-experiment you have thrown away your ability to attribute the result.

Now change one thing at a time, and go in this order:

Prompt prefix stability. Audit what varies at the front of your context. Move anything volatile to the end. Measure cache hit rate before and after. This is usually the cheapest win available and it takes an afternoon.
History compaction. Look at what you replay on turn nine that was already resolved on turn two. Summarize or drop it. Measure tokens per completed task.
Failure spend. Instrument retries separately from successes. Set a per-task token ceiling and see what fraction of your bill was spent on work that never completed.
Tool payload width. Count how many tool schemas you send on every single call versus how many the task could plausibly use.

Adopt task-completions per million tokens as the number you report upward. It survives price changes, model swaps, and vendor announcements, which makes it the only agent metric I have seen that is still true next quarter.

One honest scoping note: this work pays off with volume. Under a few hundred agent tasks a month, the instrumentation costs more engineering time than it saves in inference. Above a few thousand, it is the highest-return work on the board.

What the paper does not establish

This is a vendor paper, and you should read it as one. Writer built the harness, defined the baseline, ran the evaluation, wrote the paper, and sells the product. All 33 authors are on the Writer side, and the corresponding author is the company's CTO. That does not make the result wrong. It does mean the most contestable choice in the whole design, what counts as the "frozen conventional production loop" they compared against, was made by the party with an interest in the answer. Your loop may already do half of what theirs did not.

The sample is small. Six models, 22 tasks, one workload. The 0.99 correlation is across six data points, which is the kind of number that looks stronger than it is. The authors flag the quality result as directional themselves, and they are right to. And it is a preprint, so nobody outside Writer has refereed any of it.

The paper also does not decompose the 41 percent. Six mechanism families went in, one blended number came out. If you want to know whether cache discipline or failure governance did the heavy lifting, that answer is not in there, and it is the answer that would tell you where to spend your first week.

Separate the paper from the product claims, too. The 52 percent cost reduction, 48 percent speed gain, and 10 percent quality gain that Writer put in its launch materials come from pairing the harness with Palmyra X6, a model post-trained on Z.ai's open-weights GLM-5.2 and priced at $2 per million input tokens and $8 per million output. That is a product benchmark, not the ablation. The ablation is the 41 percent, and the 41 percent is the part that says something about your stack rather than about theirs.

The version of this you should believe

Strip out the vendor framing and a defensible claim survives: on a real workload, careful orchestration moved unit cost more than the difference between the cheapest and most expensive model tested. That is falsifiable, and you can falsify it yourself in about two weeks with tasks you already run.

If the result holds on your workload even at half strength, it changes what your next quarter looks like. The model selection debate gets shorter, the instrumentation work gets funded, and you stop treating every price cut announcement as strategy.

If it does not hold, you will have learned something more valuable than the paper offers, which is that your loop was already good and your money is going somewhere else. Either way you end up with a number you measured instead of a number a vendor gave you.

Go instrument cost per completed task. Everything else in this piece is downstream of that one decision.

Sources: The Harness Effect (arXiv:2607.06906); WRITER product announcement, August 13, 2026; SiliconANGLE on the Palmyra X6 launch.