Over-Editing Is Why Your Coding Agent's Diffs Are Unreviewable

A paper out this week measures how much extra code models rewrite when fixing a bug, and finds that one sentence in the prompt beats a model upgrade

Ask a coding agent to fix a one-line bug and watch what comes back. The tests pass. The bug is gone. And somewhere in the diff, a helper function has been renamed, a loop has become a comprehension, three lines of error handling you wrote on purpose have been consolidated into one, and a variable that was called raw now reads payload. Nothing is wrong. Everything is different. You are now reviewing a refactor you did not request in order to approve a fix you did.

Three researchers gave that behavior a name and, more usefully, a number. Tongyao Zhu, Wei Hern Lim and Min-Yen Kan call it over-editing, and their paper landed on arXiv on September 3, accepted to EMNLP 2026 Main. The arXiv page carries no affiliation line; Hugging Face's Daily Papers lists the work under the National University of Singapore. The finding that should stop you: a preservation instruction in the prompt reduced it more than a larger model or a bigger reasoning budget did.

Why nobody could measure this before

The reason over-editing has gone unquantified is not that people failed to notice it. It is that "too much" needs a floor, and code repair does not come with one. If a model rewrites twenty lines to fix a bug, you cannot say twenty is excessive unless you know the fix could have been done in two. Real bug-fix datasets do not carry a certified minimum patch, and asking a human to judge minimality across hundreds of repairs is exactly the labor everyone is trying to avoid.

Zhu, Lim and Kan solved this by working backwards. They took 400 problems from BigCodeBench and injected controlled corruptions into the reference solutions at the level of the abstract syntax tree. Because they built the damage themselves, they know precisely what undoing it costs. Every repair task in the framework ships with a known minimal patch, which turns "the model edited too much" from a complaint into a measurable distance.

I find this the most interesting part of the work, more than any individual number. The move is the same one that made adversarial robustness testable: you cannot measure the gap to correct behavior until you manufacture a situation where correct behavior is known. Everything downstream follows from that choice.

What the numbers say

Over-editing is widespread, and the paper is direct that strong models are not exempt. High Pass@1 sits comfortably alongside unnecessarily large edits and added cognitive complexity, which is the most damning sentence in the abstract. The scoreboard that everyone optimizes, does it pass, is blind to the property that decides whether a human will merge it.

Adding a preservation instruction, telling the model to stay faithful to the original implementation, moved three things at once:

Average excess Levenshtein distance dropped from 0.195 to 0.131, roughly a third less unnecessary rewriting.
Added cognitive complexity fell 26.6 percent.
Pass@1 went up by 2.3 points.

That last bullet is the one worth arguing with. The instinctive read on a constraint like this is that you are trading correctness for reviewability, buying a smaller diff by asking the model to be timid. The measurement says the opposite. Restricting the blast radius made the repairs more correct, not less. My best guess is that a model given license to rewrite whatever it touches generates more opportunities to break something adjacent, and that a preservation instruction narrows the surface where new bugs can be introduced. That is a guess, not a finding, and I would rather flag it as mine than let it pass as theirs.

The result that should change budgets is what did not produce these gains. The paper reports that they do not simply follow from a larger reasoning budget or larger models. If your plan for reviewable agent diffs was "wait for the next model," that plan is now carrying a load-bearing assumption the evidence does not support.

That finding does not sit alone. The Holistic Agent Leaderboard, a 21,730-rollout evaluation across nine models and nine benchmarks costing roughly $40,000, reported as one of its headline surprises that higher reasoning effort reduced accuracy in the majority of runs. Two independent groups, different methods, same direction: turning the dial up is not a general-purpose fix, and on some axes it is actively the wrong move.

The training result, and why it is the harder problem

The paper does not stop at prompting. It asks whether minimal editing can be learned during post-training, and gets a clean split. Supervised fine-tuning overfit to the corruption patterns it had seen. Reinforcement learning gave the better out-of-domain trade between edit fidelity and retained performance.

That split is familiar to anyone who has tried to teach a model a behavior rather than a fact. SFT on a fixed corruption taxonomy teaches "when you see this shape, produce that patch," which is memorization wearing a training loop as a disguise. RL optimizes against the objective itself, so it transfers to damage the model has not been shown. Useful to know, and mostly useful to labs. For everyone else, the prompt result is the one you can act on before lunch.

Put this into practice

The lowest-friction version of this is genuinely one sentence, and you can run the experiment on your own repository this week.

1. Add a preservation instruction to your repair path only. Something on the order of: Fix only what is required to resolve the reported failure. Preserve the existing structure, naming, and style of the surrounding code. Do not refactor, rename, or reorganize anything the fix does not require. Put it in the prompt your agents use for bug fixes and nothing else. It belongs behind a flag, not in your base system prompt, for reasons I will get to.

2. Measure the diff, not just the outcome. You do not need the paper's framework to get a signal. On every agent-authored fix, log two numbers that your CI already has access to: lines changed, and lines changed in files the failing test does not touch. That second one is a crude proxy for excess, and crude is fine when your current measurement is zero. Watch the ratio before and after you add the instruction.

3. Make oversized diffs visible rather than blocked. Set a threshold on your repair path, something like fifty changed lines for a single-test failure, and have the bot post a comment when a fix crosses it. Blocking creates a workaround culture. A comment creates a conversation, and after two weeks you will have real data on where the threshold belongs for your codebase instead of my guess.

4. Try the instruction before you try the upgrade. If a model upgrade is on your roadmap partly because agent diffs are hard to review, run this experiment first. It costs a prompt edit and a fortnight. The upgrade costs a migration, a re-evaluation, and a bill.

5. Keep a rewrite escape hatch. Some fixes genuinely require restructuring, and a preservation instruction will fight you on those. Give your team an explicit way to run the repair path without it, and log how often they reach for it. If the answer is "constantly," your bug tracker is full of design problems wearing bug labels, which is worth knowing for entirely separate reasons.

Honest limitations

I want to be specific about what this evidence does not cover, because the practical advice above is stronger than the study underneath it.

The corruptions are synthetic. AST-level damage injected into BigCodeBench reference solutions is not the distribution of bugs in your tracker. Real bugs cluster around unclear requirements, stale assumptions, and integration seams; injected AST corruptions are, by construction, local and mechanical. The over-editing behavior almost certainly generalizes, since it is a property of the model rather than of the damage. The size of the improvement almost certainly does not transfer cleanly.

This is single-shot repair, not agentic work. The framework measures a model producing a patch. It does not measure an agent reading files, running tests, revising, and producing a final diff across a loop. Multi-turn agent workflows have more chances to accumulate incidental changes and also more chances to discard them, and nothing here tells you which effect dominates.

The 2.3 point Pass@1 gain has no variance attached in the abstract, and I could not read the full text. I attempted the arXiv HTML version while writing this and the fetch was rate limited, so everything above comes from the abstract, the metadata, and the paper's listing. That means I have not seen the exact preservation instruction they used, the model list, the seed count, or the error bars. The instruction I suggested in step one is my own construction consistent with the paper's description, not a quotation from it. If you want the exact wording before you commit, read the PDF yourself; the paper is CC BY 4.0 and the identifier is 2609.04061.

"Cognitive complexity" is doing a lot of work in that 26.6 percent figure and I cannot tell you from the abstract which implementation they used. Cognitive complexity is a real, defined metric with more than one implementation in the wild, and a 26.6 percent reduction means different things depending on which one is running.

And the honest counter-position: a smaller diff is not automatically a better one. A model that mechanically minimizes edit distance will leave duplicated logic in place rather than extracting it, and will patch a symptom rather than name a cause. Edit fidelity is a quality axis, not the quality axis. The paper says exactly this, positioning it as a distinct axis of code-repair quality, and the framing is more careful than the way this result is going to get quoted.

What to do with this

The most useful thing this paper does is name a property you have been feeling for a year without a word for. Every time you have looked at an agent's pull request and thought "this is technically fine and I do not want to read it," you were reacting to low edit fidelity, and you had no vocabulary to file the complaint under and no number to attach to it. Now there is both.

Whether the specific figures hold in your codebase is a question only your codebase can answer, and the experiment is cheap enough that you should stop reading and go run it. One sentence, one flag, two weeks of diff sizes. If your excess drops by anything close to a third, you have bought back the thing that actually gates shipping agent work, which is a human being willing to read the change.

And if it does not move at all, that is worth knowing too. It would mean your bottleneck is somewhere else, and you would have found that out for the price of a prompt edit rather than a model migration.

Sources: When Models Edit Too Much: On the Fidelity of Minimal Code Edits, arXiv 2609.04061 (Zhu, Lim and Kan, submitted 3 September 2026, EMNLP 2026 Main, CC BY 4.0); Holistic Agent Leaderboard, arXiv 2510.11977 (Kapoor et al., submitted 13 October 2025).

Medium metadata

Title: Over-Editing Is Why Your Coding Agent's Diffs Are Unreviewable
Subtitle: A paper out this week measures how much extra code models rewrite when fixing a bug, and finds that one sentence in the prompt beats a model upgrade
Tags: AI Agents, Software Engineering, Coding Assistants, Code Review, Machine Learning
Suggested kicker: Edit fidelity is a quality axis separate from correctness, and it is cheaper to fix than anyone assumed
Canonical: import from the fervorai.dev URL after publication