AI Coding Agents Moved Your Bottleneck to Code Review

A study of 718 software firms found agents raise code output by about 30% with no measurable gain in finished work. Here is where the gain goes, and how to measure it on your own team.

Every team that adopts coding agents sees the same first chart. Commits go up. Pull requests go up. Somebody screenshots the dashboard for the all-hands.

Then the roadmap moves at roughly the old speed, and nobody can say why.

A new study has a good answer. Fiona Chen and James Stratton, economists at Harvard, tracked more than 700 software firms before and after they adopted AI coding agents. The agents worked: lines of code, commits and pull requests all rose sharply. The amount of work the firms actually finished did not measurably rise. The extra code piled up in one place, the review queue.

That changes what "faster with AI" should mean for your team. Your agents can already write faster than your people can read. The speed that matters now is how quickly a human can check what the agent wrote.

What the study measured

The paper is called "Artificial Intelligence in the Firm: Bottlenecks in Software Production". It is a job-market paper, the research a PhD economist presents while applying for faculty jobs, and the current version is dated August 4, 2026. It reached a wide audience this week through an Ars Technica write-up and the Slashdot summary of it.

The data comes from Jellyfish, an engineering analytics company that plugs into GitHub, Jira and similar tools for its customers. The authors worked with 725,938 workers at 718 firms between January 2021 and March 2026. They spotted agent adoption from real signals: Claude Code API usage, bot accounts showing up in GitHub, and agent signatures on commits and pull requests.

Then they compared firms that adopted agents with firms that had not adopted yet, month by month. Economists call this a staggered difference-in-differences design. In plain terms: firms adopted at different times because of procurement, security review and pilots, so the late adopters serve as a moving control group for the early ones.

Here is what changed after a firm adopted agents, per worker per month:

Lines of code added: up about 30%.
Commits: up about 20%.
Pull requests: up about 23%.
Jira issues resolved: no significant change. The statistical range rules out a gain bigger than 12%.
Jira epics resolved: no significant change either.

Issues and epics are the closest thing this data has to "finished work." The units that make up a roadmap showed no measurable gain while the raw material behind them grew by a quarter or more.

Where the extra code went

The review numbers explain it. After agent adoption, according to the paper:

The time from opening a pull request to merging it rose by 3.45 days, on a baseline of 7.03 days. That is about 49% longer.
The share of pull requests where a reviewer asked for changes went from 13% to roughly 25%. It nearly doubled.
Review comments per pull request rose about 35%.
The share of engineers doing code review rose about 14%.

Think of a restaurant that buys a second stove but keeps one person at the pass, checking every plate before it goes out. The kitchen cooks more. The checker now sends back twice as many plates. Diners wait longer, and the number of meals served barely moves.

The authors model software the same way, as two steps in a row. Coders write draft code, then reviewers catch bugs. When AI speeds up only the first step, they write, "complementary steps may serve as bottlenecks." And if AI-written code carries more bugs per line, each line needs more checking, which makes the second step slower on top of being busier.

Two details in the paper make this sharper.

First, the pull requests did not get bigger. Average size held steady. Teams were not drowning in one giant diff; they were getting more ordinary-sized ones.

Second, AI review is already everywhere and has not fixed it. By March 2026, nearly 80% of the firms had used AI tools in review at some point, and AI wrote 23.3% of all review comments. The bottleneck held anyway. Adding a bot that comments on pull requests does not take the human out of the merge decision.

The insight: measure the reader, not the writer

Most teams measure AI coding tools by output: suggestions accepted, lines written, pull requests opened. Those numbers all sit before the bottleneck. They will keep going up, and they will keep telling you nothing about whether you ship.

The more useful number is reading speed. How long does a change wait for a human? How often does it come back? How many people are spending their week reviewing?

This also explains a disagreement you have probably seen. A Karpathy post that circulated this weekend argues the gap in shared understanding of what LLMs can do is widening; Hacker News commenters quote it as saying "human review and comprehension are starting to fall behind." Some people say agents made them ten times faster. Others say nothing changed. Both can be right. The first group is counting code written. The second is counting software finished.

Open-source maintainers feel this first, because they cannot hire reviewers. On October 7, the QGIS project merged a policy banning LLM, AI and agent-assisted code from its main repository. Maintainers in the discussion cited a volume of AI contributions that human review could not keep up with. One contributor counted about 50 pull requests tagged "AI Assisted," roughly 40 of them closed without merging. That is the Harvard finding with no budget to absorb it.

You probably do not want a ban. You do want to know where your own queue stands.

Put this into practice: four moves for this week

None of the steps below come from the paper. They are practical responses to what it found, aimed at the reading step.

1. Measure your review time before and after agents

If your code lives on GitHub, the GitHub CLI (gh) can pull merged pull requests with their timestamps and reviews. Run this inside a repository:

gh pr list --state merged --limit 500 \
  --search "merged:>=2026-01-01" \
  --json createdAt,mergedAt,additions,deletions,reviews > prs.json

Then summarize it by month with a short Python script, saved as pr_stats.py:

import json, statistics as st, sys
from datetime import datetime

prs = json.load(open(sys.argv[1]))
def t(s): return datetime.fromisoformat(s.replace("Z", "+00:00"))

by_month = {}
for p in prs:
    m = p["mergedAt"][:7]
    hours = (t(p["mergedAt"]) - t(p["createdAt"])).total_seconds() / 3600
    asked = any(r["state"] == "CHANGES_REQUESTED" for r in p.get("reviews", []))
    by_month.setdefault(m, []).append((hours, p["additions"] + p["deletions"], asked))

for m in sorted(by_month):
    rows = by_month[m]
    print(m, len(rows), "PRs",
          "| median hours to merge:", round(st.median(r[0] for r in rows), 1),
          "| median lines changed:", st.median(r[1] for r in rows),
          "| changes requested:", f"{sum(r[2] for r in rows) / len(rows):.0%}")

Run python3 pr_stats.py prs.json. You get one line per month: how many pull requests merged, the median time from open to merge (which includes the author's own fix time, not only waiting for a reviewer), the median size, and the share that got a "changes requested" review. Mark the month your team picked up agents and compare both sides. If your numbers look like the paper's, with merges climbing while the wait and the bounce rate climb too, you have found the queue. A rough rule of thumb, mine rather than the paper's: if median time to merge is up by a quarter or more since adoption, or the changes-requested share has climbed by ten points, the review step needs attention before you buy more agent seats.

Two cautions. --limit 500 caps the pull: if your repository merged more than that since January, narrow the merged: date range and run it in chunks. And a single repository is a small sample, and holidays or a big launch can swing a month. Look for a trend across several months, not one spike.

2. Make the agent do the first review

The paper's model says more bugs per line means more human checking per line. So lower the bug rate before a human ever looks. Ask your agent to run the tests, paste the output into the pull request, and write a short description: what changed, why, and what it did not touch. Claude Code, Codex and most agent tools can do all of this in the same session that wrote the code.

A reviewer who opens a pull request with passing test output and a clear summary reads faster than one who has to reconstruct intent from the diff.

3. Give AI review the boring checks

AI reviewers are good at style, naming, obvious null checks and missing tests. Let them own those. Then tell your human reviewers explicitly that they can skip those categories and spend their attention on design, data handling and anything touching permissions or money. The paper shows AI comments piling up without removing the human step. The fix is to divide the work, not double it.

4. Watch reviewer load, not just reviewer count

The paper found the share of engineers doing review rose about 14%. That spreads the load, and it can also pull your strongest engineers out of building. Track who reviews what. If two senior people review most agent-written pull requests, they are the bottleneck, and every agent you add makes their week worse.

Honest limitations

This is one study, and it has edges worth knowing before you quote it in a planning meeting.

It measures adoption at the firm level. If a company bought agent licenses but only a few people used them heavily, those heavy users get averaged in with everyone else. The authors say this likely understates what individual heavy users gain. Personal tool use is invisible too: someone paying for their own subscription on the side does not show up in the Jellyfish data.

The firms are Jellyfish customers who agreed to research use. Companies that buy engineering analytics may not look like yours.

"Resolved Jira issues" is a proxy for finished work, not finished work itself. A team that ships through a different tracker, or closes big features as one ticket, will not fit neatly.

The results run through March 2026. Agents and review tools have changed since then, and a newer model with fewer bugs per line could shift the picture.

It is also a working paper, not yet peer reviewed. Jellyfish reviewed the manuscript only to check for proprietary disclosure, not for its conclusions.

None of that cancels the main pattern. It does mean you should treat your own numbers from step one as the evidence that matters.

The question to ask your team

The next time someone shows a chart of agent-written pull requests going up, ask one follow-up: how long did they wait for a human, and how many came back?

If you do not know, run the two commands above this week. Then decide where to put the next dollar: another agent, or another hour of review.

Sources: Fiona Chen and James Stratton, "Artificial Intelligence in the Firm: Bottlenecks in Software Production" (version of August 4, 2026); Slashdot summary of the Ars Technica write-up; QGIS QEP-428 pull request #387 (merged October 7, 2026); Andrej Karpathy on X (wording as quoted on Hacker News).

Medium metadata

Title: AI Coding Agents Moved Your Bottleneck to Code Review
Alternate titles: "Software Delivery Isn't a Coding Problem Anymore" / "Coding Agents Write 30% More Code. Teams Don't Finish More." / "This Changes How Engineering Teams Measure AI"
Subtitle options: (1) A study of 718 software firms found agents raise code output by about 30% with no measurable gain in finished work. Here is where the gain goes. (picked, trimmed for Medium) (2) More code, longer reviews, the same roadmap: what a Harvard study found about coding agents. (3) Your agents write faster than your team can read. Measure that.
Description: A Harvard study of 718 firms shows coding agents raise code and pull requests while review time rises about 49%. How to measure your own queue.
Tags: AI Agents (core beat), Software Engineering (broad reach), Code Review (the specific shift), Productivity (the business frame), Engineering Management (the reader who acts on it)
Article type and plan: Type B (authority: a mental model backed by new data). Post at 7 to 8 AM ET, let Medium distribute for 2 to 6 hours, send to subscribers only if it is performing.