GVS5H Turns One Mid-Size AI Model Into a Coding Team That Keeps Notes

It matched Claude Fable 5 on hard coding problems with a plan file and fresh memory every call. The cost story is messier than the headline.

The usual way to get better coding answers from AI is to pay for a bigger model. A small team at Persis Capital in Toronto just published a benchmark result that points somewhere cheaper and stranger: give a mid-size model a folder of plain text files, make it forget everything between calls, and watch its score jump.

The project is called GVS5H, short for "Gorilla vs. 5 Humans," and it reached Hacker News on Saturday night under a bold title: five Qwen3.8-27B models match Claude Fable 5. On the 100 most recent hard problems in LiveCodeBench, a coding benchmark built from real contest problems, Qwen3.8-27B, an open-weight model anyone can download and run, went from 66.8% solved to 92.4%. Anthropic's Claude Fable 5 scored 90.4% in a single call.

That headline is mostly true, a little misleading, and less interesting than what is underneath it. The real finding is a design pattern you can copy into any agent setup this week. The cost claim, on the other hand, needs a careful read before you repeat it.

What GVS5H actually does

Forget "five models." There is one model, playing different roles, and every single call starts with an empty memory.

Picture a project manager who has a terrible memory but keeps excellent notes. Each morning they forget everything, read the notebook, decide the one next task, and hand it to a contractor who also remembers nothing and works only from the notebook. The contractor does the task and updates the notebook. Repeat.

That is the whole system. The paper in the repo describes a shared workspace, a folder with five files:

• task.md: the problem statement
• plan.md: the manager's overall strategy
• tasks.json: the task list
• notes.md: curated ideas, findings and partial work
• solution.py: the current best attempt

The loop runs like this:

1. Plan. The manager reads the problem and writes a short strategy (three to six sentences) plus three to six starter tasks.
2. Brainstorm. The first worker writes no code. It names the hard part, lists approaches and pitfalls, and adds them to the notes.
3. Manage. A fresh manager call merges the plan and notes into one task list, then either declares the problem done or names the single next task.
4. Execute. A fresh worker does that one task, rewrites solution.py, and rewrites notes.md as a cleaned-up whole rather than appending to it.
5. Verify. If there is a new candidate, the surrounding code runs it against the problem's public sample tests. A failing test overrides any "done" verdict and forces another round.
6. Finalize. If the loop ends without a clean finish, one last worker writes the final answer.

A few guardrails keep it honest. The loop stops after 10 manager-to-worker rounds. It stops early if the manager hands out the exact same task twice. The plan file is capped at 4,000 characters and the notes at 8,000, so neither can bloat into a second transcript.

Workers never talk to each other. Nobody sees the full history. Every call sees only the files and one assignment.

Why forgetting helps

This is the insight worth taking home. The authors read through the transcripts where the single call failed and the loop succeeded, and they found the same three things again and again.

The hard reasoning gets written down before any code. The plan and notes commit the key idea to disk, so the next call builds on it instead of re-deriving it.

Tangled goals get split apart. A problem with two competing requirements becomes two tasks, each done separately.

Every call stays short. This one surprised me most. Left alone with a huge token budget, Qwen3.8-27B sometimes just did not stop. The paper reports that eight of its single calls became unstable and burned the full 250,000-token budget without writing any code. On one problem the model repeated the same phrase 7,743 times.

Bounded calls with state on disk fix that. In the authors' words, the workspace is "external memory that the token cap cannot truncate."

That is a useful way to think about any long agent task. The model's context window is short-term memory, and it degrades. A small file the agent rewrites each round is long-term memory, and it does not.

The results, read carefully

The authors ran six models, each on one fixed backend (the server or API that runs it), five independent passes each, and report averages with confidence intervals (error margins around each score). That is more rigor than most viral benchmark posts. Here is what they found, single call then with the loop:

• Qwen3.8-27B: 66.8% to 92.4% (the biggest jump, 25.6 points)
• Qwen3.8 Flash Next: 84.2% to 93.0%
• DeepSeek V4.1 Flash: 88.4% to 92.0%
• GPT-5.6 Terra: 80.8% to 88.0%
• GPT-5.6 Luna: 70.4% to 81.2%
• Claude Fable 5, single call only: 90.4%

Every model in this set improved. "Match" is the right word for the Qwen results, not "beat." The paper's own statistics show the gap between orchestrated Qwen3.8-27B and Fable 5 is within the noise.

Two caveats the authors state themselves. Fable 5 was run as a single call only, because it is expensive, so nobody knows how much the loop would add to it. And "gains are not universal": in other test conditions, some models were unchanged or got worse. One example in the paper shows the brainstorm step talking a model out of the correct optimization and into a slower plan that a worker then implemented with a bug.

These are also the project's own runs, on 100 problems, with no independent replication yet.

The cost claim, untangled

The headline you will see repeated is "at 9% of the cost." That figure is real, but it belongs to a different model than the one in the Hacker News title.

The paper prices every run at public list rates, with no caching or batch discounts. On those numbers, one full pass over the 100 problems cost:

• Claude Fable 5, single call: $61.11
• Qwen3.8 Flash Next with the loop: $5.76 (that is the 9%, at a slightly higher score)
• Qwen3.8-27B with the loop: $47.82

So the 27B model, the one in the headline, saved about a fifth of the cost, not 90%. Why so expensive for a small model? It talks a lot. The paper notes a single Qwen3.8-27B call emits more than twenty times the output tokens of either OpenAI model, and the loop multiplies calls.

The general rule holds across the table: the loop always costs more than the same model's single call, roughly two to four and a half times more in the paper's main table. It wins only when it is cheaper than switching to a bigger model.

If you run Qwen3.8-27B on your own GPU, list prices stop mattering and the math changes again. The authors served the Qwen models on their own machines with vLLM, an open-source model server, using FP8, a compressed 8-bit number format that cuts memory use. The paper does not report the electricity or hardware bill.

Put this into practice

You have two options: run their benchmark, or steal the pattern. The second is more useful for most people.

Steal the pattern in your own agent. If you use a coding agent like Claude Code or Codex on long tasks, you can copy the ledger without any new software:

1. Create plan.md and notes.md at the start of a hard task. Ask the agent to write a short plan and three to six concrete next steps.
2. Work in short sessions. End each one by having the agent rewrite notes.md as a clean summary of what is known, not a log of what happened.
3. Start the next session fresh, pointing the agent at the plan, the notes and one task.
4. Treat tests as the judge. A failing test overrides the agent saying it is done.
5. Set a stop rule. GVS5H uses 10 rounds and stops if the same task repeats.
6. Cap the files. If notes.md keeps growing, the agent is appending, not curating.

What to expect: more total tokens, fewer runaway sessions, and a paper trail you can read when something goes wrong. The paper also flags a trap to watch for. Every worker inherits the notes, so a wrong idea written down early can steer everything after it. If a task keeps failing, start one session from the raw problem with no notes and compare.

Run the benchmark yourself. The README's quickstart runs from the codebase/v2-current folder. You need Python 3.12 via uv, an API key for one of the supported providers (OpenAI, Anthropic, DashScope or OpenRouter), and a few environment variables the README sets first (LCB_RELEASE, ESCALATION_CLOUD_MAX_TOKENS, ESCALATION_CLOUD_TIMEOUT and MULTIAGENT_MODEL). Copy that full block from the README, including the line that exports the API key for your provider. The final command looks like this:

cd codebase/v2-current
uv run --no-project --python 3.12 --with 'datasets<4' --with numpy --with anthropic \
  python escalation/run_bench.py --engine multiagent --only lcb --lcb 100 --parallel 8

Expect it to spend real money at list rates, and pin one backend. The authors found that sending requests through a gateway that spreads them across several providers was the biggest source of run-to-run noise in their results.

Honest limitations

It is one benchmark. All results are code generation on LiveCodeBench. The authors tried math and knowledge benchmarks and dropped them because scores were already near perfect.

The verifier only covers part of the set. The sample-test check runs on 73 of the 100 problems. The other 27 use a test format the loop's code cannot run, so those get no feedback inside the loop.

Some comparisons are thinner than others. Five extra models ran through OpenRouter on an earlier, simpler version of the loop, with a single pass in some conditions. The authors report those separately for that reason.

Fable 5 had refusals. It declined 6 of 500 problem attempts, which the authors estimate cost it up to 1.2 points.

No releases, early project. The repo had about 900 stars and no tagged releases as of October 25. The code is MIT licensed; the paper, figures and run data are CC BY 4.0, and some third-party pieces keep their own terms (see NOTICE.md). Treat it as research code.

What this changes

The question most teams ask is "which model should we pay for?" GVS5H suggests a second question that is often cheaper to answer: "how should we organize the calls we already make?"

You do not need to believe the headline to use the idea. A plan, a curated notes file, short fresh calls and tests as the judge will make a long agent task easier to supervise even if your scores do not move. Try it on the next task that keeps running out of context, and see what the notebook catches.

Sources: GVS5H repository and README (master branch at commit 707e212, no releases, reviewed October 25, 2026) · GVS5H paper, paper_latest.pdf · Hacker News thread.