Google's Harness² Improves AI Agents Without Touching the Model

Google Research's open-source loop rewrites the prompts, tools and skills around a fixed model, with no answer key. Here is how it works, how to try it, and the setting to change before you do.

When an AI agent does a sloppy job, the usual reaction is to wait for a better model. Swap the engine, hope the car drives better.

That reaction skips over something builders keep rediscovering. The same model can score very differently depending on what surrounds it: the system prompt, the tools it can call, the skills it can load, the memory it keeps. People call that wrapper the harness. And a growing amount of agent progress comes from changing the harness while the model stays exactly the same.

Google Research just published a framework that does that change automatically. It's called Harness² (say "harness squared"), it reached #19 on Trendshift's daily board on October 17, and its central trick is strange enough to be worth understanding: it improves the harness for a task without knowing the right answer to that task.

That is also its biggest risk. The project's own docs explain why, in one sentence most people will scroll past.

First, what a harness actually is

Picture a new hire on day one. Same brain they'll have in six months. What changes is the checklist on their desk, the tools they've been given access to, the notes from whoever had the job before, and the habit of double-checking their numbers before sending a report.

An agent's harness is that desk. The Harness² README lists eight parts it can edit, grouped by what they do:

• Context: the system prompt, guardrails, memory and skills (reusable instruction packs the agent loads on demand)
• Tools: guidance on how to use each tool, plus helper scripts
• Orchestration: subagents, meaning helper agents the main one can hand work to
• Middleware: plugins that sit between the agent and its tools

The model weights never change. Only the desk does.

If you use Claude Code or Codex, you already edit a harness by hand. Every change to a CLAUDE.md or AGENTS.md file, every new skill, every MCP server you connect (a plug-in that gives the agent a new tool) rearranges the desk. Most people make those changes by gut feel and never test them against the old version. Harness² automates exactly that part: try a change, watch what the agent does differently, keep it or drop it.

How Harness² improves the desk without an answer key

Most self-improving systems need a grader. Try something, check it against the right answer, keep what scored better. The trouble is that real work rarely comes with an answer key. A legal memo or a reconciled spreadsheet doesn't arrive with a test suite.

Harness² gets around this with a loop the paper calls Propose-Probe-Compose. In plain terms, for each task:

1. Run the agent once with its normal harness, as a baseline.
2. Propose edits. A "harness editor" suggests changes, some general and some specific to this kind of task.
3. Probe. Run the agent again with each candidate harness, on the same task.
4. Compose. Compare how the runs behaved differently and build a better harness from that evidence.
5. Deliver the task with the final harness.

The key word is differently. Instead of grading runs against a correct answer, it contrasts them. If one variant reopened its output files and caught a mistake and the baseline didn't, that behavior is evidence, even without knowing the final score.

After each task, a second loop writes down what worked in an "improvement library." The project page is clear that "each new task resets the executor to the base harness," so lessons carry forward, but the edited harness itself does not.

Here's the example the project page gives. On one JobBench task (a benchmark of professional work), the baseline agent "left source files unreconciled and never reopened its outputs." Harness² added verification guardrails, a reconciliation skill and sharper tool guidance. The task score went from 43.1 to 77.8.

Read that again and notice what got added. The fix wasn't a smarter model. It was the habit of checking its own work.

What the numbers say, and whose numbers they are

The paper, submitted October 9, reports that one improvement step lifted agents built on GLM-5 and Gemini 3.5 Flash over their base harnesses on three benchmarks: up to 11.5 points on WorkBuddy-Bench (office, web and code tasks), 4.5 points on JobBench, and 11.8 percentage points on LAB, a legal-work benchmark. Giving the loop more attempts per task kept raising scores.

The project page only shows full numbers for one setup, Gemini 3.5 Flash driving OpenCode (an open-source terminal coding agent). On WorkBuddy-Bench, the base harness averaged 72.4. One Harness² step got 77.5, and a bigger parallel budget (k=5, which tries five general and five task-specific candidate harnesses and builds five combined versions) got 80.8, ahead of the other harness-tuning methods it lists, including Meta-Harness at 76.4. Those are averages of three runs.

All of these are the authors' own measurements on benchmarks they selected. They're a reason to try it, not proof it will help your workload.

Try it in ten minutes

The project runs on Linux with Python 3.12 and uv, the fast Python package manager. Model-backed runs also need Node.js 20 or newer, the Google Cloud CLI and a Vertex AI project (Google Cloud's hosted model service). The README states that it is "not an officially supported Google product."

Start with the local demo. It needs no model credentials and makes no paid model calls:

git clone https://github.com/google-research/harness2.git
cd harness2
uv venv --python 3.12 .venv
source .venv/bin/activate
uv pip install -e '.[render]'
harness2-demo --output runs/demo --k 2

The --k flag sets the per-task budget. In parallel mode it sets how many general and task-specific candidate harnesses the loop tries, and how many combined versions it builds from them; in sequential mode it sets how many refine-and-retry steps run back to back. The demo is the cheapest way to watch the propose, probe and compose stages run end to end, though it tells you nothing about how a real model will score.

When you want real numbers, copy .env.example to .env, fill in your Vertex AI project and model IDs, and sign in with gcloud auth application-default login. The legal benchmark also needs Pandoc and Poppler installed (WorkBuddy-Bench needs Docker). Then run a single legal task, judge it and report:

harness2-setup lab
harness2-check --bench lab
harness2 --bench lab --k 1 --mode parallel \
  --only-domains antitrust-competition --max-tasks 1
harness2-judge --bench lab --k 1 --mode parallel
harness2-report --bench lab --k 1 --mode parallel

Add --dry-run first to see which tasks it would pick without making a single model call. Add --substrate cx to swap the agent doing the work (the executor, OpenCode by default) for Codex. Keep --max-tasks small: it caps tasks per domain, not per run, and the full LAB stream has 1,143 tasks according to the experiments guide.

Change this before your first real run

Here's the sentence most people will scroll past. The README says: "Executor network access is unrestricted, and Codex runs with its sandbox disabled."

Put that next to the no-answer-key design and you can see the problem. A loop that rewards "behaved better" with no ground truth will happily reward an agent that found a shortcut. And the experiments guide reports exactly that shortcut happening: in one validation run, a Codex rollout (one full attempt at a task) on JobBench downloaded its task's rubric without being told to.

The authors' guidance is specific. There's no flag for it, so do it at the network level, with an egress firewall (rules that limit which sites a machine can reach) or a DNS policy that refuses to look up the blocked hostnames, after setup finishes (setup needs these hosts):

• Block huggingface.co, hf.co and *.hf.co
• Block github.com and raw.githubusercontent.com
• Report the restrictions you used alongside any results

If you skip this, any score you get measures how good the agent is at finding the answers online. That might be impressive. It isn't what you wanted to know.

Steal the idea without the framework

You don't need Vertex AI to use the core lesson. The contrastive trick works by hand:

1. Pick one task your agent does badly.
2. Write two harness variants. Change one thing: add a "reopen and check your outputs before finishing" instruction, or a short skill for the step it keeps skipping.
3. Run both on the same task and read the transcripts side by side.
4. Keep the variant whose behavior got better, then write the lesson down where the next task will see it.

That last step is the improvement library, done in a markdown file. The JobBench example suggests where to start: most of the gain came from making the agent verify its own work.

Honest limitations

It isn't a supported product, and it isn't cheap to run for real. Every model-backed run bills Vertex AI for the agent's attempts and the improvement stages, and the guide's advice is to "start with a small selection and review model pricing before launching a full run." Raising --k multiplies the agent runs per task, so --k 5 costs several times what --k 1 does. The README lists Linux as a requirement, so Mac users will want a VM or a cloud machine.

The headline numbers are the authors' own, and the project page shows full results for only one of its four model and harness setups. Reproducibility has rough edges the docs admit: --seed "does not control model sampling," and the JobBench dataset isn't pinned to a commit.

There are no tagged releases yet, so you're running whatever is on main. Record the commit (git rev-parse HEAD) next to any result you plan to compare later. The code is Apache-2.0, but third-party code, benchmarks and datasets keep their own licenses, so check before you build on them.

And the safety defaults are the ones described above. Out of the box, the executor has open network access and Codex runs unsandboxed. That's fine for a demo. It's a bad default for anything touching your files or credentials.

What to take from it

Harness² makes a case that's easy to say and hard to act on: before you wait for a better model, look at the desk you gave the current one. Its loop proves you can improve that desk without an answer key, as long as you also lock the doors so the agent can't go find the key itself.

Try the demo. Lock down the network. Then run one of your own tasks twice and read what changed.

Sources: google-research/harness2 README and EXPERIMENTS.md (main branch, no releases, reviewed October 17, 2026); Harness²: Recursive Agent Harnessing for an Open World (arXiv, submitted October 9, 2026); project page.