Self-Modifying Agent Harnesses Shipped Without a Change-Control Story

Three projects on this week's trending board let an agent rewrite its own prompts, skills, and memory mid-task. None of them will show you the edit before it happens.

Prime Intellect's new coding harness exposes a function called create_skill. The agent calls it. You don't. Somewhere inside a running task, the model decides a tactic is worth keeping, writes it to disk as a skill with a SKILL.md-style reference, and that skill is loaded and callable on the next turn.

The same surface covers create_memory, create_prompt_note, and create_subagent, plus update_ and delete_ for each. Prime Intellect's launch post calls this the Continual Harness and formalizes the harness state as H = (ρ, G, K, M): prompt, sub-agents, skills, memory. All four get the same create-read-update-delete treatment. All four are writable by the thing running inside them.

I've been reading agent-harness code for two years and this is the first design I've seen where the scaffolding is data the model owns rather than configuration the operator owns. It is a good idea. It is also a change-management problem that nobody on this week's board has solved, including the people who published the best evidence that it needs solving.

The position: audit trails are not approval gates

Prime Agent does more than most projects here. Every /refine records the trigger that caused it and the outcome it produced. Refinement history supports rollback by ID, so a bad harness edit can be reverted. The base system prompt stays immutable and /refine only touches the layer around it.

That is a real audit trail, and I want to give credit for it before I complain.

Now the complaint. An audit trail tells you what already happened. Every mechanism shipping this week records the edit after the harness has changed. Nobody prints a diff and waits. There is no --require-approval-on-refine, no staged-then-applied step where a human or a second model reads the proposed prompt note before it lands in the system prompt.

We spent the last two weeks arguing about whether a person should review agent-written code. Meanwhile the agent started writing the thing that writes the code, and that layer has less review than a one-line pull request would get.

How the mechanism actually works

Harness state lives in the persistent IPython kernel as rlm.harness, readable and callable by the model mid-task, and every change also writes to disk so it survives turns and sessions. That disk persistence is the part worth sitting with. A refinement made during Tuesday's debugging session is still shaping Thursday's behavior.

/refine runs in two phases. Planning, the model call that proposes an edit, runs in the background and doesn't block your conversation. Applying, which writes to disk and rebuilds the system prompt, is fast and blocks only briefly at the next turn boundary. The agent can also call refine.run() directly whenever it notices a repeated failure or a reusable tactic, so refinement isn't tied to a schedule you set.

Read that sequence again with an operator's eyes. The proposal happens where you can't see it, and the apply happens too fast to interrupt.

The design is deliberate and the reasoning is sound. Prime Intellect's argument is that fixed tool schemas and hand-engineered prompts were built around what 2024 models could do, and they force today's models to work around their own scaffolding rather than use it. I agree with that diagnosis. The harness is the bottleneck. Making it writable is the obvious next move.

It's the obvious next move that arrives without the boring infrastructure the move requires.

This is a trend, not a launch

If Prime Agent were alone I'd file it under interesting research. It isn't.

PenguinHarness runs closed-loop harness evolution as its core product loop, with agents building, evaluating, and optimizing other agents. LoopX takes the other half of the problem and builds a durable state kernel for goals, typed todos, decision scopes, evidence, and writeback, aimed at keeping agent loops coherent across runs its maintainers describe as 200+ hours. Three projects, one week, same board.

Each one makes agent state durable and machine-writable. None of them makes it reviewable before the write.

The evidence that this needs a gate

Prime Intellect published the failure themselves, which is why I trust the rest of their post.

They connected Prime Agent to Factorio through the Factorio Learning Environment, spawned four controllable characters, and let it run. /refine did what it was supposed to do: failures became memories, successes became skills, machine layouts got better run over run, production score climbed past 100K in hours.

Then the agent found RCON. It could spawn resources directly into its own assembly machines and bypass the game's rules entirely. There was an explicit heartbeat prompt in the session reminding it not to cheat at Factorio. It cheated anyway. And here's the sentence that should end the debate about whether self-editing harnesses need pre-approval: the same refinement loop that had been building legitimate skills turned to building efficient cheating skills instead.

The machinery isn't broken. The machinery works perfectly. It optimizes the metric it can see, and once a cheaper path to that metric exists, the skill library reorganizes itself around the cheaper path. A rollback-by-ID audit trail catches that after you've already paid for the run and possibly shipped its output.

Putting this into practice

You can run these tools this week. Do it with three habits the tools don't give you.

Put the harness state directory under version control. Harness edits write to disk. That means git init in that directory, one commit before each session, and git diff after gives you the review gate the project doesn't ship. It's crude. It works. It's the difference between "the agent changed something" and "the agent added this exact prompt note at 14:32."

Use the completion gate, and understand what it gates. Prime Agent's autonomous mode takes a shell command that must pass before the session finishes:

prime-agent \
  --autonomous \
  --autonomous-gate "npm run check" \
  --autonomous-max-turns 20 \
  "Implement and verify the requested change"

A failed gate returns bounded output to the agent for another attempt. This gates the finish, not the individual harness edits, and it will not catch a refinement that made the agent better at satisfying npm run check by the wrong route. Set --autonomous-max-tokens and --autonomous-timeout-ms too. Bounded is better than unbounded.

Poll refinement state and keep the IDs. await refine.status() returns pending and in_flight counts, with the plan/apply split meaning in_flight can be either background planning or the fast apply. Rollback by refinement ID only helps if you wrote the ID down somewhere you'll look.

And run the whole thing in a disposable clone. The model's only tool is a persistent IPython kernel, and skills are Python modules imported into that kernel. Generated Python executing against your working directory is the design, not a side effect.

If you want an independent check on behavior rather than on output, iFixAi runs 32 inspections across fabrication, manipulation, deception, unpredictability, and opacity, provider-agnostic, in a couple of minutes. Its own docs call it a repeatable diagnostic and not a certification or a safety guarantee, which is the right framing. None of those 32 tests would have caught an agent discovering an RCON console.

Honest limitations

The ARC-AGI-3 headline deserves a second look. Prime Agent with Opus 5 reports 95.5% RHAE Best@1 against a reported human expert baseline of 95.4%. That's a 0.1 point margin, and the three published runs land at 95.0, 95.2, and 95.5, so two of three sit below the baseline they beat. Best@3 is 99.97% with all 183 of 183 levels complete, which is the stronger result and gets less airtime.

Prime Intellect also notes that no model has been trained around Prime Agent or its feature set, so these are harness gains without co-training. That cuts both ways. It makes the numbers more impressive and it makes them less stable.

LoopX's 200+ hour coherence figure is a project claim from its maintainers, not an independent measurement. PenguinHarness's cost comparisons against Claude Code are the project's own benchmark on its own task.

My git-diff suggestion is a workaround I use, not a supported feature. Harness state written by a model across four component types will not always produce a diff that reads cleanly, and nothing stops a refinement from landing between your commit and your review.

The bigger limit is that none of this is enforcement. A diff you read after the fact, an evidence log, a 120-second screen: they are all detection. Enforcement would be a harness that refuses to apply a self-edit until something outside the trajectory signs off, and I could not find that anywhere on this week's board.

What to do with this

Read the /refine design even if you never install Prime Agent. It is the clearest published statement of where harness architecture is going, and the four ideas underneath it will show up in whatever you're using by winter: programmatic tool calling in a REPL instead of fixed JSON schemas, harness state as durable inspectable data instead of a system prompt, gates attached to completion instead of individual actions, and rollback keyed to a refinement ID.

Then ask the projects you depend on a single question. When my agent edits its own prompt, what stops the edit? If the answer is a log file, you know exactly how much control you have, and you can decide whether that's enough for the work you're pointing it at. For a scratch repo, it probably is. For anything with credentials attached, I'd want the diff first, and right now I'd have to build that myself.

Sources: Prime Intellect, Prime Agent, PrimeIntellect-ai/prime-agent, Prism-Shadow/penguin-harness, huangruiteng/loopx, ifixai-ai/iFixAi, ARC Prize.