Independent AI intelligence Two editions daily · ET
Fervor AI
Menu

Analysis · September 11, 2026 · repo

NVlabs SoL-PiPi coding agentagent-harnessagent-infrastructureagent-securitylocal-ai

NVIDIA's SoL-Pi Ships Four Agent Harness Optimizations With All of Them Turned Off

A close read of a harness extension whose constraints are more instructive than its features

Most performance projects ship with the performance on. SoL-Pi ships with all four of its mechanisms disabled and a README that tells you to turn on exactly two of them, then read the security documentation before touching the other two.

That is an unusual way to sell an optimization, and it is the reason the repository is worth an hour of your time even if you have never run the harness it extends. NVIDIA published NVlabs/SoL-Pi as a standalone extension for Pi, the coding agent from earendil-works. It sat around 780 stars this morning, MIT licensed with a real copyright line ("Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES"), and it has no tagged releases at all. The subtitle on the README is "Scaling Auto-Research Loops for Efficient Agent Harnesses," and the framing question in the introduction is the one I keep coming back to: before you scale an agent loop, can the agent first make the harness itself more efficient?

The answer they landed on is four mechanisms. Three of them are the same idea wearing different hats.

The four mechanisms, and what they are actually attacking

Here is the table from the README, translated into plain terms.

Action Fusion replaces Pi's edit and write tools so that an edit can run its follow-up validation command in the same tool call. If your agent writes a file and then almost always runs pytest or tsc against it, that is two round trips through the model for a sequence with one decision in it.

ObservationPack turns repeated large text results into stable handles with exact paged recall, and registers an obs_recall tool plus a context projection handler. The problem it targets is replay: a 40,000-token grep result that stays in the transcript and gets re-sent on every subsequent turn.

Evidence-Preserving Reducer takes long diagnostic logs and turns them into compact receipts, but only when every retained quotation matches the archived source. The stated failure mode is safe: if the reduction fails, the original result continues unchanged.

Online Context Compact marks completed plan steps as candidates for Pi's native compaction, gated on an economic check and a context-window pressure check, then starts a new turn and tells the assistant to rebuild its plan.

Three of those four are the same fight. Action Fusion cuts a turn. ObservationPack cuts replay. Online Context Compact cuts retained history. All three are attacks on how many tokens you pay for a second, third and fourth time because the agent is still carrying them. The README's own framing is "reducing repeated model turns, context replay, oversized observations, and unnecessary long-log reading," which is one problem stated four ways.

The reducer is the odd one out, and I will come back to it, because it is where the whole design gets interesting.

The four rules are the actual product

Under the feature table, the README lists four constraints the mechanisms share. Quoted directly:

  • No Pi patches. SoL-Pi imports public Pi APIs and does not vendor the Pi source tree.
  • Explicit opt-in. A missing configuration leaves every mechanism disabled.
  • Preserve evidence. Original observations remain available locally, and reducer failures leave the original result unchanged.
  • Use Pi's runtime choices. Authentication, provider URLs, the main model, and shell behavior remain under Pi's control.

I would take those four lines over the four mechanisms. They are, as far as I can tell, a complete statement of what a third-party harness modification owes the person running it, and most of the agent tooling I read does not honor even two of them.

Look at what each rule prevents. No patches means your upgrade path is Pi's upgrade path, not a merge conflict. Explicit opt-in means installing the package changes nothing, so the blast radius of pi install is zero until you write a config file. Preserve evidence means the optimization is never allowed to be the reason you cannot reconstruct what happened. And leaving authentication, provider URLs and model selection with Pi means the extension never becomes a place where credentials or a model swap can hide.

The configuration doc enforces that last one explicitly: "do not put credentials in sol-pi.json." It also says SoL-Pi reads no dedicated environment variables at all, and lists what it deliberately does not configure through them: shell paths, command prefixes, storage paths, run IDs, provider URLs, reasoning levels, timeouts, per-mechanism flags. That is a project that thought about where a configuration surface turns into an attack surface.

The config file itself is small enough to reproduce whole. This is the conservative profile the README recommends, enabling only the two mechanisms that make no extra model calls and never interrupt a run:

{
  "version": 1,
  "actionFusion": true,
  "observationPack": true,
  "evidencePreservingReducer": false,
  "onlineContextCompact": false,
  "cacheWriteReadRatio": 12.5
}

And the failure behavior is strict rather than forgiving. Per the configuration doc, "unknown keys, unsupported versions, malformed JSON, non-boolean feature values, invalid ratios, and invalid reducer model fields stop extension loading with a direct error." A typo does not silently disable a mechanism. It refuses to load.

The mechanism that breaks its own rule

Now the reducer.

Evidence-Preserving Reducer is the one mechanism that leaves your machine. From the README: it "may send eligible diagnostic-log content to its configured reducer model using Pi-managed authentication. Review SECURITY.md before enabling it. Do not enable remote reduction for logs that must remain local."

SECURITY.md is more specific, and this is the sentence that should decide whether you turn it on: "The reducer skips text matching its likely-secret detector, but that detector is a precaution rather than a complete secret scanner."

That is an honest disclosure and it is also a hard stop for a lot of workloads. Diagnostic logs are where secrets go to die. Stack traces carry connection strings. Failed HTTP calls carry authorization headers. A build log carries whatever was in the environment. If you enable remote reduction on a production debugging session, you are sending that material to a second model, and the project is telling you plainly that the filter in front of it is best-effort.

There is a second detail in SECURITY.md worth knowing, and it reads like someone thought about a specific attack. The reducer may temporarily read an overlong bash result from outside its session archive, and it "accepts only a regular, non-symlink pi-bash-*.log file directly inside the operating system's temporary directory," copying eligible content into the session archive before any nested model call. Non-symlink, regular file, exact directory, exact name pattern. That is what it looks like when someone has considered that /tmp is a shared, attacker-writable space, and it is a nice contrast with the symlink permission bugs Anthropic fixed in Claude Code 2.1.268 yesterday.

SECURITY.md opens with the line that frames all of it: "SoL-Pi is not a sandbox or permission boundary." It runs with the filesystem, process, network and credential permissions of the Pi process that loads it. Action Fusion can modify files and run shell commands. A project-local .pi/sol-pi.json can enable file mutation, shell execution, local archival and remote log reduction, which is why SoL-Pi ignores the project file unless Pi has marked the project trusted via ctx.isProjectTrusted().

Read that chain once more. A config file in a repository can turn on a mechanism that ships your logs to a model. The guard is that Pi has to consider the repository trusted. If you clone unfamiliar repositories and click through trust prompts, that guard is you.

Put this into practice

If you already run Pi, this is a thirty-minute experiment with a clean rollback.

Start with the pin, not the extension. The README requires Node 22.19 or newer and @earendil-works/pi-coding-agent at exactly 0.84.2:

npm install --global @earendil-works/pi-coding-agent@0.84.2
pi install git:github.com/NVlabs/SoL-Pi

For a single project rather than your whole machine, pi install git:github.com/NVlabs/SoL-Pi --local --approve.

That exact version pin is the most underrated line in the repository. NVIDIA's dev dependencies are pinned to Pi 0.84.2, Pi's runtime packages stay peer dependencies so Pi owns its own upgrades, and there is a node scripts/check-pi-compat.mjs in the standard check sequence. A harness extension is a compatibility liability by construction, and the project treats it like one.

Write the conservative config and nothing else at first. Put the JSON above at ~/.pi/agent/sol-pi.json. That is the global location; the project-level path is .pi/sol-pi.json and, importantly, the project file replaces the global file rather than merging with it. Two files, one wins. If you have a personal config you want everywhere, keep it global and resist the project file.

Measure before you decide it worked. Action Fusion and ObservationPack change how many turns and how many replayed tokens a session costs, so the numbers to compare across a fixed task are request count, total input tokens, and wall-clock time. Run the same task three times with the mechanisms off, then three times on. The README publishes no benchmark numbers of its own, which is refreshing and also means you have nothing to anchor on but your own workload.

Treat the other two as separate decisions with separate reviews. Turn on onlineContextCompact only if you run long sessions where context pressure is the actual bottleneck, and read the SECURITY.md section on what it writes to Pi's session log first, because those entries hold model-authored plans, paths, command names and design notes. Turn on evidencePreservingReducer only after you have answered the question "would I be comfortable if every line of these logs went to a second model," and if you do, set evidencePreservingReducerProvider and evidencePreservingReducerModel deliberately rather than accepting the built-in route.

Clean up after yourself. ObservationPack and the reducer write session archives under <session-directory>/sol-pi/<session-id>/, and the README says plainly that those copies "remain local and are not automatically deleted when the Pi session ends." That is the right default for evidence preservation and the wrong default for a laptop you use for six months. Put it in your cleanup script now, not after you find it.

Where this falls down

The honest limits, in the order they would bite you.

I have not run SoL-Pi. Everything above is a read of the README, the configuration doc and SECURITY.md, cross-checked against the raw LICENSE file. I did not read docs/compatibility.md or agents-install.md, so the supported-API details and the managed all-enabled installation protocol are unverified here. I also did not read the linked blog at nvlabs.github.io, which is where the auto-research methodology and any numbers behind the four mechanisms live. If the mechanisms have measured effect sizes, they are there and not in the README.

The repository has no tagged releases and no version except what the package manifest carries, which means pi install git:github.com/NVlabs/SoL-Pi installs whatever is on the default branch at the moment you run it. For a project this careful about pinning its upstream, not offering a pin of its own is an odd gap. If you deploy this anywhere that matters, install from a specific commit.

The Pi requirement is narrow. One tested Pi version, and if you are on a different one you are outside what the project validates. Nothing here transfers to Claude Code, Codex, or any other harness without a rewrite against that harness's extension API, and most harnesses do not have one.

And the biggest limitation is structural rather than technical. Everything good about this project comes from Pi being an extensible harness you run yourself, with public extension APIs, a trust model you can inspect, and a version you can pin. None of the four rules SoL-Pi honors is even expressible against a managed harness that runs in someone else's account. The thing that makes this a good piece of engineering is the thing the rest of the industry spent this week moving away from.

What to take from it

Borrow the contract even if you never install the extension. The next time you evaluate anything that modifies your agent's loop, a skill bundle, a plugin, a hook, a wrapper CLI, ask its four questions. Does it patch the harness or use public interfaces? Does installing it change behavior before you configure it? Can you still reconstruct what happened if its optimization misfires? And does it leave authentication, model selection and shell behavior where they were?

Four questions, thirty seconds, and they will disqualify most of what is trending this week.

Sources: NVlabs/SoL-Pi README · SoL-Pi SECURITY.md · SoL-Pi configuration docs · Pi · Claude Code CHANGELOG

Medium metadata

  • Title: NVIDIA's SoL-Pi Ships Four Agent Harness Optimizations With All of Them Turned Off
  • Subtitle: A close read of a harness extension whose constraints are more instructive than its features
  • Tags: AI Agents, Open Source, NVIDIA, Developer Tools, Software Engineering
  • Suggested kicker image: a control panel where every toggle is in the off position
  • Canonical: import from the fervorai.dev URL