Independent AI intelligence Two editions daily · ET
Fervor AI

Analysis · November 20, 2026 · concept

Harness adaptersHugging Faceagent-harnesslocal-aifine-tuning

Harness Adapters Turn a Local Model Into a Specialist for Your Coding Agent

A 319 MB add-on, trained inside the agent you already use, cut tool calls by more than half in Hugging Face's tests. Here is what that means and how to try it.

Put the same open model inside two different coding agents and you get two different workers. In one it fixes the bug in a few clean steps. In the other it reads the same file four times, runs a test it already ran, and burns a few thousand tokens before it commits to an answer.

Most of us blame the model when that happens. We go shopping for a bigger one.

Hugging Face published a result on November 20 that points somewhere else. The model was fine. It just had never been trained to work inside that particular agent. When the team trained a small add-on for each agent, success rates went up and tool calls fell by more than half, with the base model untouched. Hugging Face

The result is a different question to ask about your local setup. Instead of "which model is best?", ask "which model was trained for the harness I actually run?"

What a harness is, and why it changes the model's behavior

A harness is the program wrapped around the model that turns it into an agent. It writes the system prompt, defines the tools (read a file, run a command, edit a line), decides what the model sees after each step and when to stop. Claude Code, Codex, OpenCode and Pi are all harnesses.

Each one does this differently. The Hugging Face team's org page puts it plainly: "The same model behaves differently in Pi, OpenCode, Hermes or Claude Code: each harness has its own tools, prompts and control flow." harness-hub

Think of a skilled carpenter dropped into an unfamiliar workshop. The skill is all there, but the tools hang in different places and the foreman gives instructions in a different order. The first week is slow. Nothing is wrong with the carpenter.

General-purpose models are trained to do well everywhere, which means they are tuned for no workshop in particular. The post says it directly: models "target general performance," and pairing one with a specific harness "leaves performance and efficiency on the table." Hugging Face

What Hugging Face actually built

The team took Qwen3.8-27B, an open model that runs on a single large GPU, and trained one adapter per harness for three open-source coding agents: Pi, OpenCode and Hermes Agent.

An adapter here is a LoRA: a small set of extra weights that sits on top of a frozen model and nudges its behavior, a bit like a lens clipped onto a camera. The base model never changes. You can load the adapter or leave it off.

The training method is reinforcement learning, which in plain terms means the model tries a task, gets a reward when the task's tests pass, and gradually leans toward whatever earned the reward. Two details make this version unusual:

  • The real harness ran during training. Many projects re-create an agent loop inside the training code. Here each harness ran as it ships (Pi 1.5.2, OpenCode 2.2.7, Hermes Agent v0.22.6), and a proxy recorded the exact tokens it produced, so the model learned from the same prompts and tool formats it will see in use.
  • Efficiency was part of the reward. The reward was correctness plus a small bonus for using fewer tools, so a model that solved the task in five calls beat one that needed fifteen.

They also ran every tool call through safeagents, Hugging Face's rule-based blocker, to stop the model from misbehaving during training. Across the three runs it checked about 1.3 million tool calls and blocked 647, according to the post. Hugging Face, safeagents

The numbers that matter

The full results run to dozens of figures. Three change what you would do.

Success went up a lot under the harness each adapter was trained for. On the 51 hardest tasks of SWE-bench Pro (a benchmark of real repository bugs), the post's results table shows the Pi adapter going from 40.0% to 64.0%. The model card reports a slightly different 39.2% to 64.7%; either way, it is a jump of more than 20 points. Hugging Face, Pi adapter card

Tool calls fell by more than half. Across the three adapters, tool calls dropped 56% to 72% on SWE-bench Pro, measured on tasks both the base model and the adapter solved. On Terminal-Bench 2.1 (a benchmark of command-line tasks), the post's headline example has the Pi setup going from about 2.9K tokens per task to about 1.5K.

A small adapter got close to expensive thinking on one benchmark, not the other. The Pi card compares the adapter with thinking off against the base model at its highest thinking setting. On Terminal-Bench 2.1 the adapter edged ahead, 56.6% to 54.2%. On SWE-bench Pro the thinking base model still won clearly, 74.5% to 64.7%. Pi adapter card

That third point is the honest shape of the win. The adapter does not make a 27B model smarter. It makes it stop wasting moves inside one harness.

Why this means more on a local machine. On a hosted API, a wasted tool call costs a fraction of a cent. On your own GPU, it costs wall-clock time, and every extra round trip is another chunk of context the model has to read again. Cutting the calls in half removes a large share of the waiting on long tasks. For anyone running a coding agent on their own hardware, that is the difference between usable and annoying.

Specialist or super-adapter?

The team also merged the three adapters into one "super-adapter" that runs under all three harnesses. With thinking off it beat the base model on all six harness and benchmark pairs, lifting the average from 39.5% to 49.0%.

The post's own advice is still to pick the specialist when you can: "If you only use one harness and keep thinking off, take its specialist." Spreading one adapter across three harnesses cost Pi the most, 64.0% against 46.0% on SWE-bench Pro. Hugging Face

Put this into practice

The cheapest experiment is Pi with llama.cpp, the open-source engine for running models on ordinary hardware. The adapter file is about 319 MB and runs on top of a GGUF copy of the base model (GGUF is llama.cpp's single-file model format). The base file is the big download, tens of gigabytes at the suggested quality, so the 319 MB is the add-on, not the total. Both the adapter and its GGUF are released under Apache-2.0. Pi GGUF card

1. Download the base model and the adapter. The card suggests a Q8_0 base (8-bit, close to full quality), because heavier compression moves the results further from what was measured. You need enough GPU or unified memory to hold that file, so this is a job for a large graphics card or a high-memory Mac.

hf download ggml-org/Qwen3.8-27B-GGUF Qwen3.8-27B-Q8_0.gguf --local-dir .
hf download harness-hub/Qwen3.8-27B-pi-adapted-GGUF Qwen3.8-27B-pi-lora-F16.gguf --local-dir .

2. Serve the model with the adapter attached and thinking off, since the adapter was trained without thinking.

llama serve -m Qwen3.8-27B-Q8_0.gguf --lora Qwen3.8-27B-pi-lora-F16.gguf \
    --chat-template-kwargs '{"enable_thinking": false}' -c 65536 --port 8080 --alias qwen3.8-pi

3. Install Pi and point it at the server. Pi is published by Earendil Works and needs a recent Node.js (its repo lists 22.19 or newer). The adapter was trained against Pi 1.5.2, so if you want to match the tests closely, install that version rather than the latest. Pi

npm install -g --ignore-scripts @earendil-works/pi-coding-agent
pi install git:github.com/huggingface/pi-llama
pi --model llama-cpp/qwen3.8-pi

If you serve through vLLM instead (a popular open-source server for running models on GPUs), the card's setup wires Pi to the server with a provider entry in ~/.pi/agent/models.json, and it warns that the entry must use "api": "openai-completions". Without it, Pi falls back to a different API format the adapter was never trained or evaluated with. Pi adapter card

4. Measure it on your own work, not the benchmark. Pick ten small tasks from your own backlog: a failing test, a renamed function, a config change. Run each twice, once with --lora and once without. Count three things: did it finish, how many tool calls did it make, and how long did you wait. Ten tasks will not prove a percentage, but they will tell you within an afternoon whether the adapter helps on your kind of code. Keep in mind the published scores came from the full-precision adapter served through vLLM, so a llama.cpp run is your own test, not a reproduction.

5. Write down what you saw. If the adapter helps, keep it as your default Pi model. If it does not, you lost an afternoon and a 319 MB download.

On a GPU server, the vLLM route on the card loads the adapter with --enable-lora --lora-modules qwen3.8-pi=harness-hub/Qwen3.8-27B-pi-adapted and lets you serve the base model and the adapter side by side, which makes the with-and-without comparison easy.

Honest limitations

  • One base model, three harnesses. These adapters only work on Qwen3.8-27B. The card says the adapter "does not work with other base models." If you run Claude Code or Codex on a hosted model, there is nothing here to install. The org page lists adapters for a few other bases, including some for Claude Code on a smaller model, but they are separate releases with their own results.
  • These are the authors' own measurements. One attempt per task, temperature 0.8. The super-adapter's merge weight was tuned on Terminal-Bench 2.1, so the post calls its Terminal-Bench numbers "slightly optimistic." The blog and the model card also disagree by a point or so on the same results.
  • Scores do not compare across harnesses. Pi drops earlier reasoning turns while OpenCode and Hermes keep them, so a 56% under Pi and a 58% under Hermes are not a ranking of the agents.
  • The GGUF path was not re-measured. The card says the GGUF has not been re-evaluated, and a heavily compressed base, such as Q4_K_M, "drifts further."
  • Outside coding, the gains mostly vanished. On Zapier's AutomationBench, a different kind of agent task, the authors treat most differences as noise, and the Hermes adapter scored below the base model.
  • Training your own is not a weekend project yet. Each adapter trained on two NVIDIA B200 GPUs, one serving the model and one training it. The post gives no dollar cost or training time.

The bigger shift

This is part of a pattern worth tracking. The same week, Microsoft let browser agents in Foundry send a whole click sequence as one script, and GitHub let team maintainers set default agent plugins for their teams. In each case the improvement came from the layer around the model rather than the model itself. Microsoft Foundry, GitHub

For local AI, harness adapters change what "upgrade" means. A bigger model costs more memory and more time per token. A 319 MB adapter trained for the harness you already use costs a download and an afternoon of testing.

The model you have may already be good enough. It may just be working in an unfamiliar workshop. If you run Pi, OpenCode or Hermes on Qwen3.8-27B, spend the afternoon and find out for yourself.

Sources: Hugging Face, "Introducing harness adapters" (Nov 20, 2026); Pi adapter model card; Pi adapter GGUF card; harness-hub organization; safeagents; Pi coding agent. Model cards read November 20, 2026.