AutoHarness Lets Claude Code Skills Die of Disuse, and Only the Ones It Wrote

A self-learning skill layer that decides survival by whether a skill actually got loaded, plus the scope limit you should read before installing it

Every skill you install into a coding agent is permanent by default. There is no expiry, no usage report, no moment where anything tells you a skill has not fired in six weeks. The description sits in the model's startup context on every single session, earning its keep or not, and the only mechanism for finding out which is you remembering to look.

AutoHarness, from Tigerless Labs, is the first repository I have seen whose stated purpose is having fewer skills rather than more. Its README opens by claiming the layer "stays clean on its own," and then finishes that sentence with five words most projects would have left in a footnote: "touching only the skills it wrote itself." That clause is the whole review. What follows is a genuinely good eviction policy applied to a set of skills that is probably not the set costing you money.

What it actually does

AutoHarness runs as a Claude Code plugin, entirely in Python with zero third-party dependencies, and installs in three lines typed into the Claude Code input box:

/plugin marketplace add tigerless-labs/autoharness
/plugin install autoharness@autoharness
/reload-plugins

From there it watches your sessions and writes skills into .claude/skills/. The trigger is deterministic and counts one thing: tool calls. Every main-session tool call advances a counter, and the turn that pushes it past AUTOHARNESS_REFLECT_EVERY_N, default 50, ends with a background reflection. A conversation that only talks never fires. A working stretch does. There is also /learn, which distills the session you are in right now when you have just worked something out and want the lesson kept.

The pipeline that follows is split across components with hard boundaries between them, and the boundaries are the interesting engineering. The reflector reads the episode, compares it against the existing skill index, and decides add, merge, patch, drop a support file, or delete. It proposes only. It has no write tools, and if you run the fork carrier, which resumes and forks the session that just ended, the inherited write tools are denied at the hook rather than trusted not to be used. The promoter is the only writer: it lints every intent in memory for safety, structure, ledger completeness and self-authorship, and on pass does an atomic rename into the live skill directory.

Two of the promoter's lint rules tell you the author has been burned before. A new skill's description has to carry its trigger early enough to survive truncation in the session-start index, because a cue that lands past the cut leaves the skill as half a sentence on the very surface meant to recall it. And a merge must name the live skill that absorbed the deleted one; an invented umbrella name fails the entire intent rather than losing the content. Both of those are the kind of rule you write after watching something go wrong.

The part that matters: how a skill dies

Most self-improving-agent projects validate against a held-out benchmark. AutoHarness does not, and its reasoning is stated plainly in the README: a skill survives by being adhered to in later turns, measured as loads over the requests it was available for, with no oracle on the active path and no tokens spent on a dedicated eval.

I think that is the correct call, and not only for the cost reason. A benchmark score tells you a skill is good at the benchmark. Adherence tells you the model reached for it while doing your actual work, which is the only property that matters for something whose entire job is being reachable.

The measure is opportunity-relative rather than wall-clock. Usage rate is loads divided by the requests that arrived since the skill was created, so a closed laptop does not age anything out. That is a small design choice with a large consequence: take two weeks off and come back to a library that is exactly as it was, rather than one that expired on a timer while you were away.

Three signals are deliberately kept apart, and the separation is sharper than it first looks:

A load means the model invoked the skill. It is the only thing the survival rate counts.
A view means a session read into the skill's directory. That is evidence the skill had recall value, but not that it was followed.
A patch means the promoter landed an improvement, so a load afterward reads as reuse-after-improvement rather than as first contact.

New skills sit in probation until enough requests have arrived to constitute a fair sample, 100 in the project layer and 300 in the global layer by default, higher for global because a global skill loads in every project. During probation a skill is recalled normally but cannot be archived. At maturity, graduation review archives a skill only if its entire probation passed with no load and no view. The README's justification is the best sentence in the document: no evidence of use is not the same as evidence of no use.

After graduation, capacity contention is the only death. Nothing is archived until a layer's mature pool exceeds its cap, 50 project and 20 global by default, and then the lowest usage rates go first. And archiving is a folder move to .claude/skills/.archive/<name>/, ledger and evidence intact. Moving it back revives it. Nothing is deleted.

There is also an altitude cap I would steal outright: AUTOHARNESS_SKILL_BODY_MAX_LINES, default 25. A SKILL.md body longer than that is rejected as a transcript rather than a rule, with backing detail pushed into the skill's references/. That is a policy decision about what a skill is, enforced in a linter, and it is the single most transferable idea in the repository whether or not you install anything.

The limit, stated as clearly as I can

AutoHarness manages the skills AutoHarness wrote. Every skill it generates carries a self-authored ledger marker, and anything without that marker, meaning everything you authored and everything you installed from GitHub, is invisible to both the promoter and the lifecycle pass.

That is the honest and correct engineering choice. A plugin that silently archived a skill you installed on purpose would be a support nightmare and arguably a hostile act. I would make the same call.

It also means the tool does not solve the problem the category actually has. If your context is being eaten, the likely culprits are the eight skill packages you installed from a trending repository in July and have not thought about since, not the handful autoharness distilled from your own sessions last week. AutoHarness gives you a bounded, self-cleaning garden next to an unbounded pile it will not look at.

The complementary move exists and is one command. Claude Code shipped /skill-doctor in version 2.1.261 on September 4, and it reports which loaded skills go unused and what they cost you in context, across everything. Run that against your installed skills; let autoharness handle the ones it generates. Neither tool covers the other's territory, and using one and calling the problem solved is the mistake available here.

Put this into practice

Install it on one repository, not globally. The project layer has a lower maturity gate and a higher capacity cap than the global layer, which makes it the cheaper place to find out whether the thing earns its keep. Global skills load in every project, and you should not be volunteering that surface to a plugin you have run for a day.

Shrink the knobs for the first session so you can actually watch it work. The README's own demo settings are the fast path:

{ "env": { "AUTOHARNESS_REFLECT_EVERY_N": "3",
           "AUTOHARNESS_MATURITY_PROJECT": "5",
           "AUTOHARNESS_CAPACITY_PROJECT": "2" } }

Work a few non-trivial turns, then ls -la .claude/skills/ and read a .ledger.jsonl. One JSON line per lifecycle event with an action, a reason, and a path to a redacted transcript slice that justified it. Being able to ask "why does this skill exist" and get a real answer is worth more than the automation.

Use AUTOHARNESS_INDEX_SUSPENDED=1 as a measurement, not a preference. Setting it to 1 stops the session-start index from being injected while everything else keeps running: the lifecycle pass, the counters, the last-run summary. That is a clean A/B on what the index is actually worth in context tokens versus recall, and the README frames it exactly that way rather than as an off switch.

Reset your defaults expectation. The README says the shipped defaults are deliberate placeholders pending empirical calibration, tracked under experiments/. Treat 50, 250, 100, 300, 50 and 20 as starting points somebody reasonable picked, not as tuned values, and expect to move them.

Run /skill-doctor in the same sitting. It covers the skills autoharness will not, and the two together are the actual audit. Delete on the evidence rather than on the feeling that you might need that one someday.

Honest limitations

No releases at all. AutoHarness has zero GitHub releases. Its version lives in .claude-plugin/plugin.json, currently 0.5.3, and the README's release badge reads that file dynamically. You have no tagged artifact to pin, no release notes to diff, and the README itself warns that claude plugin update will report "already at the latest version" against a stale local catalog unless you refresh the marketplace first. For a plugin with hooks in your session and write access to your skills directory, an unpinnable version is a real cost.

Linux and macOS only. The platform badge names both and omits Windows, and it needs python3 on your PATH or the hooks and MCP server will not fire at all.

The validation story is a claim, not a result. "Validated in use, not on a benchmark" is a design philosophy, and the repository publishes no measurement of whether adherence-based survival produces a better library than the alternatives it compares itself against in its own table. The ledger exists so that a benchmark could be built from real usage later; the README says so directly. That is intellectually honest and it also means the central claim is currently unfalsified rather than supported.

The one number in the README does not check out where it points. AutoHarness opens with "Same model, different harness, 42% to 78% on CORE-Bench," cited to the Holistic Agent Leaderboard paper. That figure does not appear anywhere in HAL's abstract, which I read. It may well be in the body of a paper with more than thirty authors, and I did not read all of it, so I am not calling it wrong. I am saying it is unverified at the citation given. What HAL's abstract does say is worth more anyway: 21,730 agent rollouts across nine models and nine benchmarks at roughly $40,000, with higher reasoning effort reducing accuracy in the majority of runs.

Uninstalling does not undo it. Removing the plugin stops it running. The skills it landed and its state directories stay on disk, at ~/.claude/autoharness/ and <repo>/.claude/autoharness/. That is arguably correct behavior, and it is also a cleanup step nobody performs.

A merge is the one thing that cannot be atomically undone. The curator snapshots both skill trees before consolidating, keeping five by default, which is the author acknowledging exactly this. Five deep is not infinite, and consolidation runs on a 250 tool-call cadence.

What to do with this

The reason to read AutoHarness is not the automation. It is that somebody sat down and answered a question the whole skills ecosystem has been avoiding: what does it take for a skill to deserve its place in the context window, and what should happen when it stops deserving it. The answer here, adherence in live use, opportunity-relative rather than wall-clock, probation before eligibility, archive rather than delete, and no evidence of use is not evidence of no use, is a better policy than most teams have written down for their own libraries.

You can adopt that policy without adopting the plugin. Cap your skill count. Require every skill body to fit in twenty-five lines and push the rest into references. Keep a note of why each one exists. Check quarterly whether the model has actually reached for it, and archive rather than delete when it has not.

Then go count how many skills are loading in your agent's startup context right now. If you cannot say the number from memory, that is the finding, and no plugin is going to hand it to you.

Sources: tigerless-labs/autoharness (MIT, 2,571 stars and version 0.5.3 at a cache-busted read on 7 September 2026); Holistic Agent Leaderboard, arXiv 2510.11977; the /skill-doctor entry in the Claude Code changelog for 2.1.261, published to npm on 4 September 2026.

Medium metadata

Title: AutoHarness Lets Claude Code Skills Die of Disuse, and Only the Ones It Wrote
Subtitle: A self-learning skill layer that decides survival by whether a skill actually got loaded, plus the scope limit you should read before installing it
Tags: Claude Code, AI Agents, Developer Tools, Open Source, Prompt Engineering
Suggested kicker: The eviction policy is the product, and it deliberately does not cover the skills you installed
Canonical: import from the fervorai.dev URL after publication