Independent AI intelligence Two editions daily · ET
Fervor AI
Menu

Analysis · September 21, 2026 · repo

Hereticlocal-aiagent-securityfine-tuning

Heretic Turns Alignment Removal Into a Pareto Front, and That Is the Uncomfortable Part

A pip install, one command, and about three hours on a consumer GPU. The interesting number is not how many refusals it removes. It is what the removal costs.

Read the tutorial for Heretic and the line that stops you is not about capability. It is this: "You don't need to be a software engineer, you don't need a machine learning PhD, you don't need to understand the intricacies of how language models work internally, and you don't need expensive hardware."

Then it tells you what you do need, and the list is short. A GPU, Nvidia or AMD. Roughly 2.5 GB of VRAM per billion parameters, so about 10 GB for a 4B model. Python 3.10 or later. The worked example in the docs runs on an RTX 3060 with 11.63 GB, and it takes just under three hours.

Heretic reached the Hacker News front page at 04:35 UTC on September 21, 2026, standing at 189 points by early evening UTC. The project is not new. Its latest tagged release is v1.4.0 from June 14, 2026, the repo page shows 27.1k stars, and it ships under AGPL-3.0-or-later on a master default branch. What is new is that a lot of people looked at it at once, and what they found is more interesting than "tool removes safety training."

What they found is a measurement loop.

Abliteration, briefly, and then the part nobody talks about

The technique is directional ablation, usually called abliteration in this corner of the ecosystem. A refusal in a chat model is not a rule stored somewhere. It is a direction in activation space, and if you compute that direction and project it out of certain weight matrices, the model stops taking it.

Heretic's startup log shows exactly which matrices. On a 32-layer transformer it identifies "Abliterable components: attn.o_proj: 32 modules total, mlp.down_proj: 32 modules total," and initializes LoRA adapters targeting down_proj, o_proj and out_proj. It loads 400 harmless prompts from mlabonne/harmless_alpaca and 400 harmful ones from mlabonne/harmful_behaviors to compute per-layer refusal directions, plus 100 of each for evaluation. Before touching anything, it counts: "Initial refusals: 93/100."

None of that is novel. People have been hand-tuning abliterations for two years. The novel part is what happens next.

Heretic does not apply one ablation. It runs a search. Two hundred trials by default, driven by Optuna, over eight parameters per component group: max_weight, max_weight_position, min_weight, min_weight_distance for attn.o_proj, the same four for mlp.down_proj, and a direction_index that can be per-layer or a single float. After each trial it measures two things. How many of the 100 evaluation refusals survived, and the KL divergence between the modified model's first-token probability distributions and the original's.

Refusals down is the objective. KL divergence up is the damage. The search is looking for the frontier between them.

The number the docs are honest about

At the end of the run, Heretic does not hand you a model. It hands you a menu, and it is the most instructive artifact in the project. From the published example:

? Which trial do you want to use? (Use arrow keys)
 » [Trial  91] Refusals: 11/100, KL divergence: 0.0508
   [Trial  81] Refusals: 21/100, KL divergence: 0.0366
   [Trial 116] Refusals: 30/100, KL divergence: 0.0296
   [Trial  93] Refusals: 45/100, KL divergence: 0.0120
   ...
   [Trial  49] Refusals: 93/100, KL divergence: 0.0006

That is a Pareto front, and it reads top to bottom as a price list. Trial 49 barely touches the model, KL divergence of 0.0006, and it barely helps: 93 refusals, the same as the untouched baseline. Trial 91 does the most, 11 refusals, and pays for it with roughly 85 times the divergence.

Now read the guidance the docs attach to that menu: "Any trial with a refusal count below 10 can be assumed to strongly suppress refusals, and among those, the trial with the lowest KL divergence should be chosen."

The best trial in the project's own worked example is 11. It does not clear the threshold the same page recommends. Two hundred trials on a 4B model found no point on the frontier that the documentation would call a strong suppression.

I want to be careful about what that does and does not mean. It is one model, one run, and the docs present it as a walkthrough rather than a benchmark. Heretic will happily run more trials from the same menu. But the artifact is still remarkable for how little it oversells. A tool whose homepage says it makes models "always follow your instructions" ships a tutorial whose headline result is 11 refusals surviving out of 100 and a note warning that "KL divergence values above 0.5 usually indicate significant damage to the original model's capabilities."

That honesty is the reason this project is worth your attention regardless of whether you would ever run it.

My position

Heretic's contribution to the field is not removing alignment. That was already a weekend project for anyone motivated. Its contribution is making the tradeoff legible and cheap to measure, and that cuts in two directions at once.

For anyone thinking about open-weight models in a threat model, the takeaway is blunt: alignment on distributed weights is not a control you can rely on, and it has not been for a while. What changed is the cost. If your risk analysis contained a sentence like "removing safety training requires expertise and compute," delete it. It requires pip install -U heretic-llm, a model ID, a consumer GPU and an afternoon, and the tool asks you exactly one question at the end.

For anyone doing model surgery of any kind, the takeaway is a technique, not a warning. Heretic's structure is the right structure for every intervention on weights, and almost nobody uses it. You define the thing you want, you define the damage you are willing to accept, you search the space instead of guessing a coefficient, and you present a frontier instead of a number. Quantization work has been converging on the same shape. So has pruning. Heretic is a cleanly implemented instance of it, and it is AGPL, so you can read every line of the loop.

The uncomfortable synthesis is that those two readings are the same fact. The rigor that makes this a good piece of engineering is exactly what makes it a capable piece of tooling.

Put this into practice

You probably do not want to run Heretic. Here is what to do with it anyway.

Read the trial-selection menu before anything else. It is the whole design in twelve lines, and it takes two minutes. If you are building any system that modifies model weights, that menu is the interface you should be copying: two objectives, a Pareto front, the user picking the point, and a documented rule of thumb for which point to pick. Most tools in this space hand you one artifact and a claim.

Steal the damage metric. KL divergence on first-token probability distributions over a held-out prompt set is a cheap, sensitive measure of "did I break this model," and it works for quantization, pruning, LoRA merges and any other surgery. It is much better than perplexity at catching the kind of damage that does not show up in aggregate. Compute it on prompts that look like your traffic, not on a generic corpus.

If you do run it, run it on a model you already serve, and do not pick the top line. Start with a small model that fits comfortably: 2.5 GB of VRAM per billion parameters is the stated rule, and the docs note that 4-bit quantization via bitsandbytes cuts that by about 70 percent. Install PyTorch manually first, because Heretic deliberately does not manage that dependency and the right command depends on whether you are on CUDA or ROCm. Then let it run the full 200 trials, and when the menu appears, pick the lowest-KL trial among those under your refusal target rather than the aggressive one. The docs say this plainly and people will ignore it.

Verify the download. This is the underrated part of the project. Every official release is signed through Sigstore with the signer's identity verified at signing time via GitHub authentication, attested by Fulcio and logged in Rekor. Verification is one command with Cosign and no key import:

cosign verify-blob heretic-1.3.0.zip \
  --bundle heretic-1.3.0.zip.sigstore.json \
  --certificate-identity=pew@worldwidemann.com \
  --certificate-oidc-issuer=https://github.com/login/oauth

The maintainer also GPG-signs every commit, and the project locks dependency versions with uv while delaying updates by seven days specifically to blunt supply-chain attacks. A project that strips safety training from models is running a tighter software supply chain than most enterprise Python. Take the pattern even if you skip the tool.

And if you consume open weights from Hugging Face, look at the action menu. After choosing a trial, Heretic offers to save locally, chat, benchmark, or "Upload the model to Hugging Face," prompting for a write-scoped token if you are not logged in. That is a three-hour path from a published model to a modified model on a public hub. Check provenance on weights you pull, and treat an unfamiliar fine-tune of a familiar base as an unknown artifact rather than a variant.

What I could not verify, and where this gets thin

I did not run Heretic. Everything above comes from the project's own documentation and repository as fetched on September 21, 2026: the tutorial, the security page, the site, the LICENSE file and the release feed. The console output I quoted is the tutorial's own worked example, which is labeled v1.3.0 while the latest release is v1.4.0, so the specific trial numbers may not reproduce on the current version.

The 11-refusal result is one example run on one model, and I am treating it as a documented example rather than a ceiling. Heretic offers "Run additional trials" from the same menu, and I do not know how much a longer search improves the frontier. Anyone claiming it is a hard limit is overreading it, and that includes me if I said it that way.

I have also not evaluated whether refusal count on 100 prompts from mlabonne/harmful_behaviors means what it appears to mean. Counting refusals is a proxy, the prompt set is one public dataset, and a model that refuses less on that set is not necessarily a model that behaves differently on anything you care about. The same caution applies to KL divergence on first-token distributions: it is a sensitive early-warning signal, not a capability benchmark, and the docs are careful to call it "a rough measure."

On licensing, one detail worth stating precisely because it trips people up. The LICENSE file in the repository is the plain GNU AGPL v3 text, whose copyright line reads "Copyright (C) 2007 Free Software Foundation, Inc." The project's own attribution, "Copyright © 2025-2026 Philipp Emanuel Weidmann + contributors," appears in the README and in the site footer rather than in the license file. That is common and almost certainly not a problem, but if your legal process reads LICENSE files mechanically, it will come back with the wrong copyright holder.

Finally, I have taken no position here on whether the models Heretic produces are more dangerous in practice. I do not have that measurement, nobody I can cite has published it, and the honest statement is narrower than the one people will want: the cost of the operation has dropped a great deal, and the operation is now measurable, which is a different claim from knowing what the resulting models do.

Where this leaves you

The part of Heretic worth carrying away fits in one sentence. It replaced an intervention people used to perform by feel with a two-objective search that publishes its own price.

Go look at the trial menu. Decide whether the thing you last changed about a model could show you a list like that, and what you would have to measure to build one. If the answer is that you picked a coefficient because it seemed to work, you already know what to do next, and you do not need to abliterate anything to do it.

Sources: heretic-project.org tutorial; heretic-project.org security; heretic-project.org; p-e-w/heretic on GitHub. Star count, license text, default branch and release date verified via cache-busted shields.io, raw LICENSE and releases.atom on 2026-09-21.

Medium metadata

  • Title: Heretic Turns Alignment Removal Into a Pareto Front, and That Is the Uncomfortable Part
  • Subtitle: A pip install, one command, three hours on a consumer GPU. The interesting number is not how many refusals it removes. It is what the removal costs.
  • Tags: Open Source AI, AI Safety, Machine Learning, Local LLM, AI Engineering
  • Canonical: fervorai.dev