Kev Speaks a Paid API's Protocol From localhost, and Its Own Table Says Don't Run It on a Mac
An open decision-model family that the vendor's own Python SDK will talk to, and the Apple Silicon regression sitting in its serving table
The thing in Kev's README that should make a vendor uncomfortable is four lines of Python. You import TypeSafe's SDK, pass api_key="local" and base_url="http://127.0.0.1:8009", and call client.system_one(...). Same request shape, same response fields, same client library. The paid endpoint has been replaced by a process on your laptop and nothing above it noticed.
Kev is a family of small decision models from Jared Palmer, tagged on September 20, 2026, with a second training pass landing the following morning. Three sizes, 0.8B, 4B, and 9B, all LoRA adapters plus a pointer head on frozen Qwen3.5 bases. Apache-2.0, and the bases are Apache-2.0 too. It reached the Hacker News front page on September 21 and sits around 1.6k stars.
The interesting thing is not the model. It is the protocol.
What Kev actually is, mechanically
A decision model answers typed questions about a piece of text without writing anything. You send a state, which is whatever you want evaluated, plus a set of questions. Each question is one of three types. noul returns a probability of yes. choice returns the most likely option out of up to 255, with a probability distribution. score returns a mean level index across an ordered set of descriptions. One request, one forward pass over the state, zero generated tokens.
Send Kev-4B a support ticket that says shoes arrived two weeks late in the wrong size and there are two charges on the card, and ask which department should handle it. It comes back with returns at 0.47, shipping at 0.28, and billing at 0.25. All three things are true about the ticket and the distribution says so. That is the argument for getting probabilities back instead of a label, and it is the argument for the whole category.
Under the hood, the pointer head scores each option's closing-tag hidden state against the question's decide-token hidden state, and a softmax turns those scores into probabilities. The attention mask lets each token read the state and its own question but nothing else, and each question's position IDs restart just after the state, so the model processes the state once and answers every question independently.
Qwen3.5 complicates that, and the way Palmer handled it is the most instructive engineering detail in the repo. Qwen3.5 interleaves attention layers with Gated DeltaNet layers, which are recurrent and ignore attention masks entirely. Masking cannot isolate questions from each other in a recurrent layer. So on those bases each question runs as its own row, state plus that one question, with the server computing the state once and reusing its cache across rows. Isolation becomes exact rather than mask-enforced. On attention-only models the two forms produce identical probabilities, which the repo has a parity test for.
That is a real solution to a real problem, and it costs something. Which brings us to the part of the README nobody is quoting.
The regression in the serving table
The story around decision models this weekend was that they went local. An Apple Silicon MLX port of Convai's Laya appeared, a CoreML recipe for running Laya offline on a Mac M4 hit Hacker News with 156 points, and Kev shipped open weights with a protocol-compatible server. The narrative wrote itself: the paid endpoint is over, run it on your laptop.
Kev's own serving table says otherwise, for Kev specifically, on a Mac. Median model time in bf16 on an M5, five questions with three options each over a state of about 230 tokens:
| Model | Time | Previous generation, same request |
|---|---|---|
| Kev-0.8B | 329 ms | Kev-0.6B (Qwen3): 123 ms |
| Kev-4B | 779 ms | Kev-4B (Qwen3): 174 ms |
| Kev-9B | about 2 s | Kev-8B (Qwen3): about 300 ms |
The Qwen3.5 upgrade made every size roughly two and a half to seven times slower on Apple Silicon. The cause is stated plainly: there are no fast kernels for the DeltaNet layers on MPS, so PyTorch runs reference code. The README's own recommendation is that if you serve on a Mac and need low latency, use the Qwen3 models for now, and an MLX backend is the next planned change.
On CUDA the picture inverts completely. With flash-linear-attention installed, a five-question request takes tens of milliseconds on an H100.
So the accuracy went up and the latency on consumer hardware went up with it. Out-of-domain accuracy on the locked test moved from 0.780 to 0.852 for the 9B, 0.806 to 0.837 for the 4B, 0.642 to 0.668 for the small one. The repo calls this a controlled comparison and it is, because only the base changed while the data and settings stayed fixed. You are being offered a real trade, stated in the open, and which side you want depends entirely on what silicon you are running.
That is the position I want to leave you with, and it generalizes past Kev: "run the decision model locally" is not one claim. It is a claim about your hardware, and the hardware most people mean when they say it is the one this particular upgrade regressed on.
How the numbers compare to the thing it clones
Kev publishes its comparison against Jev, the hosted model whose API it mirrors, and it publishes the caveat with it. On the new-source development set, Kev-9B scores 0.822 against Jev's 0.857, a 3.5-point gap. On the locked test Kev-9B reaches 0.852, and Jev has not been run on that set, so the column is empty.
Palmer's own sentence about this is better than anything I would write over it: we do not know which datasets Jev was trained on, so this is not a controlled comparison of the two architectures. He also notes that no Jev outputs were used for training Kev, and that the architecture came from a third party's public write-up rather than from anything proprietary.
Two external test sets from other projects, with their published live Jev results, fill in the picture. On one project's 144 authored decisions, Kev-9B scored 0.917 against Jev's 0.965. On another's 900 support tickets, Kev-9B scored 0.952 on routing and 0.911 on tone, against Jev's 0.897 and 0.914. Which is to say: behind on curated reasoning, ahead on the bread-and-butter ticket routing that most people actually deploy this category for.
The knowledge gap is the one that will not close by training harder. MMLU 0.74 against Jev's 0.90, MMLU-Pro 0.52 against 0.84. The repo attributes that to the base model and notes that training a version on a 35B mixture-of-experts base did not move it.
Put this into practice
If you have a per-call decision API in production, the cheapest experiment available to you this week is to point your existing client at a local Kev and measure the delta. You do not have to change your code.
Start here. You need Python 3.12 and uv.
git clone https://github.com/jaredpalmer/kev.git && cd kev
uv sync --extra serve
KEV_DTYPE=bf16 uv run --extra serve python -m kev.serve --run jaredpalmer/kev-4b --port 8009
Start with Kev-4B. The first run downloads the adapter and the base model. If you are on a Mac and latency matters more than the last two accuracy points, pass jaredpalmer/kev-4b@qwen3 instead and get the older, faster generation.
Then change one line in whatever already calls the hosted API: set base_url to http://127.0.0.1:8009 and api_key to anything. Run your existing traffic through both and compare.
Two settings are worth turning on before you judge the results, because both are off by default and both fix a real problem. KEV_TEMPERATURE=2.0 applies a single fitted temperature that cuts Kev-9B's calibration error on new sources from 0.106 to 0.050 and its confident errors, meaning wrong answers asserted at 0.9 probability or above, from 8.7% to 4.4%, which is close to Jev's 3.7%. KEV_DATE_FACTS=1 appends the number of days between any two absolute dates it finds in the state, because the model cannot subtract dates reliably but can use a stated day count. On deadline policy questions that moves Kev-9B from 0.80 to 0.90, against Jev's 0.93.
If the out-of-the-box accuracy is not enough, fine-tuning on a few hundred of your own labeled examples is the next move, and there is one flag that decides whether it works. Use --init_from to start from a released checkpoint rather than from the base model. The README reports one user's test on 836 support-tool decisions where a fine-tune from the base scored 0.33 on Kev's own evaluation set, against 0.84 for the released model, while the same data with --init_from held 0.83 there and reached 0.88 on the new domain. That is one user's result on one dataset, and it is directionally the difference between a useful afternoon and a wasted one.
The training data format is the API request shape with a label added to each question, one JSON object per line. Keep ten to twenty percent aside for evaluation.
Honest limitations
The server binds to 127.0.0.1 and has no authentication. The README says to keep it local unless you add authentication yourself, and that is the warning that gets ignored the first time a teammate wants to hit your endpoint. What gets exposed when someone puts a reverse proxy in front of it is not a chat model, it is a structured-verdict endpoint that downstream systems act on without a human reading the output. Nobody audits a decision endpoint the way they audit a database.
It also handles one request at a time. It caches repeated state text, but it does not batch requests from different callers. That makes it a development and single-tenant tool as shipped, not a service.
Raw probabilities are over-confident on new sources, which is what the temperature setting exists for, and the repo says to test your threshold on your own data before trusting it. Changing the order of options can change an answer, and question isolation does not prevent that. Training used at most 384 state tokens while serving allows 8,192 for the state plus one question, so long contexts are outside what the training covered.
There is one more limitation the repo is unusually honest about. Fine-tuning made the base model worse at something it could already do. The untrained Qwen3.5-9B base scored 0.82 on deadline policy questions and the first Kev-9B scored 0.72, a regression caught by an outside contributor and filed as an issue. Training on examples that state the day count, plus the date-facts setting, recovered it to 0.90. That whole sequence, including the credit to the person who found it, is in the README rather than in a postmortem nobody publishes.
Two smaller flags. The confidence field is described as an approximation of TypeSafe's formula, which is not public, and the repo states outright that neither confidence nor probability is a measured accuracy rate. And the project was built with Devin, which the author discloses in the authors section. Neither is disqualifying. Both are the kind of thing you want to know before you build on something.
What I would actually take from this
The model is a weekend-old artifact and it will be superseded. The evaluation discipline around it is the part worth stealing.
Kev keeps a locked test set that is read once per released checkpoint, after model selection, with a Modal command whose docstring says "one read, ever." Development results choose the model. The test set only confirms it. The Qwen3.5 experiment has its criteria written down in advance in a separate file, and the results are reported against those criteria whether or not they met them. Every accuracy delta in the release notes carries a confidence interval, including the ones that are within noise and are labeled as such.
That is a higher standard of reporting than most funded labs manage, running in a repo with 1.6k stars and one listed author. If you take one thing from Kev into your own project, make it that, not the weights.
And if you are paying per typed decision call today, spend the afternoon. The interface is already compatible. The only thing you are risking is finding out.
Sources: jaredpalmer/kev README and the kev-family release, both read cache-busted on 2026-09-21. Release notes were last edited 2026-09-21 11:27 UTC. The v0.1.0 prototype release documents the retired Qwen2.5-0.5B checkpoint. Weights are in the Kev collection on Hugging Face.
Medium metadata
- Title: Kev Speaks a Paid API's Protocol From localhost, and Its Own Table Says Don't Run It on a Mac
- Subtitle: An open decision-model family that the vendor's own Python SDK will talk to, and the Apple Silicon regression sitting in its serving table
- Tags: Open Source, AI Agents, Machine Learning, Local LLM, AI Engineering
- Suggested reading time: 9 minutes
- Canonical: import from the fervorai.dev URL