deltafin Runs a 2.8-Trillion-Parameter Model on One MacBook, Then Publishes the Six-Minute Wait

One token per second is the headline. The number underneath it says local mixture-of-experts inference is a scheduling problem in a bandwidth costume.

The README opens with the win and then immediately hands you the knife. Kimi K3, all 2.8 trillion parameters, nothing pruned, running on a single M5 Max MacBook Pro with experts streamed off four SSDs, at one token per second. Second bullet, bolded: the honest limit. A 512-token prompt takes about 6.3 minutes to produce its first token. Cause found, fix planned, not built.

Most projects would bury that. This one leads with it, and the choice turns out to be load-bearing, because the six minutes is where the actual engineering result lives. Argonaut Labs published the benchmark set on September 8 alongside two supporting documents, and read together they say something that contradicts the reflex every person building a home inference rig has: buying more storage bandwidth stops helping well before the waiting stops.

What the thing is

Deltafin is a single compiled Rust binary that runs Kimi K3 without shrinking it. That constraint is the entire design. Moonshot's K3 is a 2.8T mixture-of-experts model with roughly 1.45 TB of expert weights, targeted at something like sixteen nodes and 4.8 TB of aggregate VRAM. Other projects get it onto consumer hardware by re-encoding the expert bank down to about three bits. Deltafin's README is pointed about that: those weights are no longer the ones Moonshot released, and nobody, including Moonshot, has measured what the compromise costs.

So deltafin keeps every expert byte exactly as shipped, keeps all sixteen routed experts live on every token, and buys speed only from places that cannot change the output. Small draft models are allowed to guess ahead; K3 verifies every guess. The stated rule is that K3 decides every token and nothing reaches you without its sign-off.

Worth stating plainly up front: this is a fork of gavamedia/deltafin, which built the engine. The fork credits that prominently and carries dual MIT copyright lines for GAVAMEDIA Corporation and Argonaut Labs. What Argonaut Labs added is instrumentation and a public benchmark set, and that is the part I care about here.

The numbers, and the one that matters

Measured 2026-09-08 on an M5 Max with 128 GB, four SSDs:

1.0015 tok/s steady decode over 512 generated tokens with the drafter on
1.1252 over 128 tokens
0.9631 on the 17-token public prompt where upstream reported 0.684
roughly 375 seconds to first token on a 512-token prompt

Every figure is one cold run with the exact prompt, per-run logs published. Four-drive figures are medians of two runs; the drive-ladder rungs below four are single runs. The README says all of this without being asked.

Now the drive ladder, which is the finding worth stealing. They removed drives one at a time and re-ran:

drives	generation 128, drafter on	share of four-drive
1	0.5350 tok/s	52%
2	0.7596	73%
3	0.9375	90%
4	1.0377	100%

Standalone read ceilings on the same day: internal SSD 13.6 GB/s, two externals at 7.1 each, the fourth at 5.7 falling to 5.1 under queue depth. One drive at 13.6 GB/s gives about half of what roughly 33 GB/s of combined ceilings gives. Two mirrors at 20.7 GB/s give about seven tenths. Removing the slowest drive from the four-drive layout costs 8 to 10%.

That reads like a clean bandwidth story, and the document is careful to say both readings are true at once. Below four drives, bandwidth binds hard. At four drives, after the read balancer, every drive is already running at 90 to 100% of its own ceiling and storage is no longer the constraint on decode. The remaining cost is scheduling.

Which is exactly what the prefill document proves.

Where six minutes actually goes

The engine's own phase timers, on a four-drive 512-token prompt with first token at 373.1 seconds:

282.8 s (76%) waiting for expert reads to land
about 60 s (16%) in expert kernels
20.2 s (5%) in attention
about 9 s in binding, planning and everything else

So three quarters of the wait is reads. The obvious conclusion is that you need faster storage. The obvious conclusion is wrong, and here is the arithmetic that kills it.

During that prefill window the four drives read 8,977 GB. The engine's own byte counter agrees at 9,033.8 GB. The entire expert set, read once, is 1,443 GB. That is 6.2x amplification: nine terabytes of reads for a 1.4 terabyte model, on a single prompt.

The cause is a scheduling shape, not a hardware limit. The 512 rows get processed as eight row-passes of 64, and each pass re-reads that layer's expert union in tiles of at most 64 experts. layer_passes comes out at 744, which is 8 × 93. An expert routed to rows in k of the eight passes gets read k times, and the observed average is 6.2. The control case makes it unambiguous: a 6-token prompt requests 169 GB, one pass over what it needs, no amplification at all.

The drives were sitting at their combined 24 GB/s ceiling for the whole prefill. They were not slow. They were being asked to deliver the same bytes six times.

Restructure the prompt pass to be expert-major per layer, the document argues, gathering every row's routes and grouping rows by expert so each expert reads once, and reads fall from 9.0 TB toward 1.44 TB. At 24 GB/s that is about a minute of reads instead of 283 seconds of waiting, putting first token somewhere near two to two and a half minutes before you overlap reads with kernels.

They have not built it. And the gate they set for accepting it is the most professional thing in the whole repository: requested bytes must fall and layer_passes must drop from 744 to 93, because bytes falling while passes stay at 744 would mean the experts got cached rather than the schedule getting fixed. Output must stay token-identical on both control prompts, and decode must stay inside the settled repeat spread. That is a pre-registered success criterion for a performance fix, written before the fix exists, designed to catch the specific way the author could fool himself.

Put this into practice

You are probably not streaming a 2.8T model off SSDs this week. The transferable parts are cheaper than that.

Report time to first token or do not report throughput. Deltafin's 1.00 tok/s is a fine number attached to a six-minute wait, and either number alone misrepresents the machine. If you publish a local-inference measurement, publish both, with the prompt length, because TTFT scales with prompt length in ways decode does not.

Instrument read amplification before you buy hardware. The whole diagnosis here comes from two counters: bytes actually requested during a window, and the size of the working set read once. Divide. If the ratio is meaningfully above one, you have a scheduling bug and a faster disk will just execute the same waste faster. This applies well outside inference, to any system that streams a working set larger than memory.

Build the ladder before you fill the slots. Rather than assuming four drives beats two, remove one and re-measure. Deltafin's method is simple enough to copy: point a role's environment variable at an empty directory so nothing resolves there, then re-run the identical prompt cold. The answer, 52 / 73 / 90 / 100, is the shape you need to decide whether the next drive is worth its price.

Watch the tail, not the total. The reason the ladder is sublinear is that the slowest of each layer's sixteen reads sets the pace. Aggregate bandwidth is the wrong summary statistic for any workload that must complete a fan-out before it can proceed. One slow device in a striped set costs you more than its share.

Pre-register the check that would catch you cheating. The two-part gate here, bytes down and passes down, exists because there is a cheap way to make the headline number improve without fixing anything. Before you optimize, write down the measurement that distinguishes a real fix from a cache warming up.

Where this evidence stops

These are single cold runs on one machine. The four-drive figures are medians of two, the one-, two- and three-drive generation rows are single runs, and the three-drive rung ends before its drafter-off 512 arm so it has no prefill measurement at all. Nobody has reproduced any of it on other hardware.

The comparison to upstream is not like-for-like, and the document says so first. The internal-drive-only median on the public France prompt is 0.5483, below the 0.6840 public reference, because the public setup is not documented as single-drive. The fair comparison is the four-drive 0.9631, and presenting the single-drive number against the public figure would have flattered the fork.

Determinism is bounded, not claimed. On a given storage layout the prompts of record are token-identical with the drafter on or off. Across layouts, expert tiles arrive in different orders, floating-point accumulation order changes, and at a genuine near-tie a token can flip. One flip was observed at roughly token 305 of 512 on the two-drive rung. They checked 40 of 40 sampled expert files byte-identical across holders, so nothing was corrupted, and they excluded the affected arm from the headline comparison and reported the full-length figures anyway. That is handled well and it is still a real constraint: identical output across hardware configurations is not something this design guarantees.

The prefill fix is planned and unbuilt, so the two-and-a-half-minute projection is an estimate from a byte count, not a measurement. And an earlier classification of this same engine on August 26 concluded prefill was attention-bound, which the current document says is superseded rather than contradicted, since attention then cost about 140 seconds and now costs 20. Both were true of their engine. Neither is guaranteed true of the next one.

Finally, this is research, not a product. The README says so, in those words, and prices the point of comparison at a $15,000 home setup against the roughly $2,000,000 of infrastructure Kimi recommends. Take the methods. Do not take it as a deployment plan.

What to take away

The reason to read this repository is not the trillion-parameter stunt. It is that somebody instrumented a system carefully enough to prove their own bottleneck was in their code rather than in their hardware budget, and then published that before shipping the fix.

That inversion, blaming the schedule instead of the disk, is available to almost every system you run. Most performance work starts with a purchase order because measuring is harder than buying. It took two counters here to establish that four SSDs at their combined ceiling were doing six times the necessary work, and the fix costs nothing but a restructured loop.

Go find your amplification factor. If it is close to one, buy the drive. If it is not, you already own the speed.

Sources: argonautlabsai/deltafin · SCALING.md · PREFILL.md · gavamedia/deltafin · moonshotai/Kimi-K3

Medium metadata

Title: deltafin Runs a 2.8-Trillion-Parameter Model on One MacBook, Then Publishes the Six-Minute Wait
Subtitle: One token per second is the headline. The number underneath it says local mixture-of-experts inference is a scheduling problem in a bandwidth costume.
Tags: local llm, mixture of experts, performance engineering, open source ai, kimi k3
Canonical: import from the fervorai.dev URL