TurboFieldfare Runs Gemma 4 26B in About 2 GB of RAM. The Other Number Is 14.3 GB.

A Swift and Metal runtime streams a mixture-of-experts model off SSD one token at a time so an 8 GB MacBook Air can answer. What that actually buys, what it costs, and how to try it in an evening.

The first line of TurboFieldfare's README is not a feature claim. It is a complaint. "Memory got expensive. So I gave a 26-billion-parameter model a ~2 GB budget."

The model in question is Gemma 4 26B-A4B, instruction-tuned, 26 billion total parameters with roughly 3.88 billion active per token. Installed on disk it takes 14.3 GB. The repository says it will run on an 8 GB M2 MacBook Air, a machine that cannot hold the weights, and the measured decode speed there is 5.1 to 6.3 tokens per second.

Both of those numbers are true at the same time, and understanding how is worth more than the demo.

What the constraint actually was

For two years the standard advice on local inference has been a subtraction problem. Take your RAM, subtract the OS, subtract your browser, and whatever is left is the model you get to run. Quantization moved the line. It did not move the shape of the rule, which was always that the weights live in memory.

TurboFieldfare's argument is that for a mixture-of-experts model, that rule was never load-bearing. It was a scheduling default nobody had bothered to challenge on this hardware.

The mechanism is worth reading carefully, because it is the transferable part. At each transformer layer, Metal computes attention and the router from weights that stay resident, about 1.35 GB of shared core plus an FP16 KV cache. The router emits the top-8 expert IDs for that token. The CPU takes those IDs and plans against a 16-slot LFU cache for the layer, then fills the misses with bounded parallel pread calls straight into Metal-visible buffers. While those reads are in flight, the GPU computes the resident shared-expert branch. Then the two results combine.

The GPU is working during the disk read. That overlap is the entire trick.

Andrey Mikhaylov, the author, is an iOS and Metal engineer, and it shows in which problem he chose to solve. He reports that the obvious approach, memory-mapping the weight file and letting the kernel page them in, delivers about 0.50 tokens per second. Scheduled pread with the cache plan gets 4 or more. Same disk, same model, same machine. An order of magnitude sat in how the reads were issued.

Prefill uses chunks of up to 128 tokens so a single fetched expert can serve multiple rows before it gets evicted. KV storage is FP16, circular for the 25 sliding-window layers, linear for the 5 full-attention ones. None of this is exotic research. It is careful I/O engineering applied to a problem everyone had filed under "hardware limit."

The disclosure is better than the benchmark

The repository ships a curated record of 102 measured experiments across kernels, caching, I/O, prefill, and decode, and the accompanying write-up is organized around the largest wins, the plausible ideas that failed, and early results that reversed under stronger validation.

That last category is the one almost nobody publishes. A result that looked good and then died when tested harder is the most expensive thing an engineer learns and the least rewarded thing to write down. Having 102 audited entries including the failures does more for my confidence in the throughput figures than any single benchmark number would.

Compare that to how most trending inference projects present themselves: one screenshot, one tokens-per-second figure, no error bars, no mention of what did not work. TurboFieldfare also ships a community benchmark guide asking other people to measure their own Macs, which is the behavior of someone who expects to be checked.

The README's own honesty extends to the afterword, where Mikhaylov says the project "may not be the most practical." He is right, and he said it before anyone else could.

Put this into practice

This is a clone-and-build project, not a package install. Budget an evening, mostly for the download.

Check the gate before anything else. Apple Silicon only, arm64 only, macOS 26 with Metal 4, Xcode 26 and Swift 6.2 or newer, and about 14.3 GB of free storage plus room for the transfer. Older macOS and Metal versions are unsupported, so this rules out a lot of otherwise fine hardware.
Clone and build the whole package. git clone, then swift build -c release, then run .build/release/TurboFieldfareMac. Build the complete package rather than a single target, because the Mac app needs its sibling decode-service executable.
Let the installer do the download. Choose Install in the app. It streams byte ranges from a pinned Hugging Face revision and repacks them directly into the .gturbo layout as they arrive, so it never writes a second full checkpoint to disk. The transfer is around 15 GB and it validates a manifest and file hashes before accepting the install.
Clear memory before you load. The README tells you to close memory-heavy apps and run memory_pressure -Q first, and to run only one local-model process at a time. On an 8 GB machine this is the difference between working and swapping.
Use the CLI when you want a real comparison. TurboFieldfareCLI with --prompt, --temperature 0, and --seed gives you reproducible greedy output. --messages-file gives you proper chat formatting. Timing goes to stderr, so --quiet keeps your scripts clean.
Read OPTIMIZATION_JOURNEY.md even if you never run the model. For most people this is the actual deliverable. It is a working record of how to find an order of magnitude in an I/O path, and the technique transfers to problems that have nothing to do with Gemma.

If you want to point an existing local client at it, version 0.2 adds an experimental OpenAI-compatible server. Treat "experimental" as the operative word.

Honest limitations

The 2 GB figure is a RAM measurement and only a RAM measurement. The model still occupies 14.3 GB of SSD, the first install pulls roughly 15 GB over the network, and every token you generate issues real disk reads. What TurboFieldfare does is trade a resource you do not have for one you probably do. That is a good trade on an 8 GB laptop and a pointless one on a workstation.

The speed cost is the number that decides whether you should use this. Independent rough benchmarking reported by Machine Learning Street Talk puts Ollama 0.20 and raw MLX-VLM at 65 to 75 tokens per second on the same Gemma 4 26B-A4B, on an M4 Max, with MLX-VLM holding about 16 GB dynamically. TurboFieldfare's own measurements are 5.1 to 6.3 tok/s on an 8 GB M2 Air and 31 to 35 tok/s on a 24 GB M5 Pro. Those are different chips and a rough comparison rather than a controlled one, so do not treat the ratio as precise. The direction is not in doubt. If your machine can hold the model, MLX will be faster and you should use MLX.

Scope is narrow on purpose. Text only. No tool calling, no image input, one pinned instruction checkpoint. That combination rules out agent work entirely, which is most of what I would want a local 26B for.

The project is also very young in the ways that matter for a dependency. Two releases, the latest tagged 0.1.1 on July 22, eight commits on main, and a single author. GitHub served 85 stars, 0 forks, and 0 watchers when I read the page, against a third-place placement on a daily momentum board. Momentum boards measure attention, not adoption, and this gap is the clearest example of that I have seen this month. Nothing is wrong with the repository. It just has not been used by enough people to have been broken by them yet.

One thing I could not find an answer to: sustained per-token SSD reads over long sessions are a wear pattern, and the documentation does not quantify it. I am not claiming it is a problem. I am saying the question is open and I would want a number before running this as a daily driver.

Apache 2.0 covers the source. It does not cover the weights, which the installer downloads separately and which stay governed by their own terms. The project states plainly that it is independent and not affiliated with Google.

What to take from it

The interesting claim here is not that a 26B model fits on a MacBook Air. It is that a limit everybody treated as physics turned out to be a scheduling default, and one engineer with a Metal background found an order of magnitude in the difference between mmap and a planned read.

That should make you suspicious of the other limits in your stack that nobody has questioned recently. Pick one thing you have been routing around because "it does not fit," and go find out whether it does not fit or whether nobody has scheduled it properly.

Then go read the failed experiments. That file is the part of this project that will still be useful after the model is obsolete.

Sources: drumih/turbo-fieldfare on GitHub (README, benchmarks, and experiment record); Show HN discussion; Machine Learning Street Talk's rough Gemma 4 26B-A4B benchmark on M4 Max; Gemma 4 model card.