Cua's Metal Capability Shim Made llama.cpp 11x Faster by Changing Two Answers
A macOS VM was running local models an order of magnitude slower than the same machine outside the VM. The hardware was fine. The GPU was describing itself badly.
Two values. That is the entire intervention.
Cua's Metal capability shim, published August 11 by Francesco Bonacci and Johnny Franks, answers supportsFamily: through Apple family 9 instead of family 5, and reports 64 KB of maximum threadgroup memory instead of 32 KB. Nothing else changes. Same host, same guest, same Apple GPU, same virtualization path, same llama.cpp binary.
On an M1 Ultra, TinyLlama 1.1B prompt processing went from 431.86 tokens per second to 4,786.70. Generation went from 12.63 to 206.60. That is 11.08x and 16.36x, and the prompt number lands at 98.25% of what the same model does on bare metal.
The GPU was always capable of that. It was answering a question conservatively, and every piece of Metal software downstream believed it, exactly as Apple's own documentation instructs.
Why this is more interesting than a benchmark
I have read a lot of "we made X faster" posts this year and skipped most of them. This one is different in a way worth naming, because it changes what you should check when local inference is slow.
The usual mental model for a slow VM is overhead. Virtualization taxes you, the tax is a percentage, you accept it or you go bare metal. That model is fine for CPU and mostly fine for I/O. It is wrong for the Metal path inside a macOS guest, and being wrong about it costs you an order of magnitude, not a percentage.
What is actually happening is a negotiation. Apple's Virtualization.framework gives the guest a paravirtualized graphics device backed by the host's real GPU. Apple documents GPU capability through feature tables and tells developers to query the device at runtime. So llama.cpp asks the device what it can do, the paravirtualized device gives a conservative answer, and llama.cpp picks the kernels that answer implies.
Everyone in that chain behaved correctly. The result was still a tenth of the available speed.
That is the reusable lesson, and it generalizes past this one repo: when a stack selects code paths from a runtime capability probe, the probe becomes part of your performance surface. It is as load-bearing as the driver.
The mechanism, and what stayed untouched
In Cua's stock Tahoe guest, the virtual device reported roughly an Apple 5-era family, 32 KB maximum threadgroup memory, and SIMD-group matrix support unavailable. Flipping the two reported values turns three code paths back on:
| Capability | Stock guest | Tested profile |
|---|---|---|
supportsFamily:1009 |
false | true |
| SIMD-group matrix | off | on |
| SIMD-group reduction | off | on |
| bfloat16 | off | on |
| Maximum threadgroup memory | 32 KB | 64 KB |
The shim is a process-scoped compatibility layer injected with DYLD_INSERT_LIBRARIES. It intercepts selected capability queries inside one guest process and changes what comes back. Common, Mac, Metal, and working-set-size values keep their stock settings. The host, the guest kernel, other guest processes, content protection, and licensing state all stay as they were. The work still executes on the host's Apple GPU through Apple's existing bridge.
This is not GPU passthrough in the sense most people mean it. VFIO on an x86 Linux host assigns a physical PCI device to a guest through an IOMMU. Nothing like that is happening here. The Cua team is careful about the distinction, and also notes that Tart, another Virtualization.framework frontend, has an open issue about the same capability gap under the name people search for.
They also stripped their own research build down before releasing it. The original hook had clock and timing interposition, mesh substitution, a ray-tracing override, an argument-layout guard, and a pipeline-compilation fallback. All of that came out. What ships is small enough to audit, and a malformed config leaves the process on its stock path.
The numbers, including the one that stayed flat
Three models, one host, published raw logs.
TinyLlama 1.1B Q4_K_M through llama.cpp b10167, medians of ten samples: prompt processing 431.86 to 4,786.70 tok/s (11.08x, 98.25% of host), generation 12.63 to 206.60 tok/s (16.36x, 72.06% of host).
Gemma 4 12B QAT Q4_0, same shape: prompt processing 71.66 to 515.76 tok/s (7.20x, 99.59% of host), generation 3.41 to 49.67 tok/s (14.54x, 94.82% of host).
Meta's Muse Glimmer 30B Q4_K-M in a 64 GiB guest through llama.cpp b10359: prompt processing 25.83 to 194.97 tok/s (7.55x), generation 2.38 to 21.08 tok/s (8.87x). This one is text-only llama.cpp, no Ollama, no multimodal projector, no drafter, and Cua says so directly rather than letting the number float.
Then MLX-LM, which did not move at all. Prompt processing 1,656.55 to 1,665.47 tok/s, a ratio of 1.005x. Generation 172.09 to 170.86, a ratio of 0.993x.
The flat result is the best part of the post. MLX-LM was already fast in the stock VM, so there was nothing to recover, and that null result then shaped the release. During ablation, advertising MTLGPUFamilyMetal3 made MLX request a residency set the paravirtualized device does not provide. So the shipped shim keeps Metal 3 at its stock value and limits changed answers to Apple-family enums.
A team that publishes the workload their fix does not help, and then narrows the fix because of it, has earned more trust than the speedup number alone would buy. The raw JSON, model hashes, binary hashes, exact arguments, and environment records are all in the repo. They also note that they discarded and reran a preliminary Gemma series after spotting another compute workload on the host, which is the kind of disclosure most benchmark posts leave out.
Put this into practice
You need an Apple Silicon Mac, a macOS guest under Lume, and about half an hour. Start by finding out whether you actually have this problem, because if you are on MLX you probably do not.
1. Probe before you patch. Cua ships a capability probe alongside the shim. Run it in your guest and check what family and threadgroup limit your device reports. If you see Apple family 5 and 32 KB, you are in the situation this fixes. If you are running MLX-LM, benchmark first and expect nothing.
2. Build the dylib.
cd libs/lume/metal-capability-shim
./Scripts/build.sh
./Scripts/verify.sh
The verify step checks both architecture-specific builds. Do not skip it, because a mismatched dylib fails silently by leaving the process on the stock path, which looks exactly like "the shim did nothing."
3. Enable the unrestricted feature level on the host and restart the VM.
lume stop my-vm
defaults write com.apple.gpusw.ParavirtualizedGraphics \
ForceUnrestrictedDeviceFeatureLevel -bool true
lume run my-vm
This is a host preference, set per macOS user, and it applies to VMs that user launches. It is also your rollback point.
4. Scope activation to one process.
lume ssh my-vm \
"DYLD_INSERT_LIBRARIES=/path/to/LumeMetalCapabilities-arm64.dylib \
LUME_METAL_APPLE_FAMILY_MAX=1009 \
/path/to/metal-capabilities 1009"
For an inference server or a long-running worker, do not put this in your shell profile. Use a per-workload LaunchAgent so the login session stays stock and only the workload you meant to change is changed. Cua's Lume guide has the template and the checksum steps.
5. Benchmark honestly. Run llama-bench with -r 10 on stock, then with the shim injected, in the same window, on an otherwise idle host. Cua's own numbers are medians of ten samples and they still threw out a run when the host got busy. A single-sample comparison on a laptop doing other things will tell you a number, just not a true one.
Rolling back is removing the environment variables and restarting the workload. To go all the way back, stop the VM, delete the ForceUnrestrictedDeviceFeatureLevel preference, and start it again.
Honest limitations
Cua labels this a research release and the limitations section in their post is more candid than most vendor posts get, so I will restate it rather than soften it.
The technique relies on private, version-sensitive behavior in the guest's Metal implementation. Apple can change it in any macOS release, and there is no promise it survives the next one. Cua has publicly asked Apple for clarification on the intended supportability of the unrestricted feature level, which tells you the support status is genuinely unresolved rather than merely undocumented.
The validation is narrow: one M1 Ultra with a 48-core GPU on macOS 26.6.1, one Tahoe guest, three llama.cpp models, one MLX-LM compatibility run. Different chips, different guest releases, different Metal APIs all need their own tests. A reported family describes the paths their tests covered and nothing else.
It is per-process, so hardened or platform-protected executables may refuse library injection outright. If your workload is a signed app rather than a CLI binary, this may simply not apply to you.
And it is still a VM. Generation on TinyLlama reached 72.06% of host speed, not 98%, so a real gap remains even after the fix. If your goal is maximum tokens per second and you can run on the host, run on the host. This matters when you need the isolation, the snapshotting, or the fleet, which is exactly why it comes out of a computer-use VM project rather than an inference project.
One thing I have not verified and neither has anyone else publicly: whether the recovered speed holds under sustained load rather than benchmark bursts. Every number here is llama-bench on 512-token prompts and 128-token generations. That is a clean measurement and it is not a two-hour agent run.
The question this opens
If a paravirtualized device under-reports its capability and every Metal application politely believes it, llama.cpp is not the only thing losing performance to a conservative answer.
Renderers pick paths from the same probe. Video encoders do. Any framework doing feature detection at runtime does. Cua tested inference because inference is what they needed, and the only reason we know the gap exists is that someone was annoyed enough to check the probe instead of accepting the overhead story.
So the useful thing to do is check your own. If you are running anything GPU-bound inside a macOS guest and have been telling yourself that VMs are just slower, run the capability probe and find out whether that is true or whether you have been reading a conservative default as a hardware limit.
If you test a chip or a guest version Cua has not covered, they are asking for exactly that: host chip, host and guest versions, workload, and both sets of numbers, on an issue. I would rather read ten of those than one more benchmark chart.
Sources: Cua, "Apple Silicon and macOS VMs: 11-16x Faster LLM Inference with llama.cpp" (Aug 11, 2026); Cua evidence directory, raw benchmark logs and hashes; Apple, VZMacGraphicsDeviceConfiguration; Apple, Metal feature set tables; Apple, Detecting GPU features and Metal software versions; Hacker News discussion.