NVIDIA's Personal AI Router Will Not Pool Your GPUs, and Saying So Out Loud Is the Best Thing in the README
PAIR routes whole requests across machines on your LAN. It refuses to shard a model, and the refusal is printed in a blockquote near the top. Here is when that is exactly what you want, and when you need llama.cpp's RPC backend instead
Eight paragraphs into the README for NVIDIA's Personal AI Router there is a blockquote, and it is not a feature:
PAIR routes each independent request to one node. It does not pool GPU memory, combine GPUs into a larger logical GPU, shard one model across machines, or split an in-flight inference request between nodes.
Four negations in one sentence, in bold, above the fold. Most projects bury that paragraph in a FAQ, or leave it out entirely and let the GitHub issues discover it. NVIDIA put it where you cannot miss it, which tells you they knew exactly which wrong idea people would arrive with.
They are right about the wrong idea. Say "route inference across the machines in your house" to almost any local-AI person and they hear one thing: my laptop has 16GB, my desktop has 24GB, so I can finally run the 32B model. That is not what this does. That is not close to what this does.
The gap between what people hear and what the tool delivers is worth understanding, because both things are legitimately useful and they solve completely different problems.
What PAIR actually is
NVIDIA Personal AI Router, Apache-2.0, currently 605 stars, latest release v0.1.1 on 2026-08-28. Version numbers matter here; this is early software from a large company, not a mature product.
It is a local inference router for a group of compatible computers on the same network. It discovers participating nodes, manages the inference engines running on them, and presents Ollama-compatible and OpenAI-compatible proxy endpoints. Your application talks to a normal endpoint. PAIR decides which machine serves each request.
The routing decision considers engine availability, model availability, and current workload, and prefers nodes that already hold the requested model. Pairing two machines is a six-digit PIN: the inviting node displays it, the invited node enters it.
Supported engines are Ollama and LM Studio. That is the whole list. Supported operating systems are Windows 11, Linux, and macOS, on x64 and arm64, with Windows on ARM marked experimental, and nodes running different operating systems can be paired with each other. Installers are a Windows .exe, a Debian .deb, and a macOS .dmg.
Once it is up, a request looks like this against Ollama's default port:
curl http://127.0.0.1:11434/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{"model":"qwen4:12b","messages":[{"role":"user","content":"In one sentence, what does a router do?"}]}'
Ordinary OpenAI-shaped JSON comes back, and the job shows up in PAIR's Jobs view naming the node that served it.
Concurrency, not capacity
Here is the distinction the blockquote is protecting.
Capacity is "this model does not fit on any one of my machines." Solving that means splitting the model itself: layers on one box, layers on another, tensors moving over the network mid-inference. PAIR does not do this and says so.
Concurrency is "I have eight agents running and one of them is always waiting." Solving that means having somewhere else to send request number two while request number one is still going. PAIR does exactly this, and the README names multi-agent applications as the use case.
If you run agent swarms locally, concurrency is probably your actual pain and you may have misdiagnosed it as capacity. A single Ollama instance serializes; your second agent sits in a queue behind your first. Two machines with the same 12B model loaded is not a bigger brain, it is two brains, and for a multi-agent workload two brains is the thing you wanted.
If you genuinely need capacity, the tool for that is llama.cpp's RPC backend. It serializes tensor operations over TCP to one or more ggml-rpc-server instances on other machines, so a remote box appears to the main process as just another device with memory. By default it distributes weights and KV cache across all devices, local and remote, in proportion to available memory, and --tensor-split lets you override the proportions. Exo solves the same capacity problem by a different route: its README describes topology-aware auto parallel, splitting a model using a realtime view of device resources and the network latency and bandwidth between each link, rather than one fixed memory-proportional rule. Either way the cost is real, because you are pushing tensors over your LAN mid-inference, and your Wi-Fi will feel it.
Different problems. Different tools. PAIR is honest about which one it is.
Put this into practice
The lowest-friction path in is deliberately boring, and I would take it as written rather than getting clever.
Install a signed release, not a source build. Download from the releases page. The README is direct that this is the recommended path: a released installer is signed, sets up the background services and desktop app together, adds the Windows firewall rules PAIR needs, and checks an update feed. A self-built copy is unsigned and checks nothing, so you upgrade it by pulling and rebuilding. On Linux that is sudo apt install ./NVPAIR-Setup-*.deb.
Get one machine working before you pair anything. Open the app, wait for Overview to show the machine, then use Engine settings on the node card to install Ollama or LM Studio. PAIR downloads and configures the engine for you, so nothing needs to be in place first. Add a model. Send the curl above and confirm the job appears.
Then pair the second machine and repeat the engine and model steps there. Settings, Cluster, six-digit PIN. Both nodes need a compatible engine and the model; a node only becomes a routing candidate once it is actually running an engine that can serve the request.
Put the same model on both nodes. PAIR prefers nodes it knows already hold the model. Two nodes with different models is not a routed cluster, it is two servers behind one address, and requests will pile onto whichever one can answer.
Then measure the thing you actually care about. Fire your real multi-agent workload at it and watch queue depth, not tokens per second. The win from PAIR is that request two stops waiting. If your workload is one long single-threaded conversation, you will see nothing, correctly.
Read SECURITY.md before you do any of this on a network you share. PAIR runs local HTTP endpoints, does LAN discovery, bootstraps trust with a PIN, and maintains cluster networking. That is a meaningful attack surface on a coffee-shop network or a shared office VLAN, and the README says to read the doc first rather than pretending otherwise.
Honest limitations
The scheduler is genuinely primitive, and NVIDIA says so. From the roadmap section: PAIR ships a single scheduling policy combining queued work with a coarse, smoothed GPU-utilization signal. It does not consider GPU model, available memory, model warmness, or how expensive a request looks. NVIDIA's own conclusion is that this "still makes it a better fit for similar machines than a highly mixed cluster." Read that as written. If your cluster is a 4090 desktop and a five-year-old laptop, PAIR will happily send a heavy request to the laptop, and nothing in the current policy knows that was a bad idea.
Two engines only. Ollama and LM Studio. No vLLM, no llama.cpp server, no MLX server. If your local stack is anything else, PAIR does not route it.
Running PAIR does not mean an engine will run. The README is careful about this and it is easy to skim past. PAIR itself installs almost anywhere. Whether a given engine and model work on a given machine is between that engine and that machine, and PAIR will not tell you in advance.
v0.1.1, released 2026-08-28. Two releases exist and the second was a build-tooling fix. This is early. Treat it as something to evaluate, not something to depend on.
Privacy is conditional, not guaranteed. The README's phrasing is careful: prompts and responses "are intended to remain on the local network when every configured client, model source, engine, and node is local." Every one of those four conditions is on you. Point one node at a hosted model and the sentence stops applying.
The uninstall has a macOS trap. Dragging the app to the Trash leaves the privileged helper registered. There is an uninstaller inside the bundle at /Applications/PAIR.app/Contents/Resources/installer-tools/uninstall-macos.sh, and it is the supported path. Worth knowing before you install, not after.
Why the blockquote is the story
I have spent a lot of time this year reading READMEs that oversell, and the pattern is consistent: the limitation exists, the maintainer knows it exists, and it appears nowhere until an issue thread forces it out. That costs everyone. Users install a thing that cannot do what they came for, maintainers field the same question forty times, and the project's credibility takes the hit rather than the marketing.
NVIDIA put four negations in bold near the top of a page most people skim. It will cost them installs from everyone who wanted memory pooling. It will also mean the people who do install it want the thing it does, which is a better trade than it looks.
So go read the blockquote before you download anything. If "route each independent request to one node" describes a problem you have, PAIR is a signed installer and a six-digit PIN away, and thirty minutes is enough to know. If it does not, you saved yourself an evening and you now know to look at llama.cpp's RPC backend instead.
That is what a good README buys you, and it is rarer than it should be.
Sources: NVIDIA/Personal-AI-Router README · PAIR releases · llama.cpp RPC backend
Medium metadata
Title: NVIDIA's Personal AI Router Will Not Pool Your GPUs, and Saying So Out Loud Is the Best Thing in the README
Subtitle: PAIR routes whole requests across machines on your LAN. It refuses to shard a model, and the refusal is printed in a blockquote near the top. Here is when that is exactly what you want, and when you need llama.cpp's RPC backend instead
Tags: Local AI, NVIDIA, Ollama, AI Agents, Open Source
Suggested kicker: Concurrency and capacity are different problems. Most people asking for one are describing the other.