Independent AI intelligence Two editions daily · ET
Fervor AI
Menu

Analysis · October 13, 2026 · repo

Janaslocal-aiagent-infrastructure

Janas Runs an 80B Model on a 32 GB Laptop by Treating Your SSD Like Memory

A small C project streams model "experts" from disk as they're needed. Here's how it works, what the numbers really say, and how to try it.

The rule for running AI models at home has been brutally simple. If the model file is bigger than your memory, it doesn't run, or it runs so slowly you give up.

That rule decides what hardware people buy. It's why "how much RAM?" is the first question in every local-AI thread, and why an 80-billion-parameter model has meant a workstation, not a laptop.

A new open-source project called Janas is bending that rule for one important family of models. Its README reports an 80B model, a 48.4 GB file, running and answering on a laptop with 32 GB of RAM. The trick is to stop treating the SSD as storage and start treating it as slow, very large memory.

The result is a different shopping question. For these models, how fast your disk reads may matter as much as how much RAM you have. That's worth understanding even if you never install Janas, because it changes how you size a machine for local AI.

First, why this is even possible

Janas only works this way with mixture-of-experts models, usually shortened to MoE. The name sounds technical. The idea isn't.

Picture a big law firm with 500 specialists. A client walks in with a tax question. The firm doesn't send all 500 lawyers into the meeting. A receptionist (the router) picks the few specialists who fit the question, and everyone else stays in their offices.

An MoE model works the same way. It's built from many small sub-networks called experts, and for each token (roughly a word or word-piece) a router picks only a handful of them. The model in Janas's headline result, Qwen3-Next-80B-A3B, has 80 billion parameters in total, but the "A3B" means only about 3 billion are active for any one token.

So you don't need the whole model in memory at once. You need the experts this token wants, right now.

How Janas uses that

Janas is written in C with "no dependencies beyond the C library," according to its README. Its first component, Janas-LLM, keeps the model's weights on disk. As each token is generated, "the experts each token needs are streamed from an NVMe SSD" (the fast kind of solid-state drive that plugs straight into the motherboard), and the experts used over and over stay in memory.

Back to the law firm. The specialists who get called into every other meeting move into offices next to reception. The rarely used ones stay in the building across the street. You walk over when you need them, which is slower, but it doesn't happen often.

In the headline test, Janas kept "the experts it uses most in a cache of about 19 GB" and read the rest from disk. That's how a 48.4 GB model fits into a 32 GB machine with room left for the operating system.

None of this is a brand-new idea. llama.cpp, the widely used open-source local inference engine, can also map a model from disk instead of loading it fully. What Janas adds is a design built entirely around expert streaming and caching, and that shows in its numbers.

What the numbers actually say

Here's the README's own comparison, measured October 9 to 11, 2026, against llama.cpp build 2b18470 (September 18, 2026) using its Vulkan GPU backend (Vulkan is a graphics interface that works across GPU brands), on the same files.

For the 80B model, which is bigger than the laptop's memory:

  • Reading a 1,024-token prompt: Janas 62 tokens per second, llama.cpp 16
  • Writing the answer: Janas 24 tokens per second, llama.cpp 2.8

That writing number is the one to notice. About 2.8 tokens per second is a few words a second, slow enough that most people quit. About 24 is faster than you can comfortably read.

For smaller MoE models that fit in memory, the gap shrinks. On Qwen3-30B-A3B, Janas wrote at 32 tokens per second against llama.cpp's 15. On Gemma-4-26B-A4B, 18 against 12. The README also admits an exception: "the 3-bit Qwen3-30B reads a prompt slower than llama.cpp."

Now the part that matters most for trust. All of these are the author's own measurements, on one machine: a Debian 13 laptop with an Intel Core Ultra 9 185H, its built-in Arc GPU, 32 GB of RAM and a 1 TB NVMe SSD reading at about 5.7 GB/s. The README says plainly that it was "Developed and run on one machine." The 80B llama.cpp run also used non-default options, which the README lists. Treat the numbers as a strong signal from one setup, not a benchmark you can quote for your hardware.

The shift: your disk is now part of the spec

Here's the idea worth taking away, even if you stick with llama.cpp or Ollama: for MoE models, disk speed is a memory spec.

For dense models, where every parameter works on every token, RAM is the wall. For MoE models, the wall moves. What you need is enough RAM for the hot experts plus an SSD fast enough to fetch the cold ones without stalling. The 5.7 GB/s drive in the test isn't exotic, but it's far faster than an older SATA SSD (usually around 0.5 GB/s) or a USB drive.

That changes buying advice. Someone choosing between more RAM and a faster NVMe drive for local MoE models may get more from the drive. A 32 GB laptop with a fast SSD might run models people assumed needed 64 GB. I'd want to see results from other machines before betting money on it, but it's the right question to start asking.

Why this doesn't work for every model

It's tempting to read the headline and assume any big model can now run from disk. It can't, and the reason makes the whole idea clearer.

A dense model, the older design where every parameter works on every token, has no cold experts to leave on the shelf. Each token needs all of the weights. Some rough math (mine, not the README's): a dense 70B model quantized to about 40 GB, streamed from a 5.7 GB/s drive, would need around seven seconds just to read its weights once, for every single token. That's a model you'd watch type one word every few seconds.

An MoE model with about 3 billion active parameters touches a small slice of that per token, and the cache means many of those reads never hit the disk at all. That's the whole game. The fewer experts a token uses, and the more often the same experts come back, the better disk streaming works.

So when you see a new open model, check two things in its name or model card before you assume Janas can help: whether it's MoE, and how many parameters are active per token.

Put this into practice

Janas ships prebuilt for x86-64 Linux (Ubuntu 22.04, Debian 12 and later) and Windows 10 (version 1903 or later) and 11. There's no Mac build. The current release is v2026.10.12.1, from October 12.

1. Check your disk first. On Linux, a quick read test tells you whether your SSD is in the right league:

sudo hdparm -t /dev/nvme0n1

Results in the low thousands of MB/s or higher are what the README's test machine had. If you see a few hundred, expect much slower results for models bigger than your RAM.

2. Download and run. From the README:

curl -LO https://github.com/prabanta-dev/janas/releases/latest/download/janas-linux-x86_64.tar.gz
tar xf janas-linux-x86_64.tar.gz && cd janas
./janas-chat

janas-chat lists the models it knows. Pick one and it's downloaded and checked. On Windows, unpack janas-windows-x86_64.zip and run .\janas-chat from the janas folder.

3. Start with a model that fits. Try Qwen3-30B-A3B or Gemma-4-26B-A4B before the 80B. You'll see whether Janas behaves on your hardware before you ask it to stream 48 GB.

4. Point your tools at it. janas-server offers "an HTTP API that follows OpenAI's," so tools that accept an OpenAI-style base URL can talk to it. Its serving docs give the default as http://127.0.0.1:8080/v1, with --port to change it and --api-key to require a key. Two catches: it implements only part of OpenAI's API (the rest answer with a 501 error that says why), and there's no HTTPS yet, so keep it on localhost unless you put something in front of it.

5. Bring your own models carefully. Janas reads its own .jns format, published on Hugging Face, and can convert other models from GGUF, the common local model format. Supported families include Qwen3, Qwen3.5, Qwen3.6, Qwen3-Next and Gemma 4, and the docs say models outside those families are refused. Prebuilt .jns downloads are the easy path.

What should you expect? A first run that spends time downloading, then a memory graph that climbs. Janas "fills the free memory with its expert cache" by design. Close your browser tabs before you test the big model.

Honest limitations

This is a project to experiment with, not to build a product on yet. The README says so in bold: "This is an experiment, not a product."

  • Stability isn't verified. Because it fills free memory, it can push "the desktop into swap and become slow or unresponsive." Using a GPU "has, once, caused a driver reset that took the desktop with it."
  • Hardware coverage is thin. It was tested on one Intel CPU and an integrated Arc GPU. Other CPUs, "especially without AVX-VNNI, or AMD," discrete GPUs and slower disks are untested. No macOS.
  • The interface moves. "The public API and the command line change without notice."
  • The license has reach. Janas is GPL version 3 or later, which matters if you plan to ship it inside other software. The README notes the models themselves aren't covered by that license.
  • It's small. About 210 GitHub stars as of October 13, 2026, and one author. If you hit a bug, you may be the first.

One more thing, and it's not a flaw so much as a cost. Streaming experts means constant reads from your SSD. Reads don't wear out drives the way writes do, but you're trading disk activity and heat for memory you don't have.

The question to ask before your next upgrade

Janas won't replace llama.cpp or Ollama for most people this month. It runs on fewer platforms, its numbers come from one laptop, and its author calls it an experiment.

What it does prove is the idea. For mixture-of-experts models, a fast SSD and a smart cache can stand in for a lot of RAM. That's useful to know whatever tool you run.

So before you spend money on memory for local AI, check your disk speed, look at whether the models you care about are MoE, and try the small Janas models on the machine you already have. Your results are worth more than the README's, and the project could use them.

Sources: prabanta-dev/janas README at v2026.10.12.1 · Janas serving docs · Janas release v2026.10.12.1 · Hacker News discussion. Version reviewed: v2026.10.12.1, October 13, 2026.

Medium metadata

Title: Janas Runs an 80B Model on a 32 GB Laptop by Treating Your SSD Like Memory

Alternates to test:

  • Your SSD Is Now Part of Your Local AI Spec
  • You Don't Need 64 GB of RAM to Run an 80B MoE Model
  • Janas Turns Your SSD Into Memory for 80B AI Models

Subtitle options:

  • A small C project streams model "experts" from disk as they're needed. How it works, what the numbers say, and how to try it. (picked)
  • Mixture-of-experts models change what limits local AI. Janas is the clearest demo yet.
  • The 80B model is bigger than the laptop's memory. It still writes 24 tokens a second.

Description: Janas streams mixture-of-experts weights from an NVMe SSD so an 80B model runs on a 32 GB laptop. How it works, the honest numbers, and how to try it.

Tags:

  1. Local LLM (the exact search intent)
  2. Artificial Intelligence (broad reach)
  3. Open Source (it's a GPL project readers can install)
  4. Machine Learning (MoE explanation draws learners)
  5. Hardware (the RAM versus SSD buying angle)

Article type: A (spike). A real repo and release with a clear payoff and install steps.

Publish plan: Post at 7 to 8 AM ET, let Medium distribute for 2 to 6 hours, and send to subscribers only if it is performing. Canonical import from the fervorai.dev URL.