That Leaked AI Model on Hugging Face Was Partly Random Noise. Here's How to Check One
A 7.7 TB "Anthropic checkpoint" drew thousands of downloads. The weights were never the dangerous part.
A model file is a strange thing to trust. It is terabytes of numbers nobody can read, wrapped in a description anybody can write. When the description says "secret frontier model," people download first and ask questions later.
That is what happened with a repo called numbat on Hugging Face, the site where most open AI models are shared. Its card claims to be a checkpoint (a saved snapshot of a model's weights) "pulled mid-pretraining" from "Anthropic's next Mythos-class training run." It showed 5,846 downloads in the past month as of November 1.
Then someone checked the numbers, and every block they tested turned out to be seeded random noise.
The result is a useful lesson that has little to do with Anthropic. A frontier label travels fast with zero proof behind it, and the risk in "just trying it" sits in a single loading flag, not in the 7.7 terabytes. This piece walks through what the audit found, why the loader matters more than the weights, and a short routine you can run before you touch any model that arrives with a story attached.
What the repo claims
The numbat repo, uploaded by an account named burner22, describes a model of "~14T param, 114 layers, hybrid GDN/MLA, MoE (1,224 experts, top-48 + 1 shared), NVFP4." GDN and MLA are architecture labels you can skip. In plain words: a giant mixture-of-experts model (one that routes each word through a small subset of many specialist sub-networks), stored in a compressed 4-bit number format.
Even the size is fuzzy. The card's body says about 14 trillion parameters, while Hugging Face's automatic header for the repo says 6.8 trillion. The packed 4-bit format may explain part of that gap, since the two numbers may be counted differently, but nothing on the page reconciles them. The files total roughly 7.72 TB across 772 shards (the weights are split into 772 separate files).
No license is listed, which means you have no stated permission to use or redistribute it at all. Its answer to "How did you get this?" is "Can't say more than what's already in the repo." Its answer to "Is this real?" is "Yes." The only link to Anthropic is the codename itself, which the card says appeared in an earlier Claude Code source leak, with no source you can check. Anthropic has not commented that ExplainX could find.
None of that proves anything either way. It just means the whole case for the model is the uploader's say-so.
What the audit found
In the repo's discussion #13, a user named FloxUnder posted a short, careful test.
The idea is simple. When researchers need placeholder numbers, they often generate them with a random number generator, and a random generator given the same starting value (a seed) produces the same numbers every time. So FloxUnder guessed a recipe: take each tensor's name (a tensor is one named block of weights inside the model), hash it, use the first eight bytes of that hash as a seed for NumPy's generator, then draw numbers and apply a simple scaling.
If the weights were trained, that recipe should match nothing. If they were generated, it would match exactly.
It matched exactly. Forty-three complete tensors, 730,368 values, every single one. The auditor then predicted 21 tensors nobody had read yet, and those matched too when they were fetched.
Two details make this audit worth copying:
- It downloaded almost nothing. The test used HTTP range reads, which ask the server for a small slice of a file instead of the whole thing. The reproduction script pulls about 90 KB from a 7.7 TB repo.
- It ran none of the repo's code. Every check happened on raw bytes, against one pinned revision of the repo (commit
6b7f8fc), so anyone can rerun it on the same files.
The auditor was just as careful about the limits. The post says "I have not checked every tensor or reproduced full-model inference," notes that the matches "cannot logically exclude a training setup that deliberately kept all these parameters frozen," and adds that they "do not authenticate the claimed Anthropic origin."
So the fair summary is this: a sampled set of the model's weights is provably generated noise, the rest is unchecked, and nothing in the repo ties it to Anthropic. On Hacker News, one commenter who reproduced the audit went further and called the matrices "literally just random data." That goes beyond what the audit itself claims, so treat it as one person's read.
The part that can actually hurt you
Here is the turn. Even if every tensor were random, downloading random numbers is mostly a waste of bandwidth. The weights in this repo use safetensors, a file format Hugging Face describes as "a new simple format for storing tensors safely (as opposed to pickle)" in its own docs. A safetensors file is a header plus raw numbers. Opening it does not run code. That protection covers the weight files only, not anything else that ships beside them.
The older format is the problem child. PyTorch's default pickle files can run code while they load, and Hugging Face's pickle security page says plainly: "There are dangerous arbitrary code execution attacks that can be perpetrated when you load a pickle file."
numbat avoids pickle. But it ships two Python files, configuration_numbat.py and modeling_numbat.py, and its card's loading example sets trust_remote_code=True.
That flag is the real story. Think of it as the difference between receiving a box of parts and receiving a box with a robot inside that starts working the moment you open it. Without the flag, the Transformers library (the Python package most people use to load these models) builds the model from code already on your machine. With it, the library downloads the Python files from the repo and runs them, with every permission your user account has: your SSH keys, your cloud credentials, your API tokens.
Hugging Face's Transformers documentation says to "Take extra precaution when loading a custom model," and notes that while "the Hub includes malware scanning for every repository," you "should still be careful to avoid inadvertently executing malicious code." Its pickle page adds a sentence worth taping to a monitor: "It is your responsibility as a user to check if something is safe or not."
Nobody has reported malicious code in numbat's Python files. That is not the point. A repo with an exciting label, a fake-looking pile of weights, an anonymous uploader and a loader that runs whatever Python it ships is the shape every model-hub lure will take. The weights are the bait. The loader is the hook.
Why frontier labels travel this far
There is a simple reason a 7.7 TB file with no license, no named author and no proof collected thousands of downloads in a month. Model weights are the one artifact in software that people cannot review by reading.
With a library, you can skim the source. With a model, the "source" is billions of numbers. So people fall back on the label, the size, the architecture buzzwords and the download count. Every one of those is something the uploader controls or something that grows simply because other people trusted the label first.
The sensible response is to stop treating model files as opaque. As the audit showed, you can learn a lot from a few kilobytes and zero code execution.
Put this into practice
Here is a routine for any model that arrives with a story, whether it is a "leak," a surprise mirror of a famous model, or a fine-tune from an account you have never heard of. Most of it takes minutes.
1. Read the file list before anything else. On the repo's "Files and versions" tab, look for two things. Weights in .safetensors are the safer format. Treat .pkl as pickle, and .bin, .pt or .pth as probably pickle (that is the PyTorch convention, though the extension alone does not prove it); the Hub's scan results on each file page help here. Then look for .py files. Python in a model repo is a warning sign that the model's configuration may point at custom code, which only loads with trust_remote_code=True.
2. Read the Python files in the browser. The Hub shows them as plain text. You are looking for anything a model definition has no business doing: network calls, subprocess, os.system, reading files outside the model folder, or encoded blobs that get decoded and run. If you cannot tell what a file does, do not run it.
3. Avoid the flag, or pin it. Leave trust_remote_code off unless you have read the code. If you must turn it on, pin the exact commit you reviewed so a later push cannot swap the code under you. The Transformers docs show the pattern:
model = AutoModelForImageClassification.from_pretrained(
"sgugger/custom-resnet50d", trust_remote_code=True, revision=commit_hash
)
The commit hash comes from the repo's commit history.
4. Check the card against itself. Does the parameter count in the metadata match the prose, or at least get explained? Is there a license? Does the "evidence" link anywhere you can verify? numbat had no license and no checkable evidence before anyone opened a tensor.
5. Read the Discussions tab. On numbat, the most useful page in the whole repo was a community thread, not the card.
6. Sample the weights instead of downloading them. The audit's method generalizes. Range reads and the safetensors header let you inspect tensor names, shapes and sample values for a few kilobytes. If a "trained" model's normalization weights look like textbook random draws, you have your answer before spending a weekend of bandwidth. The discussion #13 write-up describes the full recipe.
7. If you still want to run it, run it somewhere disposable. A throwaway cloud VM or container with no saved credentials, no SSH keys, no mounted home folder and limited network access. Pinning the repo does not pin the Python packages it installs, so pin those versions too. If the code does something bad, it does it to an empty room.
Expect step 1 and step 4 to settle most cases. Real releases from known labs usually come from the lab's own organization account, with licenses, model cards that agree with themselves, and usually a paper or a blog post you can find on the lab's own site.
Honest limitations
This routine catches the lazy and the obvious. It will not catch everything.
Reading Python by eye misses well-hidden payloads, and many builders are not in a position to audit a few hundred lines of model code. Pinning a revision protects you from the code changing later. It does nothing about code that was malicious from the first commit.
Hugging Face's scanners help, but the company itself calls its pickle scanner "not 100% foolproof" and says its antivirus scan "doesn't cover pickle exploits," per the same security page.
The audit has limits too. It covers 43 tensors from a 772-shard model. It proves those tensors were generated, not that every byte is noise, and not who made the repo or why. "This is fake" and "this is malware" are different claims, and only the first has evidence behind it here.
And the sampling trick in step 6 needs some comfort with Python and byte offsets. If that is not you, steps 1 through 5 still do most of the work.
The label is not the provenance
The numbat episode will fade, but the pattern will not. Every big model launch now spawns mirrors, "uncensored" variants, "leaked" checkpoints and suspiciously fast fine-tunes, and each one asks for the same thing: your trust, plus one flag that turns a download into code execution.
You get to decide what earns that flag. A verified account, a license, a card that agrees with itself and Python you have actually read is a reasonable bar. A codename and the word "Yes" is not.
Sources: burner22/numbat model card, numbat discussion #13 audit, ExplainX analysis, Hugging Face pickle security docs, Transformers custom model loading docs, safetensors docs. Repo state as of November 1, 2026.