Zero Data Retention Isn't a Guarantee. Test Your AI Provider With a Fake Key
Two million booby-trapped prompts found the privacy checkbox failing at resellers, breached hosts and one model lab. Here is how to run the same test on your own stack.
Every AI API sells you the same comfort: a setting that says your prompts will not be kept. Most teams tick it, write "ZDR enabled" in a security questionnaire, and stop thinking about where the text actually went.
That habit just got expensive. Over the past week, a security firm showed that prompts marked "do not store" were read anyway, a hosting company admitted an intruder downloaded stored prompt samples, and the biggest model router pulled 125 endpoints off its privacy-safe list in one go.
The checkbox was never a lock. It was a promise, and almost nobody was checking it. The good news is that you can, with the same trick the researchers used: a fake key that screams when someone touches it.
What Zero Data Retention is supposed to mean
Zero Data Retention, or ZDR, is a provider's commitment that it will not store your prompt or the model's response after the request finishes. Dragos, the industrial security firm behind this week's research, puts it plainly: ZDR "specifies that the prompt and response are not stored."
For builders, ZDR is what makes it feel safe to send customer emails, internal docs or support tickets to a model you do not host. Your agent pastes a contract into a prompt, the model summarizes it, and the contract vanishes. That is the theory.
The catch is in how requests travel. Many teams do not call a model maker directly. They call a router like OpenRouter, one API that forwards each request to whichever of dozens of hosts can serve that model. Some of those hosts resell capacity from other hosts. Your request can pass through two or three companies before a GPU sees it, and each one has its own logs, staff and credentials.
ZDR is a sentence in each company's policy. Nothing in the request itself enforces it.
What two million canary prompts found
On November 19, Dragos published "AI Supply Chain Security: What Canary Tokens Found", written by HD Moore. The method is simple enough to explain in one breath.
A canary token is bait: a fake AWS key or a unique web address that does nothing except alert you the moment anyone uses it. Think of a marked bill in a bank's cash drawer. Nobody should ever spend it, so if it shows up at a store, you know exactly which drawer leaked.
From July to November 2026, Dragos sent nearly two million prompts containing these marked bills to AI providers, then waited to see which ones got spent. The results, from the post:
- Data from at least three AI providers was exposed through breaches. In one, attackers stole a couple hundred AWS keys and used them for tens of thousands of AWS API calls.
- Marketplace resellers could read prompts through ordinary request logging, the kind of debugging feature most engineers would turn on without a second thought.
- Privacy settings, ZDR included, were not always honored. In two of the breaches, some stolen requests were marked ZDR. As Dragos writes, "A privacy flag doesn't help when the attacker is already inside."
The finding that should change how you think, though, involves no breach at all. Dragos reports that an unnamed Chinese AI lab's scraper fetched the secret URLs from roughly two-thirds of requests marked "no data collection" or ZDR. That is the model maker itself, reading prompts that were flagged as off-limits.
Dragos does not name every provider or the lab. That is frustrating if you want a blocklist, and it is also the point. You cannot outsource this to a list.
The leak that started with a coding agent
Two days later, Together AI, a large host for open-weight models (models whose trained weights anyone can download and run), published its own security event disclosure. CEO Vipul Ved Prakash wrote that an unauthorized party downloaded stored prompt samples for requests to MiniMax M3, GLM-5.2 and Kimi K2.6, "primarily on June 24, July 4, July 8, and July 14, 2026."
The root cause is the part every builder should read twice. An AI coding agent, working on a public repository called xorl, committed a credential. Together's automated scanning detected the exposed credential and the commit was reverted, but the intruder still got in, downloaded samples and started testing credentials found inside them. Five of those credentials were Dragos canaries.
Together says those samples were collected during routine software testing, "authorized for that use," so nothing in the post says they came from ZDR requests. The lesson is about the path, not the checkbox: prompts sat in storage, the storage was reachable with one leaked key, and the key leaked because an agent did what agents do all day, write files and commit them. Together now defaults new accounts to ZDR, checks every pull request for credentials and has new rules for coding agents. It has not published how many requests or customers were affected, and its review of personal data is still open.
The router pulled the label
On November 22, OpenRouter posted "Data Processing Incidents on Inference Providers". It off-boarded three providers it does not name, removed 125 endpoints (a specific model served by a specific host) from ZDR eligibility (some flagged, some as a precaution), notified potentially affected customers and said it will require assurances before restoring the label. It also says the findings "did not show any breach or unauthorized access of OpenRouter."
A customer email reposted on Hacker News adds a sharper detail: test URLs in prompts sent to MiniMax models were accessed by a third party "40 to 130 hours after the requests were sent," via hosts that forwarded to MiniMax. That email is secondhand, OpenRouter's public post does not repeat it, and MiniMax has not published its own account. Treat it as a lead, not a verdict.
The shift: ZDR is a claim, not a control
Here is the one idea worth keeping from all of this.
A control is something your system enforces: a firewall rule, a scoped key, a request that fails if it cannot go where you said. A claim is something another company tells you about itself. ZDR has always been a claim, made by each company about its own storage, and you had no way to check it.
That framing tells you what to do. You do not stop using hosted models. You stop treating the checkbox as proof, and you add two things that are controls: routing you pin yourself, and bait you plant yourself. The router decides who sees your request. The canary tells you if anyone misused it.
Put this into practice
None of this needs a security team. Start with the cheapest step and work down.
1. Find the secrets your agents already send. Search your prompt templates, tool outputs and agent logs for anything that looks like a key, token, internal URL or password. Agents love pasting whole config files and error dumps into context. Dragos's own advice includes filtering secrets out before prompts leave your network, and it is the step that would have made this week's stolen samples far less useful.
2. Pin the providers on any router. If you use OpenRouter, the provider object in each request lets you narrow who gets the call. Per its provider routing docs, data_collection: "deny" skips providers that may store data, only lists the provider slugs (OpenRouter's short names for each host) you allow, and allow_fallbacks: false stops it from trying a backup provider on its own when your first choice is down. The zdr: true flag has a catch worth knowing. OpenRouter's ZDR guide says it routes to ZDR endpoints "plus Safety Retention endpoints if your settings allow them," meaning endpoints that may keep prompts to detect abuse, and that self-serve accounts allow Safety Retention unless you choose otherwise. If you want the strict version, pick "ZDR only" in your privacy settings. A pinned request looks like this:
{
"model": "your/model-id",
"messages": [{"role": "user", "content": "..."}],
"provider": {
"only": ["provider-slug-you-vetted"],
"zdr": true,
"data_collection": "deny",
"allow_fallbacks": false
}
}
Expect more failed requests when that host is busy. That is the trade, and for customer data it is a good one. Remember that zdr: true still relies on the same label OpenRouter just stripped from 125 endpoints, so pair it with step 3.
3. Plant your own canaries. This is the Dragos method at a scale of one. Canarytokens, from Thinkst, will generate a pair of fake AWS keys: choose "AWS Keys," add a comment so you remember where you planted them, and you get an alert if anyone ever uses them. The docs warn alerts can lag "between 2 and 30 minutes" because they pass through Amazon's logging first. Put a different token in a test prompt for each provider and route you depend on, label each one, and send a few requests a week. Use only these fake keys, never a real secret, and only on accounts you own. If a token fires, you know which path leaked: turn off that route, rotate any real credentials that ever traveled it, save the alert details and tell the provider.
4. Scan what your coding agents commit. Together's leak began with an agent commit. Run a secret scanner on every pull request, including the ones an agent opened, and give agents short-lived, narrowly scoped credentials so a leaked one buys an attacker as little as possible.
5. Write down the real policy. For each workflow, note which provider sees the data and what you checked. "ZDR enabled" is not an answer for a security review anymore. "Pinned to one host, canary planted, no alerts since October" is.
Honest limitations
A silent canary proves less than it feels like. It tells you nobody has used the bait yet. It cannot tell you whether a prompt sits in a log somewhere, waiting.
Pinning helps only as far as the host you pin. If that host forwards internally, or gets breached like the three in the Dragos study, your routing choice will not save you.
Some of this week's story is still unconfirmed. Dragos does not name the lab or every provider, the MiniMax detail comes from a reposted email, and Together has not published counts. Expect these facts to shift as more disclosures arrive.
The strongest fix, running models on hardware you control, is real but costly. Dragos lists self-hosting first among its suggestions with an "if you can afford it" attached, and for most small teams that caveat does a lot of work.
Where this leaves you
The privacy checkbox will not go away, and it should not. It is still better than a provider that admits it keeps everything. But this week moved it from "safe by default" to "a claim to verify."
You can verify it this afternoon. Pick the one workflow that sends the most sensitive data to a model, pin its provider, and plant a fake key in a test prompt. Then you will know something about your stack that, until this week, almost nobody knew about theirs.
Sources: Dragos, "AI Supply Chain Security: What Canary Tokens Found" (Nov 19, 2026); Together AI security event disclosure (Nov 21, 2026); OpenRouter, "Data Processing Incidents on Inference Providers" (Nov 22, 2026); Hacker News repost of an OpenRouter customer email (Nov 21, 2026); OpenRouter provider selection docs; OpenRouter ZDR guide; Canarytokens AWS keys guide.