Cloudflare's GLM-5.3 Leak Shows Your KV Cache Is a Security Boundary
A rejected request came back HTTP 200 carrying someone else's data. The check that would have caught it takes five minutes.
The scariest line in Cloudflare's latest postmortem is six words long: "a rejected request reported itself as live."
Picture what that means from the outside. You send a very long prompt to a hosted model. Somewhere inside the serving engine, your request gets turned away because it does not fit. Nobody tells you. The engine keeps going, hands your request a stretch of memory that still holds pieces of a stranger's conversation, and the model writes its answer from that. You get HTTP 200, a normal streaming response, and text that was never yours.
That is roughly what happened on Cloudflare Workers AI between August 27 and September 9, 2026. Cloudflare published the full account on October 26, alongside a coordinated security post from the SGLang team, the open-source engine it was running on.
The result is a lesson for anyone serving models to more than one person: the memory that holds a prompt's working state is a wall between users, and the number that sizes it is a security setting. The fix starts with two numbers you can compare today: the context window you advertise and the memory pool that has to hold it.
What actually leaked
Cloudflare hosted GLM-5.3 (@cf/zai-org/glm-5.3) and found that 155 responses, all to prompts longer than 681,280 tokens, could contain content from other users' prompts and partial generations made to the same model shortly before. Cloudflare says "a small number of customers" were affected, that the requests looked like ordinary long-context use, and that it "found no evidence of deliberate exploitation." No other Workers AI model was affected.
A security researcher, Gadi Evron of Knostic, reported it through Cloudflare's HackerOne program on September 8. Cloudflare pulled the model the next day and had it back by evening with a fixed configuration and a patched engine. The full timeline is unusually specific and worth a read.
The part worth your time is how two boring mistakes combined into a data leak.
The KV cache, in plain terms
When a model reads your prompt, it does a lot of math on every token and saves the results so it does not have to redo them while it writes the answer. That saved working state is the KV cache (short for "key-value," the two kinds of numbers it stores). It lives in GPU memory, and it is big: a million-token prompt needs room for a million tokens' worth of it.
Serving engines like SGLang do not hand each request its own fresh memory. They keep one shared pool, carved into fixed-size pages, and lend pages out per request. Think of a stack of shared whiteboards in an office. You grab a few, write on them, hand them back. Nobody wipes them. In Cloudflare's words: "Pages are not currently zeroed between requests; the next request that gets a page simply overwrites it."
That is fine as long as you only ever read the boards you wrote on yourself. The whole design rests on that one rule.
Mistake one: the window was bigger than the room
Cloudflare's deployment advertised a context window of 1,048,576 tokens. The prefill KV cache pool, the memory that holds a prompt while the model first reads it, held 681,280.
So a prompt could be legal by the published limit and still not fit in memory. Cloudflare says it "expected the engine to return an error for any prompt it could not process." It even tested long contexts before launch. Those tests showed "successful HTTP status codes and a streaming body with no error frames," so, as the post puts it, "we did not believe this to be an issue."
Read that again if you run evals. A test that only checks for a 200 cannot tell a real answer from a confident one built on someone else's data.
Mistake two: a queue that asked the wrong question
The second piece was a bug in SGLang's disaggregated mode. Disaggregated serving (often written "PD disaggregation," for prefill and decode) splits the work across two sets of machines: one reads the prompt and builds the KV cache, the other generates the answer, and the cache gets shipped between them over a fast network link.
When a prompt did not fit, SGLang marked it rejected and "swapped the prompt for a one-token placeholder." It should have stopped there. It did not, because "the admission queues on both prefill and decode checked whether a request was finished, not whether it was rejected." A rejected request is not finished. So it rolled on through the transfer, and decode read whatever was sitting on the pages it was given.
The bug was not secret. Cloudflare notes that another user had reported it upstream as SGLang issue #30233 on July 6, about seven weeks before GLM-5.3 launched on Workers AI.
The SGLang post adds a second, related problem in the same mode: if a request is aborted mid-transfer, late network writes can land in cache slots already reassigned to someone else. "The transfer reported success, and no component raised an exception or logged an error," the SGLang team writes.
Two different paths. Same failure shape. Something said "fine" and was wrong.
The shift: capacity numbers are security numbers
Most teams treat context length and KV pool size as performance knobs. You tune them for throughput, for cost per token, for how many requests fit on a GPU. Nobody puts them in a threat model.
This incident moves them there. A mismatch between what you advertise and what memory can hold used to mean a slow request or a crash. Combined with an engine that fails open, it meant one customer reading another's prompt. And the only symptom at the edge was a perfectly normal 200.
There is a broader habit here too. Every layer in this story trusted a status signal: the tests trusted 200, the queue trusted "not finished," the transfer trusted "success." None of those signals measured what people assumed they measured. If you serve models, the useful question is no longer "did it return OK?" It is "what would OK look like if this had gone wrong?"
Who should care, and who can relax a little
Be precise about scope, because the SGLang team is. The bugs live in PD disaggregation with a KV transfer backend, the library that ships cache data between machines (Mooncake, NIXL, MORI or a custom one). SGLang says a single-instance deployment "without this cross-engine transfer path is not affected by this specific race," and that its single-node rejection path "returned an error instead."
So if you run one SGLang server on one box for your team, this exact bug is not yours. The configuration lesson still is. Any engine, in any mode, that advertises more context than its memory holds is betting on a rejection path you have probably never watched fire.
If you run disaggregated serving, especially for more than one tenant, this is a this-week item. SGLang says default configurations were affected "from the v0.4.x line (early 2025)" and that the fixes were not backported to older release branches.
Put this into practice
Here is the short version of what Cloudflare and SGLang recommend, translated into steps you can run.
1. Put the two numbers side by side. On every prefill and decode node, find the KV pool size SGLang selected (Cloudflare says SGLang prints it in its startup logs and exposes it as a metric) and compare it with the context length you advertise. Cloudflare's rule is blunt: "If the pool is smaller, lower the context length to fit." Redo this after any change to weights, quantization, parallelism or memory settings, since each one can shrink the pool.
2. Upgrade. Cloudflare says to run SGLang v0.5.22 or later, which includes the admission fixes (#36988 and #38935). The SGLang post lists one more fix, for failed NIXL and MORI sends, in v0.5.23. If you use either of those transfer backends, aim for v0.5.23.
3. Hold freed memory until the network is done with it. SGLang calls this deferred KV release: decode keeps an aborted request's slots until prefill confirms its writes have drained. Turn it on for both roles. On v0.5.19 and v0.5.20 it needs SGLANG_DISAGGREGATION_DEFERRED_DECODE_KV_RELEASE=1; treat that as a stopgap until you can upgrade. If you use Mooncake, cap transfer size with SGLANG_MOONCAKE_MAX_TRANSFER_BATCH_INDICES (the default of 0 means no cap).
4. Turn on corruption detection where your model supports it. Pass --disaggregation-enable-kv-checksum to both prefill and decode; it is off by default. Cloudflare also mentions SGLang's --kv-canary, which detects and logs potential corruption. Then check that requests are actually being verified, which is the step people skip.
5. Prove the rejection path works. This one is my addition, and it is the cheapest test on the list. Send a prompt slightly larger than your pool and confirm you get an error, not a 200. Cloudflare's own advice says the same thing in fewer words: "do not assume the engine will reject what does not fit." If you serve several tenants, put a unique marker string in one tenant's long prompt and search other tenants' responses for it during load tests. Cloudflare ran its own canary-marker tests before it restored the model.
6. Stop generating aborts you don't need. Cloudflare changed its load balancer so client disconnects no longer abort requests mid-transfer, and recommends you "allow requests to proceed up to the first token." Fewer aborts means fewer chances to hit the race.
Expect step 1 to take minutes and step 5 to take an afternoon the first time. Expect step 2 to be the slow one if you pin engine versions per model.
Honest limitations
A few things this incident does not tell you.
It does not tell you that vLLM or other engines have the same bug. Neither post mentions them. The general lesson about advertised context and pool size applies anywhere; the specific bug is SGLang's.
It does not give you a ready-made log line to grep. Cloudflare says the pool size is printed at startup and exposed as a metric, but neither post quotes the exact field, so look at your own startup output rather than trusting a snippet from the internet.
The checksum and canary flags catch some kinds of corruption, where they are supported, and Cloudflare says they "do not yet cover every model architecture." It is still working with SGLang to extend them to GLM 5.3, GLM 5.3 Flash and DeepSeek V4. They also do not prevent a misconfigured pool. Step 1 is still the control that would have stopped this.
And Cloudflare's own hardening is partly in progress. It describes work on tagging KV pages and checking them at read time ("integrity by construction") and on formally verifying its KV transfer protocol. Those are plans, not shipped features.
What to do with this
You probably do not run Workers AI. You might run SGLang, vLLM or something homegrown for a team, a product or a few customers. Pull up two numbers today: the context length you promise and the memory you actually have to keep that promise. If the first is bigger, you have found your weekend.
And the next time a long-context test comes back green, ask what the green light checked.
Sources: Cloudflare, "Cross-request KV cache exposure in the GLM-5.3 Workers AI model, and our remediation with SGLang" (October 26, 2026); SGLang Team, "SGLang Security Update: Improving Request Isolation" (October 26, 2026). SGLang versions as described in both posts as of October 27, 2026.