MAI-Cyber-1-Flash Scored 95.95% on CyberGym. The Model Didn't.
Microsoft's first cybersecurity model shipped with a benchmark number that belongs to the system around it, and reading the label on that bar chart is the whole lesson.
Look at the bar chart on Microsoft's launch post for MAI-Cyber-1-Flash and the top bar reads 95.95%, with four competing systems clustered between 83.2% and 85.6%. Now read the label on that top bar. It does not say MAI-Cyber-1-Flash. It says "MDASH: MAI-Cyber-1-Flash + GPT-5.4."
That distinction is doing more work than the twelve-point lead.
MAI-Cyber-1-Flash on its own, measured in a lightweight terminal harness, scores 0.314 on CVEBench, 0.553 on CyberSecEval4 threat intel, 0.33 on malware analysis, and a flat zero across all three ExploitGym categories. The zeros are on purpose. Microsoft's team says the model was trained to patch bugs and not to write exploits, which is a defensible design choice for a defender-only product. The rest of those numbers are just modest. They belong to a 137-billion-parameter sparse mixture of experts with five billion active per token, and a 5B-active model is not going to top a leaderboard by itself.
So where does 96% come from? Microsoft answers this directly, in the post, in a sentence most coverage skipped: the research team says that replacing 80% of the existing models inside MDASH moved the harness from 88.45% in May to 95.95% in July.
The number is a property of the system. The headline is a model name. Those two things are not the same, and the gap between them is where your evaluation budget should go.
What the benchmark actually asks for
CyberGym comes out of Dawn Song's group at UC Berkeley, and it is a good benchmark, which is exactly why it is worth being careful with. It holds 1,507 instances built from real vulnerabilities found and patched across 188 OSS-Fuzz projects. An agent gets a written description of a vulnerability and the codebase at its pre-patch commit, then has to write a proof of concept that crashes the vulnerable build and does not crash the patched one. Success is mechanical, not judged. Either the crash reproduces or it doesn't.
That configuration is Level 1, and Microsoft evaluated at Level 1, which is also what the public leaderboard ranks. The benchmark defines four levels by how much information the agent gets. Level 0 removes the text description, and the paper reports that only 3.5% of instances could be reproduced without it. Levels 2 and 3 hand over a stack trace and then the ground-truth patch, and success climbs accordingly.
None of that makes Level 1 easy. It does mean "95.95% on CyberGym" is shorthand for "95.95% on the configuration where the agent is told what it is looking for."
The harness is the product
MDASH stands for Microsoft Security multi-model agentic scanning harness, and it runs more than a hundred specialized agents through five stages: Prepare, Scan, Validate, Dedupe, Prove. Auditor agents raise findings. Debater agents argue about exploitability, and the disagreement between them is treated as signal rather than noise. The Prove stage runs the candidate input under AddressSanitizer for C and C++ targets, so a finding is either demonstrated or discarded.
Microsoft is unusually blunt about this. The post has a section headed "Three things matter today: Model. Data. Harness," and it explains all three. The data is the part nobody else can assemble: more than 100 trillion security signals a day across identity, endpoint, cloud, and network, plus decades of Microsoft Security Response Center history connecting attempted exploits to what actually got contained. The harness was built by the Autonomous Code Security team, which includes people from the DARPA AI Cyber Challenge winning Team Atlanta.
I read that section as a company telling you plainly what it is selling. It is not selling a checkpoint. The model isn't even available as a standalone endpoint.
The one piece of this that transfers to your work costs nothing to copy: the routing. MAI-Cyber-1-Flash handles up to 90% of MDASH's tasks, and the system escalates the hardest 10% to GPT-5.4. Microsoft reports that this cut cost by 50% against its own previous configuration of GPT-5.4 plus 5.4 mini plus 5.3 codex. A small specialized model carrying the routine volume with a frontier model on call for the hard tail is a pattern you can build this month with two API keys and a confidence threshold. It is the most reusable idea in the whole announcement, and it has nothing to do with owning a cyber model.
Put this into practice
Build the two-tier router before you shop for a security model. Pick your cheapest capable model as the default and your best model as the escalation path. Define the escalation trigger in code: a failed validation step, a self-reported low confidence, an input over some size, a task type you already know is hard. Then log the escalation rate. If your cheap tier is handling 90% of calls, you have Microsoft's cost structure without Microsoft's model. If it is handling 40%, you have learned something more useful than any vendor chart.
Ask four questions of every security-AI benchmark number you see this quarter. What exact configuration produced it, model plus harness plus tooling? At what difficulty setting? Is the scoring component purchasable and testable on its own? And is there a per-slice breakdown, or only an aggregate? A vendor that answers all four is worth more of your time than one with a taller bar.
Run your own retrospective instead of borrowing theirs. Microsoft reports that MDASH recovered 96% of 28 historical MSRC cases in clfs.sys and 100% of 7 in tcpip.sys over a five-year window. You can run the same shape of test at your own scale. Take twenty vulnerabilities your team already found and fixed, check out the pre-fix commits, and see how many your candidate tool reproduces. It is a weekend of work and the result is about your codebase, which is the only result that decides anything.
Separate the finding from the fixing in your evaluation. MDASH's Prove stage exists because a security finding without a reproducing input is a ticket someone has to triage by hand. When you evaluate any tool in this category, count only the findings that arrive with a working reproduction. That single filter will change your ranking more than model choice will.
What I don't trust yet
Every number here is Microsoft's, measured by Microsoft, on a system Microsoft operates. The model is gated and does not ship standalone, so no third party can reproduce the result or isolate the model's contribution. That is not an accusation. It is the standard state of vendor benchmarks, and it should set your confidence level accordingly.
The per-slice question is the one I most want answered. CyberGym's own authors found that agents succeeded roughly 10% of the time on instances whose ground-truth proof of concept ran longer than 100 bytes, and those instances are 65.7% of the benchmark. Their measurement used the agents available when the paper was written, so it is a snapshot rather than a ceiling. Still, a 95.95% aggregate implies near-total success on that hard majority, and Microsoft has not published the breakdown that would show it. Until someone does, the aggregate and the difficulty analysis sit next to each other unreconciled.
The retrospective CVE numbers are known-answer tests. Recovering 96% of cases that were already found, documented, and patched is a real signal about search quality and a weak signal about discovery on code nobody has looked at yet.
The 50% cost saving is measured against Microsoft's own prior configuration, not against whatever you are spending now. It tells you Microsoft made its own product cheaper. It tells you nothing about your bill.
And the benchmark itself looks like it is saturating for orchestration systems. Sakana AI reported 86.9% on CyberGym two days before Microsoft's announcement with a different orchestration model. When several unrelated systems crowd the high eighties and nineties on a benchmark whose published leaderboard sat in the low twenties for single agents a year ago, the interesting question stops being who is highest and starts being what the benchmark can still discriminate between.
There is also a source detail worth flagging. Microsoft's post describes MAI-Cyber-1-Flash as derived from the MAI-Thinking-1 lineage, while secondary coverage describes it as a cybersecurity fine-tune of MAI-Code-1-Flash. Both may be true of different stages of training. I would not build an argument on either until the model card settles it.
The part that survives the hype cycle
The shift worth internalizing is that the deployable unit in AI security stopped being a model some time before anyone updated the marketing. Microsoft shipped a hundred agents, a five-stage pipeline, a routing policy, a validation step that executes code, and a data history nobody can replicate, then put a model name in the headline because that is what the industry knows how to compare.
You cannot buy that data history. You can absolutely build the routing policy, and you can insist that the tools you evaluate prove their findings instead of listing them.
Go look at what your current security tooling reports and count how many of its findings arrive with something you can run. That number is your real baseline, and no vendor is going to hand it to you.
Sources: Microsoft AI, "Introducing MAI-Cyber-1-Flash inside MDASH"; CyberGym project site and leaderboard, UC Berkeley; CyberGym paper (arXiv 2506.02548); Microsoft Security Blog on MDASH, May 12 2026; MarkTechPost analysis of the release.