Your Research Agent Sounds Certain. A New Benchmark Checks If It's Right.

A live-web research test judges twelve models by the answer they reach, and the best one is wrong more often than not

A research agent is a confidence machine. Ask it a hard question that needs digging across a dozen pages, and it will come back with a clean paragraph, a few citations, and a tone that says the matter is settled. The writing is the easy part now. The part nobody shows you is whether the answer is correct.

A paper that went up on arXiv on October 19, Deep Search in Real-World Web Environments, and got picked up on Hugging Face's daily papers list this week, is built to separate those two things. It puts twelve frontier models on the live open web and builds every task around a single correct answer that can be verified. What the benchmark is designed to reward is reaching that one verified answer, not producing research that looks thorough.

The best model scored 39.0%. Human evaluation, over 1,000 independent attempts, scored 21.1%.

Both numbers are worth sitting with, because they say opposite-feeling things at the same time, and both are true. One says the machine is already better than a person at this. The other says the machine is still wrong most of the time.

What the benchmark actually measures

The test is called OpenDS-World, and it has 200 tasks. Each is built around multi-step web research: the agent has to search, interact with pages in a browser, and process evidence from several independent sources before it can land on an answer. Human verifiers checked every task to confirm the evidence is valid and the answer is genuinely unique, so there is one correct response and the agent either finds it or does not.

The key design decision is the same one showing up across agent tooling right now. Because every task is built around one human-verified answer, what the score turns on is whether the agent reaches that answer. You can take a messy, expensive, roundabout route through forty pages, and if you arrive at the verified answer, you are counted right. You can take an elegant three-step route and confidently return the wrong thing, and you are counted wrong. The abstract does not publish the full scoring rubric, so I am describing the design intent rather than claiming the route is ignored down to the last detail. The point stands either way: the answer is what the benchmark is built to reward.

All twelve models ran under the same tool environment and the same execution rules, so the comparison is about the models and their strategies, not about who got a fancier browser.

Reading 39 percent honestly

Thirty-nine percent sounds like a failing grade, and for a product that promises answers, it mostly is. The best available model, given real web access and real tools, gets fewer than two of every five hard research questions right. The other eleven do worse. If your users are treating a research agent's output as a finished answer, this is the number to tape to your monitor.

But the 39% figure sitting above the 21% human figure is easy to misread. The human baseline came from 1,000 independent attempts at these deliberately hard, multi-source tasks. People are not great at this either. Deep research across unfamiliar sources, with a demand for one verifiable answer, is genuinely difficult, and the measured human rate reflects that. So the best model's reported accuracy is higher than the reported human rate, and it is still wrong most of the time. The abstract does not give enough detail to call the two conditions perfectly comparable, so read it as two hard numbers side by side rather than a clean head-to-head.

The honest summary is narrow: the top agent posted the higher reported rate under these benchmark conditions, and it is still confidently wrong three times out of five. That is a useful assistant and a dangerous oracle, and which one it becomes depends on whether a human checks the answer.

Where the agents actually fail

The more useful part of the paper is not the score. It is where the models break, because that tells you what to watch in your own system.

The authors report that the models differ a lot from each other in how they research. They vary in which tools they reach for, how much searching they do, when they decide to stop, and whether they recover after a bad step. Two models with similar final scores can get there through completely different behavior, which means a benchmark average hides a lot.

The failures they name are specific: tool coordination and long-horizon search management. In plain terms, the models struggle to keep a multi-step search organized over many turns, and they struggle to use their tools in the right sequence without losing the thread. An agent will open a promising page, get distracted by a secondary lead, and never come back to close the first one. Or it will decide it has enough and stop, three sources short of the one that would have changed the answer.

Stopping is the quiet killer. An agent that stops too early returns a confident answer built on half the evidence. An agent that cannot stop burns cost and time wandering. Neither failure looks like a failure in the output. Both produce a clean paragraph. That is the trap: the research can be badly broken and the writing can be flawless, because the writing was never the hard part.

Picture the shape of it. You ask for the current board composition of a mid-size company that reorganized twice this year. The agent finds a press release naming four directors, writes a tidy answer, and stops. It never reaches the later filing where two of them resigned and three were added. The paragraph it hands you is well-sourced, fluent, and out of date by six months, and nothing in the output signals that it quit early. A human doing the same task badly at least tends to feel the uncertainty. The agent reports the stale four with the same even tone it would use for a fact it had triple-checked.

Why this is the right way to score

There is a reason outcome-only grading is spreading. The old habit was to judge a research agent on whether its process looked good: did it cite sources, did it reason step by step, did the chain of thought read well. All of that can be present while the answer is wrong, and a model that writes convincing reasoning is exactly the model most likely to fool a process-based grade.

Grading the verified answer refuses to be impressed by the performance. It asks the only question a user actually cares about, which is whether the thing is true, and it checks against a known-unique answer so there is no partial credit for sounding right. It is harder to build, because someone has to construct tasks with one verifiable answer and confirm the evidence, which is why a set of 200 hand-verified tasks is a contribution in itself and not a dataset you can scrape together in an afternoon. But it is the grade that matches what the tool is for.

The same instinct is showing up elsewhere in the same week, which is how you know it is a real shift rather than one paper's preference. Training setups that reward agents for the correct final state rather than the right-looking steps are landing at the same time. The field is converging on a simple discipline: judge the result you can check, not the account the agent gives of itself.

Put this into practice

If you build or rely on a research agent, this paper is a template for evaluating your own, and you do not need their 200 tasks to start.

Build a small set of questions where you already know the one correct answer and can point to the evidence for it. Twenty is enough to begin. Make them the kind that need several independent sources, because single-source questions do not test the thing that breaks. Then run your agent and score only the final answer against the known-correct one. Resist grading the process. If the answer is right, it passes, however ugly the path. If it is wrong, it fails, however elegant the reasoning.

Watch the two failure modes the paper names. Log how many distinct sources the agent actually consulted before it stopped, and you will usually find the wrong answers clustered among the runs that stopped early. Log when the agent abandons a lead and whether it ever returns, and you will see the coordination failures directly. These two signals, sources-before-stopping and lead-recovery, will tell you more than any single accuracy number. They are also cheap to capture, because both fall out of the tool-call log your agent already produces, so you can start measuring them this afternoon without rebuilding anything.

And set the product expectation to match the evidence. At the current state of the art, a research agent's output is a strong draft to be checked, not an answer to be trusted. If your interface presents it as a finished answer, you are shipping the 39% and hiding it. Present it as a lead with its sources exposed, make the human verification one click rather than an afterthought, and you turn a confidently-wrong oracle into a genuinely useful researcher.

The score that should change your mind is not 39 and not 21. It is the gap between how certain these agents sound and how often they are right. Until that gap closes, the most valuable thing you can build around a research agent is not a better prompt. It is a cheap, fast way for a person to check the answer.

Sources: Deep Search in Real-World Web Environments, arXiv:2610.30181, submitted October 19, 2026. Benchmark figures (200 tasks, 12 models, best 39.0%, human baseline 21.1% over 1,000 attempts) are from the paper's abstract as reviewed on October 24, 2026.