ThinkingBox Trains Agents by Grading the Database, Not the Transcript

How Microsoft turned a yes-or-no state check into a reinforcement-learning reward, and what it fixed

An agent can take forty correct-looking steps and still leave your database wrong. It calls the right tools, narrates a clean plan, tells you the task is done, and the record it was supposed to update says something else. If you have ever watched a tool-using agent confidently refund a customer who was not owed a refund, you know the gap between a good transcript and a good result. Microsoft just published a way to train that gap out, and the trick is almost rude in its simplicity.

Microsoft's research team spent the last few months staring at exactly that gap. Their first blog post, on October 3, carried the blunt title "The Agent Said It Was Done. The Database Disagreed," building on a benchmark paper they had put on arXiv back in August. On October 22 they followed it with something more useful than a complaint. They released ThinkingBox-Training, a reinforcement-learning setup, written up jointly with Hugging Face, that trains an agent by checking the database it leaves behind and rewarding it only when the final state is correct.

The idea is small enough to say in one line, which is usually the sign it is worth paying attention to. Stop grading the steps. Grade the result.

What ThinkingBox actually is

ThinkingBox started as a benchmark, not a trainer. It is a set of 507 tasks that each run against a real backend with a database and a set of tools the agent reaches through MCP (the Model Context Protocol, the standard way agents call external tools). A task might be a retail return, a bank transfer, a travel rebooking, an insurance claim, or a consulting workflow. The agent talks to a simulated user, uses the tools, and tries to get the job done.

Here is the part that matters. ThinkingBox does not score whether the agent called the "right" tools in the "right" order. It runs the task, lets the agent finish, and then looks at the database. In the team's words, the check compares "the backend record that needs to be correct, not the specific tool calls." If the records match what should be there, the task passes. If anything is wrong, missing, or extra, it fails.

That design choice is what makes the training trick possible. A check that returns a clean yes or no is also a reward function. The team saw it plainly: this can be a reward. So they took the same state check they used to grade the benchmark and wired it into reinforcement learning, where it returns a 1 when the final database is right and a 0 when it is not.

A small number of tasks need a little more. Some requirements, like whether a written message to the user was appropriate, cannot be read off the database. For those, a judge model (GPT-5 Chat) answers a narrow yes-or-no rubric question that feeds the same pass-or-fail check. The same model also plays the simulated user in every rollout, but as the user it never grades the agent. Most of the reward comes from the deterministic state check, not the judge.

The example that makes it click

The post's clearest case is a corporate hotel booking. A traveler asks to downgrade their room. Company policy says a downgrade needs manager approval first.

The base model, Qwen3.8-27B (an open-weight 27-billion-parameter model from Alibaba's Qwen family), changed the booking and issued refunds without getting approval in 15 of 20 attempts. It was helpful, fast, and wrong, because being helpful was the thing it optimized for. After reinforcement learning against the state check, the trained model preserved the booking and payment records in all 20 attempts, and matched every expected record in 19 of them.

Nobody wrote a rule that said "do not refund without approval." The database check encoded it. The policy lived in what the correct final state looked like, and training taught the model to produce that state.

The numbers, and the catch inside them

Across the five task domains, the trained Qwen3.8-27B improved its average pass@1 (the share of tasks it gets right on a single attempt) from 51.70% to 60.78%. The count of tasks it solved on all 20 of 20 tries, which the team treats as its measure of dependability, jumped from 38 to 107. Per domain, pass@1 rose everywhere: retail 64.03% to 67.91%, auto insurance 47.85% to 56.45%, travel 53.41% to 62.69%, neobank 47.88% to 62.40%, and consulting 45.69% to 54.51%. These are the team's own measurements on one reference run of one model, so read them as a direction, not a leaderboard.

Now the catch, which the team reports instead of hiding. The share of tasks the model could solve at least once in 20 tries, which the team calls pass@20 and treats as its breadth measure, actually fell, from 89.35% to 87.97%. On that breadth measure, auto insurance lost about seven points. The model got more repeatable on the tasks it could already sometimes do, and it got narrower on the long tail it rarely solved.

That is the real finding, and it is more honest than the headline. A binary reward is a strong teacher. It teaches the model to reliably clear the bar you set, which is wonderful until you notice it has stopped trying the problems the bar does not cover. The team's own phrase for the result is that the model "did not become broader. It became more repeatable."

How it decides which tasks to train on

One design detail is worth lifting out, because it is the kind of thing that decides whether your own RL run works or wastes a week of GPUs.

Reinforcement learning with a group-based method like GRPO (a training method that compares a batch of attempts at the same task against each other and pushes the model toward the ones that scored better) needs contrast. If the model always fails a task, there is nothing to reinforce. If it always passes, same problem. So before training, ThinkingBox probes the base model on each candidate task several times and estimates how often it succeeds, using a standard statistical smoother (a Jeffreys posterior) so a task tried a handful of times does not get a wildly overconfident score.

Then it buckets. Tasks the base model solves almost never (below 8% of the time) are dropped. Tasks it solves almost always (above 90%) are dropped. The middle, where the model sometimes wins and sometimes loses, is kept, because that is where the learning signal lives. The training pool for the full run was 187 tasks, held separate from the 507 benchmark tasks so the model is never tested on what it trained on.

If you take one transferable idea from this project, it is not the database trick. It is this: spend real effort picking tasks your model is genuinely uncertain about, and throw away the ones it already aces or always flunks. Most of your training budget should sit on the fence.

What it costs to run

This is a serious piece of infrastructure, and the requirements are not shy about it. The documented reference run trained Qwen3.8-27B with full-parameter GRPO for 50 steps, 18 groups of 8 rollouts per update, on three nodes of eight H100 GPUs each, at a learning rate of 1e-6. To run it for real you need Linux, Python 3.12, NVIDIA GPUs, a local copy of the model, the ThinkingBox backend services, endpoints for the simulated user and the judge, and a specific build of the training framework: VeRL v0.9.0 with three compatibility patches the repo ships.

You can read the machinery without any of that. Set up the environment with uv venv --python 3.12, install the patched framework with the repo's scripts/prepare_verl.py, and then run the launcher in dry-run mode:

DRY_RUN=true ./scripts/run_train.sh

That adds a --dry-run flag so the launcher prints every configuration override it would use and exits without starting Ray or touching a GPU. The data-preparation guide includes an inline Python snippet that hydrates the 39 public development tasks and asserts it got 39, so you can open one and see what a graded task looks like from the inside. Think of the dry run as reading the recipe. You will not be plating the dish on a laptop.

One honest caveat on the economics the post reports. It quotes a "cost per dependable task" that falls from $24.36 to $8.00, a 67% drop, which sounds like money saved. Read the definition before you repeat it: it is the estimated cost of running the whole 507-by-20 evaluation campaign divided by the number of tasks that pass all 20 times. The dollar figures are a comparison index for serving, which the team calls "not invoice or cloud bills." They also exclude the cost of training and of the difficulty probing. The model gets cheaper to trust per reliable task, not cheaper to build.

What this does and does not prove

ThinkingBox-Training is a real result with real limits, and the team states most of them. The 187-task training pool is private, so the public repo lets you exercise the interface but not reproduce the headline run. Every public benchmark task is a synthetic reconstruction of a business workflow, not a live production system. The results come from a single reference run on a single model, so nobody should treat 60.78% as a law of nature. And the transfer from the private training tasks to the public benchmark is out-of-distribution by design, though the two share the same framework, simulated user, grading machinery, and task style, which softens the claim in both directions.

What it does show is narrower and more useful than a benchmark score. If the thing your agent produces can be checked by a deterministic rule, you can train against that rule, and a mid-size open model will learn to produce the correct end state more reliably than it did before. That is a different and better lever than prompting the model to please be careful.

Put this into practice

You do not need a GPU cluster to use the idea. You need to find the check.

Start by writing down, for one task your agent does, what the world should look like when it is finished. Not the steps. The state. Which row changed, which field holds which value, which side effect should and should not have happened. If you can express that as a function that returns true or false by reading your own database or API, you have the same object ThinkingBox built its training on, and you can use it today as a test even if you never train anything. For the hotel case it is almost trivial: booking.status == 'downgraded' and refund_count == 0 and approval.exists(). One line. It encodes the policy the model kept breaking, and it either holds after the run or it does not.

Run that check as an evaluation first. Point your current agent at the task 20 times and count how often the final state is actually correct, not how often the agent said it was done. The two numbers are usually not the same, and the distance between them is the problem you are actually trying to solve.

Only after you have a trustworthy check should you consider turning it into a reward. If you do, probe your base model's success rate on each candidate task and train on the middle band, the tasks it gets right sometimes, not the ones it always wins or always loses. And keep watching the long tail. The thing that will slip, without showing up in any single run, is the set of tasks your agent used to occasionally solve and now never attempts, because a binary reward pays it to be safe rather than broad.

An agent that tells you it finished is giving you a claim. A database that matches what should be there is giving you a fact. ThinkingBox is a bet that you should train on the fact. The bet mostly pays, and the place it costs you is exactly the place the team was honest enough to measure.

Sources: ThinkingBox-Training, Hugging Face post (October 22, 2026); microsoft/thinkingbox-training repository, reviewed at main on October 24, 2026; the first ThinkingBox post; benchmark paper arXiv:2608.19741. Benchmark and training figures are the project's own measurements on one reference run of Qwen3.8-27B.