Computer-Use Agents Should Learn the Job Once, Then Hand It to Code
CVS Health just open-sourced a kit that lets an AI agent figure out a messy workflow, then turns what it learned into ordinary code that runs without a model.
Computer-use agents have become good at the hardest part of automation: working out what to click on a screen nobody designed for robots. The strange part is what happens next. In most setups, the agent works it out again on the next run, and the one after that, paying for the same reasoning every time.
That is a lot of thinking spent on a solved problem.
On November 16, CVS Health published a different answer. Its CUA Agent Kit, released as part of OpenAI's healthcare computer-use announcement, uses an AI agent to discover a workflow and then hands the job to plain code. It is Apache-2.0 licensed. In the README's words, "It uses AI to discover the steps, then code to repeat them."
The result is a simple rule you can apply to almost any agent project: use the agent as the explorer, not the worker. Once it has found the path, write the path down as code you can read, test and diff.
Why this matters right now
Computer use means an AI model that operates software the way a person does, by looking at screenshots, clicking and typing. It matters most where there is no API (a programmable interface that lets one system talk to another directly). In healthcare, that is a lot of places. OpenAI's post walks through a prior authorization where the facts live in the medical record but still have to be retyped into an insurer's portal by hand.
The models are getting cheap enough to try. On HealthAdminBench, a Stanford benchmark of administrative tasks like prior authorizations and appeals, OpenAI reports that GPT-6 Astra earned 94.3% of the rubric points and GPT-6 Luna earned 91.8% "despite costing 98% less at 8.5 cents per task." Both ran at maximum reasoning effort, and these are OpenAI's own published results (OpenAI).
Eight and a half cents sounds small. Multiply it by the volume CVS describes, an agent handling the system steps for "tens of thousands of consults each day," and the per-run bill starts to matter. So does the variability. A model that reasons fresh each time can take a slightly different path each time, and in a clinical system, "slightly different" is exactly what an auditor does not want to hear.
The idea in one picture
Think about how a new hire learns a clunky internal tool. On day one, they poke around, ask questions and take notes. By week two, they follow a checklist without thinking. Nobody pays them to rediscover the checklist every morning.
The kit splits agent automation the same way, into two jobs that most setups blur together:
- Discovery. An agent figures out how to do the task from a plain-language description. This is where AI earns its keep, because screens are messy and instructions are vague.
- Execution. Something repeats the task, the same way, many times. This is where ordinary code wins, because it is cheap, fast and predictable.
The README puts the argument plainly: "discovering how to do something and doing it repeatedly are different problems." Once a workflow is understood, "asking a model to reason through the same steps on every run adds inference cost and variability to work that ordinary code can perform" (README).
That is the whole insight. The rest is plumbing.
How the kit actually works
There are three stages. Discovery and running are command-line steps; codifying is a request you make to Codex.
Discover. You write the workflow in plain language in a Python file, prompt.py. Then uv run cua-agent-discover launches an OpenAI computer-use agent that tries the task in a visible browser (or on a remote Windows desktop). Every run saves evidence: screenshots, an event log called events.jsonl, a run.json summary, and for browser runs a Playwright trace. A trace is a recording of every page state and action that you can replay later to see exactly what happened.
Codify. You ask Codex, OpenAI's coding agent, to run a skill that ships with the repo: Use $codify-playwright-workflow to codify the successful discovery runs. The skill reads the recordings and writes deterministic code. Deterministic just means the same input produces the same steps every time. For browsers, the output uses Playwright, the popular browser automation library. For Windows apps, it uses pywinauto, a Python library that drives Windows controls.
Run. uv run cua-agent-run executes the codified browser workflow (Windows has its own run command). The README is explicit: it "runs without model calls or an API key," and each run still saves its own evidence.
When the workflow breaks later, because a vendor moved a button, the README describes the loop: Codex uses the production traces to diagnose the issue and update the code, "with engineer review, testing, and verification."
The part worth stealing: the codify rules
The most useful file in the repo is not the agent. It is the codify skill, the instructions Codex follows when it turns recordings into code. Reading it, a few rules stood out because they apply to any agent pipeline, with or without this kit.
"Treat successful discovery runs as evidence, not as executable specifications." The agent's run is proof that a path exists. It is not the code. Codex has to understand why each step happened before it writes anything.
Do not trust a success message alone. The skill tells Codex to confirm success "from the final screenshot, URL, recorded assertion, or other observable evidence." An agent saying "done" does not count.
Prefer multiple successful runs. Comparing several runs separates the steps that matter from incidental waits, mouse coordinates and recovery moves.
"Do not invent selectors." A selector is how code finds a button or field on a page. If the recordings cannot prove a stable target, the skill says to ask for another discovery run instead of guessing.
Replace coordinates with meaning. Instead of "click at pixel 412, 300," the code looks for an element by its role and accessible name first (the label a screen reader would announce), then label, test ID, placeholder and stable text, with scoped CSS "as a last resort." That ordering is why the codified script can survive a layout change that would break a recorded macro.
Use real state changes, not sleeps. Wait for the next control or the final state to appear, rather than pausing a fixed number of seconds and hoping.
Each of these rules moves trust away from the agent's own account of what it did and toward something you can check. That is the shift worth carrying into your own projects.
When the agent should stay in the loop
Codifying everything would be the wrong lesson. The kit itself says it fits "workflows with reliable, repeatable steps." Some work does not have them.
OpenAI's post includes a good counterexample from Ambience: when a patient's record is ambiguous, its agent "stops and asks the clinician, then waits for approval before submitting." That is judgment, not repetition, and a script cannot make it. Abridge describes agents coordinating discharge barriers across medication access and home health, which changes case by case (OpenAI).
A practical way to sort your own workflows:
- Codify it when the steps are the same every time and the only hard part is finding them in a clunky UI. Data entry, document sending, closing out a referral in an EHR screen.
- Keep the agent when each case needs reading and deciding. Triage, ambiguous records, anything a person would escalate.
- Use both when a stable workflow has one judgment step in the middle. Let code run the fixed parts and put a human or an agent gate on the decision.
Put this into practice
You can try the full loop on your laptop with the sample app the repo includes. You need Python 3.12 or newer, uv (a fast Python package manager), and an OpenAI API key for the discovery step.
git clone https://github.com/cvs-health/cua-agent-kit.git
cd cua-agent-kit
uv sync
uv run playwright install chromium
Set the sample app's address and demo login in your environment or a .env file, exactly as the README shows:
DISCOVERY_BACKEND=browser
START_URL=http://127.0.0.1:8081
WORKFLOW_ACCOUNT_JSON='{"username":"demo","password":"demo-password"}'
Start the sample app in one terminal with uv run python sample_app/app.py, then run discovery in another with uv run cua-agent-discover. A browser opens and you can watch the agent work. The README suggests repeating discovery "until the workflow succeeds consistently." Recordings land in artifacts/discovery/<run-id>/.
Next, open Codex in the repo root and ask it to codify the successful runs with the $codify-playwright-workflow skill. Read the code it writes. Then run uv run cua-agent-run and confirm the workflow completes with no API key in play.
If you are not ready to adopt the kit, you can still steal the pattern this week:
- Pick one repetitive browser task your agent already does well.
- Save three successful runs with screenshots or traces.
- Write (or have your coding agent write) a Playwright script from that evidence, using role-based locators and explicit waits.
- Run the script, and keep the agent only for the cases where the script fails.
Honest limitations
Discovery is not free. The exploring step still calls an OpenAI model and needs an OPENAI_API_KEY. The savings only arrive once a workflow runs often enough to beat that up-front cost.
Codified scripts break when the UI changes. Meaning-based locators (role, label, visible text) survive small layout changes, not a redesign. The kit's answer is another Codex pass on the new traces, which means an engineer reviewing a diff whenever a vendor ships a new screen.
Windows takes more setup. Windows discovery runs against a remote VM with a server installed, and the README asks you to keep that desktop session open and free of competing input while automation runs.
The numbers are vendor numbers. The HealthAdminBench scores and the backlog story come from OpenAI's post. They are a reason to test, not a promise about your workflows.
Recordings can hold sensitive data. Discovery sends your prompt, the demo login and screenshots to the model endpoint, and the README warns that traces and other artifacts can keep credentials, cookies and visible data, with redaction on a best-effort basis. In a clinical setting, treat the artifacts/ folder like patient records. Nothing in the kit by itself makes a deployment HIPAA-compliant; OpenAI's post ties HIPAA workflows to eligible services covered by a Business Associate Agreement.
It is a starter kit. CVS calls it "a starter kit," not a platform. You own the prompts, the scheduling, the credential handling and the review process around it.
The real shift
The interesting thing about computer-use agents was never that they can click. It is that they can figure out where to click without anyone writing it down first.
CVS's kit takes that seriously and then stops paying for it twice. Let the agent explore. Keep its evidence. Turn the path into code a person can read, and save the model for the moments that need judgment.
Look at the agent runs you are paying for today. How many of them are rediscovering a path they already found yesterday?
Sources: CVS Health CUA Agent Kit README and codify-playwright-workflow skill (main branch, read November 16, 2026); OpenAI, "Advancing computer-use agents for healthcare teams" (November 16, 2026); HealthAdminBench.