Independent AI intelligence Two editions daily · ET
Fervor AI
Menu

Analysis · November 13, 2026 · repo

vercel-labs/uiUI SDKWebMCPagent-infrastructureai-skillsagent-security

Vercel's UI SDK Lets AI Agents Use Your App by Name, Not by Screenshot

The labels a screen reader needs are the same handles an agent grabs. A new beta library turns accessibility into an agent API, with a permission layer on top.

Watch an AI agent use a web app and you will see something odd. It takes a screenshot, or dumps the page's structure, reads thousands of characters, guesses which div is the Save button, clicks, and then reads the whole page again to see whether anything happened. Every app gets treated like a stranger's house in the dark.

Screen readers have dealt with the same problem for decades. They cannot see the page either. They find controls by their accessible names ("Project name," "Save") and their roles ("button," "listbox"). When a developer skips the label, a blind user and an agent hit the same wall.

That overlap is the bet behind vercel-labs/ui, the number one repo on Trendshift's daily board on Friday morning. Vercel calls it UI SDK, and its pitch is short: "A library of behaviors for agents to build the next generation of user interfaces." Read the docs and a sharper idea shows up. In this library, the accessible name is the agent's API. Do the accessibility work and you get the agent work almost free.

What UI SDK actually is

Most component libraries hand you a finished button or dropdown with a look attached. UI SDK hands you behaviors: the invisible rules that make a control work for keyboards, screen readers and pointers. Per the README, "Keyboard navigation, focus, selection, dismissal, popups, drag and drop and the ARIA they need live in @ui-sdk/core." ARIA is the set of HTML attributes that tells assistive technology what a control is and what state it is in.

The behaviors index reads like a list of things every developer has hand-rolled badly at least once. createRovingFocus "Gives a group one Tab stop and moves focus with the arrow keys." createFocusRecovery "Restores focus when the focused item is removed." createTypeahead "Finds an item as someone types its name."

You combine behaviors into a primitive (a component's state, actions and parts), then render it in React, Vue, Svelte, Solid or Web Components. The components ship with no styles of their own, according to the bundled skill, with an optional default stylesheet.

The README is blunt about who writes the primitives: "Don't worry, you don't need to learn how to write this. Your agents will." The library ships an agent skill (a packaged set of instructions a coding agent loads) that walks it through reading the installed docs, picking behaviors and testing the result.

The docs show how small a primitive can be. Their example is a mute button: one piece of state (muted), one action (toggle), and a slot (one rendered part of the component, here the button itself) that sets aria-pressed from the state, so assistive technology hears "pressed" or "not pressed" as it flips. That last attribute is the detail people forget when they build a toggle by hand, and it is the detail an agent needs to know whether the sound is on.

That is the first half of the story: agents build the UI. The second half is more interesting.

Agents can also use the UI, without screenshots

UI SDK includes an agent bridge called createAgent. Per the Built for Agents docs, once the page renders, agent.snapshot() lists the components on it with their state and the actions they allow. A project-name field comes back as a small record:

{
  "ref": "c1",
  "component": "input",
  "label": "Project name",
  "value": "Apollo",
  "actions": ["setValue", "clear"]
}

Compare that with what a browser agent usually gets: a screenshot to squint at, or a page dump where the field is one node among hundreds. The record says what the thing is, what it holds, and what you may do to it. That is the whole contract.

The agent then acts by name:

await agent.act({
  steps: [{ ref: "Project name", action: "setValue", args: ["Atlas"] }],
})

No pixel hunting. No guessing which element is the input. The change runs through the component's own API and its change callback, the same path a person's typing would take.

Here is why the accessibility framing matters. The agent-ready checklist says "The bridge's label is the accessible name." The skill's rules say it in one line: "People using screen readers and agents both find controls by name." Skip the label and your field is unreachable for both.

Vercel published one benchmark for this, and it is worth reading with care. In a project-settings task (rename a project, update members and notifications, save), an agent using browser tools on a page built with another library took a median of 16 tool calls, read 39.8k characters, and ran 41 seconds at $0.073. The same task through the UI SDK bridge took 5 calls, 6.8k characters, 16 seconds and $0.032. Those are Vercel's own numbers: medians of three runs per approach, Sonnet through Claude Code, one task. Treat it as a direction, not a law.

The part security people will like

Giving an agent direct access to your UI's actions sounds alarming until you see the brakes. UI SDK has three layers.

An allow callback. Per the permissions docs, you pass allow to createAgent and return true or false for each requested action. It is opt-in: without it, the bridge accepts every supported action. It runs before the step changes anything. Return a promise and the bridge waits for a person's answer, up to 5 minutes by default.

Change callbacks know who did it. Every change reports details.source === "agent" when an agent made it, and details.cancel() refuses it. Your app can treat agent edits differently from human ones.

Three HTML attributes for the dangerous stuff. From the agent-ready checklist:

  • data-agent-confirm="<what it does>" on anything an agent must not do alone, like delete, pay or send. Without an allow callback, the agent's press simply fails with a message telling it to ask the person. With one, your callback receives the sentence as step.confirm, shows it to the person, and returns their answer. The library gates the action; the dialog is yours to build.
  • data-agent-secret on a field an agent may fill but never read. Password, card and one-time-code fields are secret by default.
  • data-agent-hidden on a region an agent should not see at all, such as recovery codes.

I like this design for one plain reason. The person writing the Delete button is the person who knows it is dangerous. Putting the guard on the button, in the markup, keeps that knowledge where it is created instead of in a policy file three folders away.

Put this into practice

UI SDK is in beta. The React package is 0.1.0-beta.1, first published to the npm registry on November 12, and it needs React 19 or newer. Start in a throwaway branch.

1. Install core and your framework's adapter. From the README:

npm install @ui-sdk/react @ui-sdk/core

Swap @ui-sdk/react for @ui-sdk/vue, @ui-sdk/svelte, @ui-sdk/solid or @ui-sdk/web as needed.

2. Give your coding agent the skill.

npx skills add vercel-labs/ui --skill ui-sdk

If your agent does not support skills, the skills page says the same file ships at node_modules/@ui-sdk/core/skills/ui-sdk/SKILL.md, matched to your installed version. Point the agent there.

3. Ask for one small component. The docs suggest a prompt shaped like this: "Use ui-sdk to build a five-star rating in React. Arrow keys should move between stars and choose a rating. Use our existing styles and test the keyboard behavior." Expect the agent to read the bundled docs, define a primitive, render it, and write tests. The skill tells it to report every command it ran with its exit code, so you get a paper trail.

4. Turn on the bridge and look at what an agent would see.

import { createAgent } from "@ui-sdk/react/agent"

const agent = createAgent({
  allow: ({ component, label, action }) =>
    component === "select" && label === "Region" && action === "setValue",
})

Call agent.snapshot() in the console. Any control missing from the list, or listed without a clear label, is a control a screen reader user is probably struggling with too. That one check is worth the install.

5. Mark the dangerous buttons before you connect anything real. Add data-agent-confirm to delete, pay and send. Write the sentence for the person who will be asked: "Deletes the project and all its files."

6. Optional: expose it to a browser's built-in agent. The WebMCP page shows registerTools(agent, { allow }), which publishes snapshot, act and query through the browser's document.modelContext API. WebMCP is the browser-side interface that lets an agent built into the browser call tools a page registers, much like an MCP server does for desktop agents. In a browser without that API, it does nothing.

Honest limitations

It is a beta, under beta terms. The README says the framework, APIs, docs and behavior "may change before general availability," under Vercel's public beta agreement. Do not ship this into a product you cannot refactor.

The bridge only sees UI SDK parts. The checklist is clear that snapshot() lists only UI SDK components. A plain <button> or a bare <input> is invisible to the agent. Adopting this means migrating controls, not sprinkling a script on an existing app.

Labels can collide. Per the checklist, a label matches every label that contains its words, and an exact match does not win. "Tip" and "Custom tip" collide. You will need to write labels with more care than most teams do today.

Agent changes skip your click handlers. Agent actions call the component's API, so checks you put in onKeyDown or onClick never run. Validation has to live in the change callbacks. Even minLength does not stop an agent-set value, because browsers only check it while a person types.

The benchmark is small and self-run. One task, three runs, one model, published by the vendor. It is a reasonable signal. Check it against your own app before you quote it in a planning doc.

The core reference is big. The skill warns that docs/core.md is "over 300 KB" and tells agents to search it by heading. Expect your agent to burn tokens on custom primitives.

The takeaway

For years, accessibility work has been sold as the right thing to do, and too often skipped as the expensive thing to do. UI SDK puts a second price tag on skipping it: agents cannot use what screen readers cannot find.

Whether or not you adopt this library, run that test on your own app this week. Pretend you can only see names and roles. Every control you cannot find is a bug for two kinds of users now.

Sources: vercel-labs/ui README; Built for Agents docs and benchmark; Permissions; WebMCP; ui-sdk skill and agent-ready checklist; behaviors index; @ui-sdk/react on npm. Reviewed at commit 3aa0973 (November 13, 2026), package 0.1.0-beta.1. License Apache-2.0; about 2.9k stars per shields.io on November 13, 2026.