Independent AI intelligence Two editions daily · ET
Fervor AI
Menu

Analysis · October 16, 2026 · concept

GitHub AI ScanGitHub Agentic AutofixCodeQLagent-securityagent-harnesscodex

GitHub's AI Scan Can't Block Your Merge. That's the Point.

AI Scan and agentic autofix went GA together. How GitHub wired them is a blueprint for every AI check in your pipeline.

Every team adding AI review to its pull requests hits the same wall. The bot leaves forty comments, three of them matter, and nobody can tell which three without reading all forty. So people stop reading.

GitHub just shipped two features that take a clear position on that problem. On October 15, AI Scan for pull requests and agentic autofix for code scanning alerts both reached general availability. One uses a model to find vulnerabilities. The other uses an agent to fix them.

And the company that sells Copilot wrote this sentence into the launch: "AI Scan findings are advisory."

That one line is the most useful design decision in AI tooling this month. The model gets to speak. Something boring and repeatable gets to decide. If you build agents, review bots, or any pipeline where a model's output could stop or ship something, this split is worth copying exactly.

What GitHub actually shipped

AI Scan is a second reviewer that runs inside GitHub code scanning. Code scanning already runs CodeQL, GitHub's static analyzer (a tool that reads code without running it and matches it against known vulnerability patterns). CodeQL is strong where it has rules, and it has gaps. AI Scan targets those gaps: the changelog lists PHP, Shell and Bash, Terraform, Dockerfiles, JSP for Java and Blazor for C#, "and more."

Findings show up on the pull request next to CodeQL's, tagged with an "AI" marker, often with an explanation and a suggested fix. Then come the limits. AI Scan findings do not fail the code scanning check. Rulesets (GitHub's branch protection rules) cannot use them to block a merge. The scan skips pull requests from forks and from Dependabot. It covers public repositories, and organization-owned private or internal repos need GitHub Code Security or Advanced Security. You do not need a Copilot license, and usage draws down Code Security AI Credits (GitHub's metered usage unit for these AI security features, with budgets admins can cap). GitHub does not say which model runs it.

None of this is new as a stance. When the feature entered preview on July 14, the changelog said "These findings are informational and won't block pull request merges." Three months of preview later, GitHub kept that line.

Agentic autofix works the other direction. When CodeQL raises an alert on your default branch, you can hand the alert to Copilot. The changelog describes the loop in one sentence: Copilot "explores relevant files, proposes a fix, and reruns CodeQL to confirm the fix closes the alert." If the first try fails, it tries again. When it passes, it opens a draft pull request with a summary, its reasoning, and the validation steps it took.

So one feature is a model that can only advise. The other is an agent that can act, but only after a non-AI tool signs off.

Spell check versus the compiler

Here is the mental model that makes this click.

A spell checker underlines a word in red. You can ignore it, and you often should, because it does not know your product name or your cousin's. A compiler is different. If the code does not compile, nothing ships, and nobody argues with it, because it gives the same answer every time you ask.

AI Scan is the spell checker. CodeQL is the compiler.

The reason comes down to repeatability. A static analyzer run twice on the same code returns the same alerts. A language model asked the same question twice, worded slightly differently, can return different answers. That is not a knock on any one model. It is how these systems behave, and it shows up even in narrow tests. In a trolley-problem experiment posted October 14, Zach Landes found that reordering how the options were described flipped one decision model's choice in 370 of 2,620 pairs, roughly one time in seven. He is careful to call the counts descriptive and warns against sweeping conclusions, and a moral dilemma is not a SQL injection. But the lesson transfers: a gate that changes its mind when you reorder a sentence is not a gate.

That is why GitHub put the model in front of the reviewer instead of in front of the merge button. A false positive from an advisory reviewer costs a minute of reading. A false positive from a blocking reviewer stops a release, and after the third time, someone turns the check off.

Why the CodeQL rerun is the clever part

Agentic autofix could have shipped as "the agent says it fixed it." Plenty of coding agents work that way, and the agent's confidence is the only evidence you get.

GitHub instead made the agent prove it against the same tool that raised the alert. The alert came from CodeQL. The fix only counts when CodeQL, run again, no longer raises it. The agent iterates when it needs to, and none of those attempts matter until the referee agrees.

Call it a verifier loop: the agent proposes, a deterministic check confirms, and only a confirmed result moves forward. You have seen it before if you have ever let an agent loop on a failing test until it goes green. The difference here is that the check is the one your security team already trusts.

The loop also has a blind spot, and it matters. CodeQL confirms the vulnerable pattern is gone. It does not confirm your code still does what it did yesterday. A fix that closes an injection alert by deleting the feature would pass. That is why the result lands as a draft pull request with its reasoning attached, so a person and your test suite still get a vote.

Put this into practice

You can try both features, and the pattern behind them, without rebuilding anything.

1. Turn on AI Scan for a public repo. If you maintain a public repository with Terraform, shell scripts or Dockerfiles, enable code scanning and then the AI Scan setting for the repo. GitHub also exposes the setting over REST, in public preview since September 10, at /repos/{owner}/{repo}/code-scanning/ai-scan, with an organization-level twin at /orgs/{org}/code-scanning/ai-scan. One rule to know: a repo setting does not override an organization that has AI Scan disabled. Expect "AI"-tagged findings on your next pull request that touches those file types.

2. Decide who reads the AI rows. This is the step teams skip. An advisory finding nobody owns is decoration. Pick a person or a rotation, and give them a rule as plain as "every AI finding gets a reply or a dismissal before merge." That keeps the signal from turning into the forty-comment problem.

3. Hand a batch of alerts to Copilot. If your org has Code Security or Advanced Security on GitHub Enterprise Cloud, an enterprise admin enables the "Agentic autofix" policy, an org or repo admin turns it on in Code Security settings, and a Copilot billing method has to exist. After that, an Assign to Copilot button replaces the old free Generate Fix option. You can select up to 25 alerts and get one combined pull request. To script it, the Update a code scanning alert endpoint takes an assignees list:

curl -L -X PATCH \
  -H "Accept: application/vnd.github+json" \
  -H "Authorization: Bearer $GITHUB_TOKEN" \
  -H "X-GitHub-Api-Version: 2026-03-10" \
  https://api.github.com/repos/OWNER/REPO/code-scanning/alerts/ALERT_NUMBER \
  -d '{"assignees":["copilot-swe-agent[bot]"]}'

The token needs write access to code scanning alerts (or the security_events scope on a classic token, or public_repo for public repos only). Expect a draft PR, not a merged one. Run your tests on it like any other.

4. Copy the split into your own agents. This is where the real value is, and it costs nothing. Go through any pipeline where a model's output can stop or ship something and ask one question of each step: if I ran this twice, would I get the same answer? If not, that step advises. Something repeatable decides. In practice that looks like:

  • An LLM reviewer comments on a PR; your test suite and linters gate it.
  • A coding agent says it fixed a bug; the failing test passing is the proof.
  • An agent classifies a support ticket as urgent; a rule on the customer's plan decides whether anyone gets paged.
  • An agent flags a leaked key; a check against the provider confirms the key is live. Cloudflare shipped this exact idea for its Gateway on October 15, letting a Worker check whether a detected credential is still active.

Honest limitations

It costs real money. AI Scan burns Code Security AI Credits, and agentic autofix burns those plus GitHub Actions minutes. Organization-owned private repos need Code Security or Advanced Security, and autofix is limited to GitHub Enterprise Cloud. Small private teams are mostly locked out of the autofix half.

GitHub publishes no accuracy numbers. The changelog names no model and gives no false-positive or miss rate. You will learn the noise level from your own pull requests, and you should keep notes for the first few weeks before you decide whether it earns attention.

The gaps are specific. AI Scan skips fork pull requests, which is where outside contributions arrive on public projects, and Dependabot pull requests. Autofix only touches alerts on the default branch, so findings on feature branches or open PRs cannot be assigned to Copilot.

Advisory can decay into ignored. The same choice that keeps AI Scan from blocking releases makes it easy to scroll past. Without an owner (step 2), it becomes background noise within a month.

Passing CodeQL is not passing review. The rerun proves the alert closed. Your tests, and a human, prove the code still works.

The pattern to keep

The interesting part of these launches is not that GitHub found more vulnerabilities with a model. It is where GitHub chose to put the model: next to the diff, talking, and nowhere near the merge button unless something deterministic agrees.

You can argue with the price, and you can wish for published accuracy numbers. The wiring is still right. Take one pipeline this week, find the step where a model's opinion currently decides something, and put a check you can rerun behind it. Then decide for yourself how much the model gets to say.

Sources: GitHub changelog, AI Scan GA (Oct 15, 2026) · GitHub changelog, agentic autofix GA (Oct 15, 2026) · GitHub changelog, AI security detections preview (Jul 14, 2026) · GitHub changelog, AI Scan APIs preview (Sep 10, 2026) · GitHub REST docs, update a code scanning alert · Zach Landes, "Jev meets the Trolley Problem" (Oct 14, 2026) · Cloudflare blog, programmable Gateway and DLP (Oct 15, 2026)

Medium metadata

Title: GitHub's AI Scan Can't Block Your Merge. That's the Point.

(Previous working title: GitHub's New AI Code Reviewer Can't Block Your Merge. That's the Point.)

Alternate titles to test:

  • The AI Suggests, the Compiler Decides: GitHub's Blueprint for AI Code Review
  • You Don't Have to Trust AI to Let It Fix Your Code
  • AI Code Review Isn't a Merge Gate Anymore

Subtitle options:

  • AI Scan and agentic autofix went GA together. How GitHub wired them is a blueprint for every AI check in your pipeline. (picked)
  • GitHub made its AI reviewer advisory and made CodeQL the referee for its fixing agent. Copy that split.
  • Why the smartest AI review setup lets the model talk and lets something boring decide.

Description: GitHub's AI Scan can flag but never block, and its autofix agent must pass a CodeQL rerun. Here's why that split is right and how to copy it.

Tags:

  1. GitHub: the launches are GitHub's, and the tag reaches people already using code scanning.
  2. AI Agents: the verifier-loop pattern applies to any agent pipeline.
  3. Software Development: practical workflow change for working developers.
  4. Cybersecurity: the features are security scanning and fixing.
  5. DevOps: the advice lands in CI and merge gates.

Article type: B (authority), with a Type A hook from the October 15 GA launches.

Publish plan: post at 7 to 8 AM ET, let Medium distribute for 2 to 6 hours, send to subscribers only if it is performing.