GitHub’s CodeQL 2.26.0 adds a built-in query that spots AI prompt-injection patterns, and the change is already causing CI pipelines to flag new risks. The upgrade alone isn’t enough—teams need a regression test suite that guarantees the rule stays effective as code evolves.

Why a regression fixture matters

Prompt injection lets an attacker slip malicious instructions into a prompt that a language model later follows. With the new query, static analysis can trace data from an untrusted source to a model-calling sink. If the rule is simply turned on and never verified, a later refactor could break the data-flow path and the alert would disappear silently. A regression fixture captures the exact paths that should trigger (or not trigger) the rule, turning the static-analysis result into a contract that the build enforces.

The three ingredients of a reliable fixture

  1. Untrusted source – any function that brings in data from outside the trusted code base (e.g., a GitHub issue body, a webhook payload).
  2. Prompt construction – the code that assembles the model request, typically a call to a client SDK.
  3. Model sink – the SDK method that sends the prompt to the model. CodeQL’s data-flow engine needs to see a real call from your production stack to recognize the sink.

Only when all three are present does the query fire.

Organising the test files

A conventional layout keeps the suite easy to audit:

security-fixtures/prompt-injection/
├─ positive/
│  ├─ direct-flow.ts
│  └─ helper-flow.ts
├─ negative/
│  └─ trusted-instruction.ts
└─ expected-alerts.json

Positive files contain code that should be flagged; negative files hold safe patterns that must stay silent.

Writing the positive cases

The simplest example shows a direct flow from an untrusted value to the model call:

import { model } from "./supported-client";

declare function loadIssueBody(id: number): Promise<string>;

export async function summarize(id: number) {
  const untrusted = await loadIssueBody(id);
  return model.generate({
    system: "Summarize the issue",
    user: untrusted,
  });
}

Here loadIssueBody is the untrusted source, model.generate is the sink, and the data passes without any sanitisation step—exactly what the query is designed to catch.

A second positive case should route the data through a helper function, proving that the analysis follows indirect paths:

function wrapUserInput(input: string) {
  return { system: "Summarize the issue", user: input };
}

export async function summarizeViaHelper(id: number) {
  const raw = await loadIssueBody(id);
  return model.generate(wrapUserInput(raw));
}

Both files belong under positive/.

Writing the negative case

The negative fixture must demonstrate that user input cannot alter the model’s instruction. A common mistake is to assume a function called sanitize() guarantees safety. The static analyzer does not treat the name as a proof, so the test should avoid any misleading sanitisation stub:

export async function safeSummarize(id: number) {
  const trusted = "Summarize the issue";
  const user = await loadIssueBody(id); // not used in the system prompt
  return model.generate({
    system: trusted,
    user: "Static placeholder",
  });
}

Because the untrusted data never reaches the system field, the rule should stay quiet.

Declaring expectations in JSON

The suite’s contract lives in expected-alerts.json. It lists required alerts and explicitly forbidden paths:

{
  "required": [
    {
      "ruleId": "USE_ACTUAL_RULE_ID",
      "pathSuffix": "positive/direct-flow.ts"
    },
    {
      "ruleId": "USE_ACTUAL_RULE_ID",
      "pathSuffix": "positive/helper-flow.ts"
    }
  ],
  "forbiddenPathSuffixes": [
    "negative/trusted-instruction.ts"
  ]
}

Replace USE_ACTUAL_RULE_ID with the identifier shown in the CodeQL documentation or SARIF output. Do not guess IDs; the exact string matters for the CI check.

Hooking the fixture into CI

  1. Pin the CodeQL CLI version used in the pipeline to 2.26.0 (or later).
  2. Build a disposable database from the current checkout before running the fixture.
  3. Run the query, capture alerts, and compare them against expected-alerts.json.
  4. Fail the build if any required alert disappears or if a forbidden path starts alerting.

Do not assert a total alert count across the repository—unrelated changes could inflate the number and cause false failures.

What to watch after an upgrade

When you bump CodeQL to a newer version:

  • Required alert still appears – proceed with the normal review.
  • Required alert vanishes – block the build; investigate whether the new version changed the query logic or whether a code change broke the data flow.
  • New positive location appears – add it to the required list after confirming it is a genuine injection path.
  • Negative control starts alerting – revisit the mitigation strategy; the rule may have become stricter.

Static analysis cannot prove how the model will react at runtime. Complement the regression suite with adversarial tests that actually send crafted prompts to the model and verify the response.

Takeaway

CodeQL 2.26.0 gives you the ability to catch prompt-injection bugs before they ship, but only if you lock that ability down with a focused regression fixture. By defining untrusted sources, real SDK sinks, and clear expectations in a JSON contract, you turn a static-analysis rule into a gate that stops regressions and forces continuous attention on a rapidly evolving attack surface.