A new approach forces an LLM-driven pentest agent to prove a breach instead of merely claiming one, using challenge-response nonces that remove false positives. The technique, demonstrated in the HALO framework, turns “looks like we got a shell” into “we actually have a shell”.

Why false-positive breaches matter

Automated exploitation engines built on large language models can churn out dozens of “successful” port compromises in a single run. Many services echo strings such as “uid=0” in banners, and a crafted target can mimic those outputs without ever executing the attacker’s code. When the agent trusts those echoes, every subsequent decision—whether to pivot, exfiltrate data, or move laterally—rests on a lie. Security teams waste hours chasing phantom footholds, and incident responders may mis-prioritize real threats.

Turning a claim into proof

The fix borrows from classic authentication tricks. Before launching an exploit, the attacker’s system generates a unique token, or nonce, and embeds it in the payload. The exploit must return the exact token for the controller to accept the result as a genuine breach. A fake banner cannot guess the nonce; it must execute the attacker’s code to embed the token in the response. If the returned data lacks the matching nonce, the attempt is discarded as a false positive.

This shift changes the verification model from “the output looks right” to “the output proves execution”. It removes the optimism bias that plagues autonomous offense tools.

Building a reliable delivery ladder

Getting the payload to the target still requires a solid delivery chain. HALO classifies three common paths:

  • Reverse shells – the compromised host initiates a connection back to a listener the attacker controls. Useful when inbound traffic is blocked.
  • Bind shells – the attacker connects directly to a listening service on the target. Works when outbound filters are lax.
  • Blind callbacks – a one-way signal (e.g., DNS request) that confirms execution in highly restricted environments where no direct channel can be opened.

Each step in the ladder must preserve the nonce, or the proof step fails downstream.

Ensuring self-contained exploits

Another source of false confidence is reliance on external libraries that may be missing on the target. HALO bundles every required component into a single file before shipping. The bundle is then tested in a sandbox that deliberately lacks the original dependencies. If the exploit still runs, the artifact is truly self-contained and can be trusted on a locked-down system.

Cleaning the development trail

While preparing for public release, the author discovered real IP addresses lingering in the Git history. A clean working tree does not erase those records; Git retains every commit. The author rewrote the repository to a single clean commit and replaced the leaked addresses with documentation-only ranges defined by RFC 5737 (for example, 192.0.2.0/24). This prevents accidental exposure of production infrastructure when the tool is shared.

Practical rules for security tooling

  • Use documentation-only IP ranges in every test fixture.
  • Strip secrets and scope files from the initial commit.
  • Apply challenge-response nonces to verify any claimed breach.
  • Validate the exact file that will be shipped, not a loosely related script.

Proof beats optimism. By forcing an autonomous pentest agent to present a verifiable token, HALO shows that a breach is only a breach when the target can prove it ran the attacker’s code. The gate is built first; everything else follows.