AI Agent Proposes Security Fixes Via Pull Requests

A security alert triggers. An LLM reads the context. It writes a configuration fix. It opens a GitHub Pull Request. A second LLM reviews the work. A human merges the fix.

The agent never touches production. It never merges by itself.

Normal Kubernetes controllers just retry. If a pod crashes due to bad config, the controller restarts it forever. It does not fix the root cause. Security has the same problem. A SIEM raises an alert. A human must read it and write a fix. This process is slow.

We built an agent to solve this. It reads security alerts and looks at infrastructure to write a draft. This is not an autopilot. It is a draft generator. Everything else stays under human and GitOps control.

How it works:

• Wazuh detects attacks and saves alerts to a database. • A bridge pulls new alerts and starts the agent. • The agent uses limited tools. It can read/write files, create branches, and use the GitHub API. It only edits configuration files. • ArgoCD watches the repo. Nothing deploys until a human merges the PR.

We use two safety layers:

  1. Code checks. Non-LLM rules verify the diff. We check if the change stays within config files. We check if the server path matches the alert. We check the size of the diff. If it fails, the PR stops.

  2. A second LLM review. A different model family reads the PR. It provides an opinion. We use a different model because models often agree with themselves. We also tell the model not to trust the PR text. This prevents attackers from hiding commands in the alert.

What we changed during development:

• We moved from Cloud LLMs to our own server using Ollama. This keeps data private and prevents service interruptions. • We replaced "ToolCalling" agents with "CodeAgent." Small models struggle to send clean JSON. CodeAgent writes Python instead. This reduced errors and steps from 20+ down to 3-8. • We replaced "run_shell" tools with specific Git tools. This prevents dangerous commands like "rm -rf". • We moved from a "push" setup to a "pull" setup for better network stability.

We treat the LLM like a fast junior developer. Its drafts are useful because humans and secondary checks stand between the agent and your production environment.

Source: https://dev.to/aisecops/an-ai-agent-that-proposes-security-fixes-as-pull-requests-2hjj

Optional learning community: https://t.me/GyaanSetuAi