Foreman turns large-language-model (LLM) agents into native Kubernetes resources, letting teams run AI-generated code in production while keeping costs and safety under strict control.
How the idea fits into today’s dev workflow
Enterprises have been experimenting with LLMs that can write code, but most implementations treat the model as a trusted black box. A single “done” signal from the model can push untested changes straight into a repository, raising security and reliability concerns. At the same time, running AI services on the cloud can quickly become expensive, especially when the same model is called repeatedly from CI pipelines.
Foreman’s answer is to embed the entire coding loop inside a Kubernetes cluster. Foreman runs as Kubernetes resources.
The four core objects that make it happen
- Agent – Defines a worker. It names the LLM to call, lists the tools the model may invoke (e.g., file-write or git-push), and sets a budget that caps model calls. Roles attach here; a coder agent writes code, a verifier agent checks it.
- Workload – The user-authored unit of work. It holds the high-level intent (e.g., “add unit tests for module X”), a reference to the target repository, and a list of agents that should handle the job.
- AgenticTask – The concrete task that a Workload spawns. As the job progresses, each AgenticTask records status updates, letting operators monitor the pipeline in real time.
- FleetNode – A Kubernetes node that actually runs the tasks. The built-in scheduler matches pending AgenticTasks to FleetNodes that have the required role and resources.
Verification replaces blind trust
Foreman does not assume the model’s output is correct. When a coder agent finishes its work, it sends a request instead of a final result. A verifier—usually a deterministic script, not another LLM—runs the code through linters, unit tests, or full builds. Only if those checks pass does Foreman write the new branch back to the repository.
If the verifier fails, the task is marked rejected and the changes never land. This separation lets the model generate creatively while the safety net stays fully under human control.
Installing the stack on a cluster
- Deploy the LLMKube core chart with Helm.
- Deploy the Foreman chart, also via Helm.
- Switch the agent mode to “native” so the real request-response loop is active.
- Assign roles (coder, verifier) to the FleetNodes that will host the work.
Two credential sets are required: git credentials for reading issues and pushing branches, and model credentials for calling either a hosted API or a self-hosted inference service.
Cost and security knobs you can actually turn
Foreman lets operators limit a model’s authority by pruning tools from the agent definition. Removing “bash” or “write_file” stops the model from executing arbitrary shell commands or writing outside the designated workspace.
A turn limit caps the number of model invocations per task, directly controlling spend. Adjusting the context window—how much of the prompt the model sees—further trims token usage. When the model runs locally on on-prem hardware, no data leaves the organization, satisfying strict data-privacy policies.
Where the platform shines, and where it still stumbles
Foreman excels at mechanical, well-scoped jobs:
- Fixing a documented bug.
- Adding missing test cases.
- Updating documentation for clarity or style.
These tasks have clear success criteria that a verifier can check automatically. The system still struggles with high-level architectural redesigns or ambiguous feature work where “correctness” depends on human judgment.
Takeaway
By treating LLM-driven coders as first-class Kubernetes resources and enforcing a deterministic verification step, Foreman offers a pragmatic path to production AI code generation that keeps spend visible and security under control. It is not a silver bullet for all development work, but for repeatable, testable tasks it provides an auditable workflow that fits naturally into existing cloud-native operations.