Your AI-driven assistant obeys its instructions about 99 % of the time, but that missing 1 % is where attackers strike. By feeding a crafted prompt, a malicious user can make the model invoke functions it shouldn’t, stealing data or performing privileged actions. The fix isn’t more polite wording—it’s treating the flaw as an authorization issue and removing the dangerous tools from the model’s reach.
Why prompt injection isn’t just a wording problem
Developers often try to harden agents with all-caps warnings, numbered rules, or “do not call admin functions” clauses. Those defenses assume the model will obey a sentence that says “don’t do X.” In practice, the model can be coaxed into ignoring the instruction by re-phrasing the request, role-playing a different persona, or simply appending extra context. The English boundary is negotiable; the attacker’s prompt is unlimited and costs nothing to test.
The real vulnerability lies in the tool list the agent receives. When the prompt schema contains a function that grants admin rights, the model now has a map to that power. Even if the prompt says “don’t use it for customers,” the model can still be persuaded to call it because the function exists in its execution environment. The problem is therefore an authorization gap: the system is exposing privileged capabilities to a caller that has no right to them.
Securing agents by limiting exposure
The simplest way to close the gap is to stop giving the model access to tools it isn’t authorized to use. Think of the tool list as an API key: if the key isn’t present, the call can’t happen. No clever wording can summon a function that isn’t in the current context.
The wrong way
Prompt: “You are an assistant. Do not use the adminDeleteUser function for regular customers.”
The model still sees adminDeleteUser in its toolbox and can be tricked into invoking it.
The right way
Prompt schema for a regular customer: { “functions”: [ “searchCatalog”, “placeOrder” ] }
adminDeleteUser never appears, so the model has no path to call it.
Three practical rules for developers
- Build tool lists per request – Generate the function catalog dynamically, based on the authenticated caller’s permissions. A customer sees only the functions they need; an admin sees the full set.
- Fail closed – If the user’s identity cannot be verified, return an empty list rather than a generic “all tools available” fallback. This guarantees that an unauthenticated request never gains unexpected power.
- Avoid shared state – When caching tool definitions, never write user-specific data onto a shared object. Use copy-on-write or per-session copies so that one user’s permissions cannot bleed into another’s request.
If the schema presented to a regular user looks identical to the one shown to an admin, the security boundary is still the prompt text, and prompts are not a reliable security mechanism.
What led us here
Prompt injection surfaced when developers started plugging large language models (LLMs) into production workflows that required the model to call external APIs, run code, or modify databases. The model’s “reasoning” is guided by a prompt that also includes a list of available tools. Early prototypes assumed the model would obey a natural-language rule like “don’t delete records for non-admins.” Attackers quickly demonstrated that a few extra sentences could bypass those rules, prompting the model to call the same delete function anyway.
The community’s first reaction was to tighten the prompt language, add “never do X” clauses, or embed regex filters that strip suspicious tokens. Those measures reduced accidental misuse but did not stop a determined adversary who could simply rephrase the request. The underlying cause—exposing privileged functions to an untrusted caller—remained.
Who wins, who loses
Enterprises that adopt per-request tool scoping gain a clear, enforceable boundary. Their agents can be deployed at scale without fearing that a single malformed prompt will unlock admin capabilities. Compliance teams also appreciate the audit trail: the list of functions sent to the model is a concrete artifact that can be logged and reviewed.
Developers who rely on prompt-only guards continue to face a moving target. Their agents may appear functional in testing but could be compromised in the wild, leading to data breaches, unauthorized transactions, or compliance violations. The cost of a breach far outweighs the effort of building a dynamic tool list.
Counter-argument: “Better prompts are enough”
Some argue that with enough instruction engineering—layered prompts, system messages, and reinforcement learning from human feedback—the model can be made to respect “do not” clauses. The reality is that language models are probabilistic generators; they weigh the most likely continuation, not a hard security rule. Even with fine-tuned guardrails, a novel phrasing can slip through, especially when the attacker can iterate endlessly at zero cost. Guardrails are useful for reducing noise but should not be the sole line of defense.
What to watch next
- Frameworks that expose tool scoping as a first-class API – Expect new libraries that let you declare per-user capabilities and automatically prune the function list before the prompt is built.
- Standardized “function manifests” – Industry groups may define a JSON schema that separates public and privileged functions, making it easier to generate request-specific manifests.
- Runtime enforcement – Some platforms are experimenting with sandboxed execution that checks the caller’s token against the function being invoked, adding a second layer beyond prompt scoping.
The takeaway is clear: treat prompt injection as an authorization flaw. By removing unauthorized tools from the model’s toolbox, you eliminate the attack surface that a cleverly worded prompt seeks to exploit. Prompts can guide behavior; they cannot replace proper access control.