Your AI-driven assistant obeys its instructions about 99 % of the time, but that missing 1 % is where attackers strike. By feeding a crafted prompt, a malicious user can make the model invoke functions it shouldn’t, stealing data or performing privileged actions. The fix isn’t more polite wording—it’s treating the flaw as an authorization issue and removing the dangerous tools from the model’s reach.
Why prompt injection isn’t just a wording problem
Developers often try to harden agents with all-caps warnings, numbered rules, or “do not call admin functions” clauses. Those defenses assume the model will obey a sentence that says “don’t do X.” In practice, the model can be coaxed into ignoring the instruction by re-phrasing the request, role-playing a different persona, or simply appending extra context. The English boundary is negotiable; the attacker’s prompt is unlimited and costs nothing to test.
The real vulnerability lies in the tool list the agent receives. When the prompt schema contains a function that grants admin rights, the model now has a map to that power. Even if the prompt says “don’t use it for customers,” the model can still be persuaded to call it because the function exists in its execution environment. The problem is therefore an authorization gap: the system is exposing privileged capabilities to a caller that has no right to them.
Securing agents by limiting exposure
The simplest way to close the gap is to stop giving the model access to tools it isn’t authorized to use. Think of the tool list as an API key: if the key isn’t present, the call can’t happen. No clever wording can summon a function that isn’t in the current context.
The wrong way
Prompt: “You are an assistant. Do not use the adminDeleteUser function for regular customers.”
The model still sees adminDeleteUser in its toolbox and can be tricked into invoking it.
The right way
Prompt schema for a regular customer: { “functions”: [ “searchCatalog”, “placeOrder” ] }
adminDeleteUser never appears, so the model has no path to call it.
Three practical rules for developers
- Build tool lists per request – Generate the function catalog dynamically, based on the authenticated caller’s permissions. A customer sees only the functions they need; an admin sees the full set.
- Fail closed – If the user’s identity cannot be verified, return an empty list rather than a generic “all tools available” fallback. This guarantees that an unauthenticated request never gains unexpected power.
- Avoid shared state – When caching tool definitions, never write user-specific data onto a shared object. Use copy-on-write or per-session copies so that one user’s permissions cannot bleed into another’s request.
If the schema presented to a regular user looks identical to the one shown to an admin, the security boundary is still the prompt text, and prompts are not a reliable security mechanism.
What led us here
Prompt injection surfaced when developers started plugging large language models (LLMs) into production workflows that required the model to call external APIs, run code, or modify databases. The model’s “reasoning” is guided by a prompt that also includes a list of available tools. Early prototypes assumed the model would obey a natural-language rule like “don’t delete records for non-admins.” Attackers quickly demonstrated that a few extra sentences could bypass those rules, prompting the model to call the same delete function anyway.
The community’s first reaction was to tighten the prompt language, add “never do X” clauses, or embed regex filters that strip suspicious tokens. Those measures reduced accidental misuse but did not stop a determined adversary who could simply rephrase the request. The underlying cause—exposing privileged functions to an untrusted caller—remained.
Who wins, who loses
Enterprises that adopt per-request tool scoping gain a clear, enforceable boundary. Their agents can be deployed at scale without fearing that a single malformed prompt will unlock admin capabilities. Compliance teams also appreciate the audit trail: the list of functions sent to the model is a concrete artifact that can be logged and reviewed.
Developers who rely on prompt-only guards continue to face a moving target. Their agents may appear functional in testing but could be compromised in the wild, leading to data breaches, unauthorized transactions, or compliance violations. The cost of a breach far outweighs the effort of building a dynamic tool list.
Counter-argument: “Better prompts are enough”
有人认为，通过足够的指令工程——分层提示词、系统消息以及来自人类反馈的强化学习（RLHF）——可以使模型遵守“不要做”之类的条款。现实情况是，语言模型是概率生成器；它们权衡的是最可能的后续内容，而不是硬性的安全规则。即使有了微调过的护栏（guardrails），新的措辞仍可能绕过限制，尤其是当攻击者可以以零成本进行无休止的迭代时。护栏对于减少噪声很有用，但不应作为唯一的防线。
下一步值得关注的方向
- 将工具作用域（tool scoping）作为一等公民 API 暴露出来的框架 – 预计会出现新的库，允许你声明每个用户的权限，并在构建提示词之前自动修剪函数列表。
- 标准化的“函数清单”（function manifests） – 行业团体可能会定义一种 JSON schema，将公共函数与特权函数分开，从而更容易生成针对特定请求的清单。
- 运行时强制执行（Runtime enforcement） – 一些平台正在尝试沙盒化执行，通过检查调用者的令牌（token）与被调用的函数是否匹配，在提示词作用域之外增加第二层防护。
结论很明确：将提示词注入视为一种授权缺陷。通过从模型的工具箱中移除未经授权的工具，你可以消除那些试图通过巧妙措辞的提示词来利用的攻击面。提示词可以引导行为，但不能取代适当的访问控制。