Your AI-driven assistant obeys its instructions about 99 % of the time, but that missing 1 % is where attackers strike. By feeding a crafted prompt, a malicious user can make the model invoke functions it shouldn’t, stealing data or performing privileged actions. The fix isn’t more polite wording—it’s treating the flaw as an authorization issue and removing the dangerous tools from the model’s reach.
Why prompt injection isn’t just a wording problem
Developers often try to harden agents with all-caps warnings, numbered rules, or “do not call admin functions” clauses. Those defenses assume the model will obey a sentence that says “don’t do X.” In practice, the model can be coaxed into ignoring the instruction by re-phrasing the request, role-playing a different persona, or simply appending extra context. The English boundary is negotiable; the attacker’s prompt is unlimited and costs nothing to test.
The real vulnerability lies in the tool list the agent receives. When the prompt schema contains a function that grants admin rights, the model now has a map to that power. Even if the prompt says “don’t use it for customers,” the model can still be persuaded to call it because the function exists in its execution environment. The problem is therefore an authorization gap: the system is exposing privileged capabilities to a caller that has no right to them.
Securing agents by limiting exposure
The simplest way to close the gap is to stop giving the model access to tools it isn’t authorized to use. Think of the tool list as an API key: if the key isn’t present, the call can’t happen. No clever wording can summon a function that isn’t in the current context.
The wrong way
Prompt: “You are an assistant. Do not use the adminDeleteUser function for regular customers.”
The model still sees adminDeleteUser in its toolbox and can be tricked into invoking it.
The right way
Prompt schema for a regular customer: { “functions”: [ “searchCatalog”, “placeOrder” ] }
adminDeleteUser never appears, so the model has no path to call it.
Three practical rules for developers
- Build tool lists per request – Generate the function catalog dynamically, based on the authenticated caller’s permissions. A customer sees only the functions they need; an admin sees the full set.
- Fail closed – If the user’s identity cannot be verified, return an empty list rather than a generic “all tools available” fallback. This guarantees that an unauthenticated request never gains unexpected power.
- Avoid shared state – When caching tool definitions, never write user-specific data onto a shared object. Use copy-on-write or per-session copies so that one user’s permissions cannot bleed into another’s request.
If the schema presented to a regular user looks identical to the one shown to an admin, the security boundary is still the prompt text, and prompts are not a reliable security mechanism.
What led us here
Prompt injection surfaced when developers started plugging large language models (LLMs) into production workflows that required the model to call external APIs, run code, or modify databases. The model’s “reasoning” is guided by a prompt that also includes a list of available tools. Early prototypes assumed the model would obey a natural-language rule like “don’t delete records for non-admins.” Attackers quickly demonstrated that a few extra sentences could bypass those rules, prompting the model to call the same delete function anyway.
The community’s first reaction was to tighten the prompt language, add “never do X” clauses, or embed regex filters that strip suspicious tokens. Those measures reduced accidental misuse but did not stop a determined adversary who could simply rephrase the request. The underlying cause—exposing privileged functions to an untrusted caller—remained.
Who wins, who loses
Enterprises that adopt per-request tool scoping gain a clear, enforceable boundary. Their agents can be deployed at scale without fearing that a single malformed prompt will unlock admin capabilities. Compliance teams also appreciate the audit trail: the list of functions sent to the model is a concrete artifact that can be logged and reviewed.
Developers who rely on prompt-only guards continue to face a moving target. Their agents may appear functional in testing but could be compromised in the wild, leading to data breaches, unauthorized transactions, or compliance violations. The cost of a breach far outweighs the effort of building a dynamic tool list.
Counter-argument: “Better prompts are enough”
Alcuni sostengono che con un'ingegneria delle istruzioni sufficiente — prompt stratificati, messaggi di sistema e apprendimento per rinforzo dal feedback umano — sia possibile rendere il modello rispettoso delle clausole di "divieto". La realtà è che i modelli linguistici sono generatori probabilistici; valutano la continuazione più probabile, non una regola di sicurezza rigida. Anche con guardrail perfezionati, una formulazione inedita può passare inosservata, specialmente quando l'attaccante può iterare all'infinito a costo zero. I guardrail sono utili per ridurre il rumore, ma non dovrebbero essere l'unica linea di difesa.
Cosa monitorare in seguito
- Framework che espongono lo scoping degli strumenti come API di prima classe – Aspettatevi nuove librerie che consentano di dichiarare le capacità per utente e di eliminare automaticamente l'elenco delle funzioni prima che il prompt venga costruito.
- "Manifesti delle funzioni" standardizzati – I gruppi industriali potrebbero definire uno schema JSON che separi le funzioni pubbliche da quelle privilegiate, facilitando la generazione di manifesti specifici per ogni richiesta.
- Enforcement a runtime – Alcune piattaforme stanno sperimentando l'esecuzione in sandbox che verifica il token del chiamante rispetto alla funzione invocata, aggiungendo un secondo livello oltre allo scoping del prompt.
La conclusione è chiara: trattate la prompt injection come un difetto di autorizzazione. Rimuovendo gli strumenti non autorizzati dalla cassetta degli attrezzi del modello, si elimina la superficie di attacco che un prompt formulato abilmente cerca di sfruttare. I prompt possono guidare il comportamento, ma non possono sostituire un adeguato controllo degli accessi.