Safer AI agents: design permissions to limit blast radius
Prompt filters are a helpful layer, not a trust boundary. This article moves from identity and action brokering to rollback exercises so an AI agent does not become a production weakness.
An AI agent often starts harmlessly: reading documents, answering internal questions, or summarizing tickets. A few weeks later, users ask it to look up CRM records, create requests, update a spreadsheet, or call an operations API. At that point the risk is no longer a wrong answer. The risk is a probabilistic system affecting real data and services faster than people can notice.
That is why prompt filters deserve the right expectation. Filters, classifiers, and guardrails can block many unsafe commands or attack patterns. They are worth having. They do not, however, create a hard trust boundary like database access control or a network policy. Untrusted content can arrive through email, files, web pages, or search results; the model can misunderstand context; and attack patterns can change faster than rules.
Ask the useful question: what can the agent do when it is misled?
Rather than trying to prove an agent will never be deceived, assume it can be steered toward an unwanted action. Map four areas for every agent: data it reads, untrusted data sources, tools it can call, and destinations for data or actions. An agent that can read sensitive information, read external email, and send content to an outside service should be treated as high risk even when its current prompt looks safe.
The map needs both a business owner and a technical owner. The business owner explains the value and acceptable actions. The technical owner explains identity, available tools, log location, and how the agent is stopped during an incident. Without both owners, an agent is not production-ready.
Identity and permissions come before every guardrail
An agent should not inherit a user’s raw long-lived token, and it should never run with a shared administrator account. A safer design grants a separate workload identity, short-lived tokens, and a scope small enough for one task. When an agent acts on a user’s behalf, effective permission should be the intersection of user permission, agent capability, and resource policy. This prevents a low-privilege user from accidentally gaining the breadth of an overpowered service account.
Secrets should also remain outside model context. The model may propose a tool intent, but a broker or tool runner retrieves credentials from a vault at execution time. Credentials should not appear in prompts, application logs, responses, or debugging transcripts. It is a simple principle that is easily broken when a team rushes a demonstration.
Let the model propose; let a broker execute
A strong design separates the agent from direct API authority. The agent returns structured intent, such as “open a P2 ticket” or “query order X.” An independent broker checks schema, user context, policy, endpoint allowlists, quotas, and parameters before execution. For material actions, the broker sends the request to an approval queue instead of calling a tool automatically.
This separation has two benefits. First, policy is checked by deterministic code instead of trusting a model to be well behaved. Second, logs become readable: the team knows what the agent proposed, which rule accepted or refused it, and what the tool actually did. That is the evidence required when a customer, auditor, or investigator asks why an action occurred.
Human approval does not need to become a bottleneck
People do not need to approve every operation. That creates approval fatigue until everyone accepts by reflex. Instead, tier actions by impact. Agents can automate read-only work or small reversible changes within controlled scope. Deletion, production changes, data export, firewall changes, cost creation, or customer commitments require explicit approval.
A useful approval gives the reviewer context: what the agent intends to do, which data is involved, which resources are affected, which policy permits it, and how rollback works. An Approve button without context is not a control; it simply transfers risk to a person.
Observability, kill switches, and real exercises
Every tool call should create an audit trail with redacted input, identity, effective permission, policy decision, result, and correlation ID. The system needs rate limits, agent quotas, abnormal-behavior alerts, and a kill switch that operates independently of the model. Test the kill switch: who may use it, how long it takes, and what happens to in-flight queues.
Before release, exercise cases such as a file containing malicious instructions, an API returning an unexpected response, a token expiring mid-session, a tool timeout, a rejected approval, and rollback after a simulated change. These tests do not prove perfection; they prove that consequences are contained and the team knows how to operate when something goes wrong.
Expand through maturity stages
In stage one, allow only reading and summarization of selected data. In stage two, let the agent create proposals that people approve. In stage three, automate limited low-risk actions with quotas, allowlists, and rollback. Only after telemetry, incident process, and ownership are mature should more material action be considered. Moving carefully does not reduce AI value; it prevents that value from being paid for with an incident the organization cannot explain.
Published ; updated