What's a good strategy for managing the risks of AI agents?

The three failures that actually cost money are an agent acting outside its intended scope, an agent acting on bad data, and an agent acting silently.

Scope failures are the most common. An agent given broad access to a mailbox, a CRM or a file share will eventually take an action nobody scoped for, because the instruction it received was ambiguous and it resolved the ambiguity on its own. Data failures follow: an agent reading a stale price list or a duplicate customer record produces confident, wrong output that looks exactly like every other output it produces. Silent failures are the expensive ones. An agent that quietly mis-files fifty records a week does more damage over a quarter than one that fails loudly on day one, because nobody is looking for it.

Note what is not on that list. Model quality is rarely the binding constraint. The controls that matter are about access, data and visibility, and they are the same controls you would apply to a new staff member or a new integration.

Give each agent its own credentials, scoped to the smallest set of systems and actions it needs, and never let it inherit a person's access.

In practice that means one identity per agent rather than a shared service account, so every action is attributable to a specific agent. Read access by default, write access only where the process genuinely requires it. No standing access to production data where a filtered view or a copy would do. An explicit list of actions the agent may take, with anything destructive or financial excluded by default rather than allowed by default.

This is the highest-value control available to you, and it is also the one most often skipped, because it is easier to hand an agent broad access during a pilot and tighten it later. Tightening later rarely happens. Ask your IT lead or MSP to scope the permissions before the pilot starts and to give you the permission set in writing. If they cannot describe what the agent can and cannot reach in about a page, it is not scoped yet.

Any action that is irreversible, externally visible or financially material should require a human approval step before it executes.

That covers sending anything to a customer, changing a record you cannot restore, committing money, and altering access or permissions. Most other work can run unattended once it has proven itself. The mistake to avoid is a blanket approval requirement on every action: reviewers stop reading within a fortnight, and approval becomes a rubber stamp that adds delay without adding safety.

A better pattern is to start narrow and widen. Require approval on everything for the first few weeks, track how often a reviewer actually changes or rejects an action, then drop the approval step from the categories where the rejection rate is effectively zero and keep it where it is not. That gives you a control based on evidence rather than a policy written before the agent existed. Set the review point where a person can still undo the action, not after it has left the business.

Log every action in enough detail to reconstruct why the agent did it, and review a sample on a fixed schedule rather than waiting for a complaint.

At minimum you want the input the agent received, the action it took, the systems it touched and the timestamp, retained long enough to investigate something reported weeks later. That log needs to be readable by someone who is not the person who built the agent.

The review cadence matters more than the tooling. A sample of agent output checked weekly by someone who knows what good looks like will catch drift that no dashboard surfaces, because the usual failure mode is plausible but wrong rather than obviously broken. Agree the sample size and the reviewer up front, and treat a missed review as a real issue rather than an admin task.

Agree what triggers a stop as well. A defined threshold, an error rate or a volume spike that pauses the agent automatically is worth considerably more than an intention to notice.

Ask for six things in writing before you approve an agent for production, and treat a missing answer as a reason to delay rather than a detail to chase later.

1. The permission set: exactly which systems and actions the agent can reach.
2. The data it touches, where that data goes, and whether it leaves your tenancy.
3. The approval points: which actions stop for a person, and who that person is.
4. The log: what is recorded, who can read it, and how long it is kept.
5. The stop condition: what pauses the agent, and who has authority to pause it.
6. The owner: one named person accountable for the agent's behaviour after go-live.

None of this requires you to understand the model. It is a governance conversation, and the answers should be short. If they come back vague, that is information about readiness, not a communication problem to work around.