Incident response for AI agents

What changes for agent incidents

1. Authorisation is ambiguous. A compromised user account produces obviously anomalous behaviour. A manipulated agent does exactly what it is capable of doing, using permissions it legitimately holds. There is often no technical signal distinguishing the incident from normal operation.
2. Speed is asymmetric. By the time a human notices, an agent has completed operations a human attacker would still be enumerating. Your detection-to-containment interval matters far more than usual.
3. The blast radius is a graph, not a list. One agent may have acted for hundreds of principals, and each action inherited a different scope.
4. Attribution runs backwards through delegation, not forwards through network flow. Conventional network forensics tells you which service called which; it does not tell you whose authority was used.

Triage: was this authorised?

Three questions, in order, each answerable from artefacts you should already hold:

• Was the agent acting within its registered scope? Compare actions taken to the reach recorded at provisioning. If it exceeded that, you have a control failure regardless of intent.
• Was there a delegating principal, and did they intend this? Pull the delegation chain. An action with no principal on a user-delegated agent is a strong signal of manipulation.
• Did the input come from a trusted source? If the agent processed external content before acting, treat prompt injection as the leading hypothesis rather than credential compromise, and preserve the input.

If you cannot answer question two, note that as a finding in the incident record. It will recur, and an incident report is a more effective route to fixing delegation logging than a risk register entry.

Six containment actions, in order

1. Suspend the agent's identity at the issuer. No new credentials minted. Fast and insufficient alone.
2. Push revocation to enforcement points. Without this your containment latency equals your token lifetime. This is what CAEP and the Shared Signals Framework exist for.
3. Halt the runtime. Distinct from revoking identity. An agent mid-plan with cached context and queued calls keeps trying, generating retries and side effects.
4. Revoke downstream artefacts the agent created: OAuth grants, webhooks, API keys it minted. These survive the parent identity and are the most common live remnant.
5. Suspend sibling agents sharing the credential or trust path. If agents share an identity, you cannot contain one, which is why per-agent identity matters operationally and not only for audit.
6. Notify the delegating principals whose authority may have been used. This is frequently a regulatory obligation and it depends entirely on step four of the scoping work below.

Determining blast radius

1. Enumerate actions in the window from the agent's own logs and from every downstream system it touched. The second source matters because the first may be manipulable.
2. Map each action to its delegation chain to identify the principal whose authority was used.
3. Separate reads from writes. Writes are remediation; reads are notification and often the larger regulatory exposure.
4. Group by principal to produce the notification list. Without delegation records this step is impossible and you must assume every principal the agent ever served.
5. Check for onward delegation. In a multi-agent system, enumerate descendants. A stable chain identifier makes this a query; without one it is an archaeology project.

The difference between having delegation records and not is the difference between notifying eleven principals and notifying all of them. That is the concrete cost, and it is worth putting in front of whoever funds the logging work.

Evidence preservation

• Capture agent memory and context before restarting anything. This is the equivalent of memory capture on a compromised host and it is routinely destroyed by a well-meaning restart in the first ten minutes.
• Preserve the inputs, including retrieved documents, tool outputs and inter-agent messages. If injection is the hypothesis, the payload is in there.
• Snapshot the configuration as it was: system prompt, tool set, granted scopes. All are commonly changed during response, destroying the record of the state that failed.
• Preserve the credential material metadata: issuance time, scope, chain identifiers, even after revoking the credential itself.

What to prepare before you need it

1. A measured revocation time. Run the drill, record the number. Every timeline in your plan depends on it.
2. A per-agent kill procedure that a responder can execute at 03:00 without the building agent's author.
3. Delegation logging, because it cannot be retrofitted to actions already taken.
4. An agent inventory with owners, so the responder knows who to call. See agent registries.
5. Agreed correlation identifiers with any third party whose agents interoperate with yours. Negotiating log formats during an incident does not go well.
6. A tabletop that includes a manipulated-but-not-compromised scenario, since that is the case conventional playbooks handle worst.

Frequently asked questions

How is AI agent incident response different from normal incident response?

Four ways. Authorisation is ambiguous, because a manipulated agent does exactly what it is capable of doing with permissions it legitimately holds, so there is often no anomalous technical signal. Speed is asymmetric, since an agent completes in minutes what a human attacker would still be enumerating. Blast radius is a graph rather than a list, because one agent may have acted for many principals at different scopes. And attribution runs backwards through delegation rather than forwards through network flow.

How do you contain a compromised AI agent?

Six actions in order: suspend the identity at the issuer, push revocation to enforcement points so containment latency does not equal token lifetime, halt the runtime because an agent mid-plan keeps attempting queued calls after its credentials are gone, revoke downstream artefacts the agent created such as OAuth grants and webhooks, suspend sibling agents sharing the credential or trust path, and notify affected delegating principals. Network isolation is largely ineffective because the agent may never traverse your network.

How do you determine the blast radius of an agent incident?

Enumerate actions from both the agent's logs and every downstream system it touched, map each action to its delegation chain to identify whose authority was used, separate reads from writes, group by principal to produce a notification list, and check for onward delegation to descendant agents. Without delegation records the fourth step is impossible and you must assume every principal the agent ever served, which is the concrete cost of not logging chains.

What evidence should be preserved in an agent incident?

Agent memory and context before any restart, which is the equivalent of memory capture on a compromised host and is routinely destroyed in the first ten minutes. Also the inputs including retrieved documents, tool outputs and inter-agent messages, since an injection payload lives there; a snapshot of the configuration as it was, covering system prompt, tool set and granted scopes; and credential metadata such as issuance time and chain identifiers even after the credential is revoked.

What single preparation matters most for agent incident response?

A measured revocation time. Run a drill on a non-production agent with representative permissions, issue a revocation through the normal process, and record the interval to confirmed loss of access. Every timeline in an incident plan depends on that number, and most organisations have never established it, so their plans rest on an assumption rather than a measurement.