Agent Runbooks: What On-Call Looks Like When AI Runs the Process

An agent runbook is the page a responder opens at 2am when an AI system has already acted. Most teams do not have one. Grant Thornton found that 20% of organisations have tested a response plan for AI failure, while 73% are already piloting, scaling or running autonomous AI. This is the template that closes that gap.

What an agent runbook actually is

A runbook is a written procedure for a named failure. It exists so that a tired person at 2am does not have to invent one.

An agent runbook is the same object, scoped to a system that takes actions on its own behalf. It is not a policy document, not a model card, and not a monitoring dashboard. Those things tell you what should happen and what did happen. A runbook tells you what to do next.

It has to answer three questions inside the first minute. What can I stop. What did it already do. Who do I wake up.

I would not approve an agent for production without those three answers written down and rehearsed once. Not reviewed. Rehearsed, on a working day, with a stopwatch.

Write one runbook per agent, keyed to that agent's authority level. A general AI incident policy is a governance artefact, and it is useless to the person holding the pager.

Three ways agent incidents break a human runbook

Traditional incident response assumes a system that stopped working. Agent incidents rarely present that way, and every part of the standard procedure bends because of it.

The action already happened

A database goes down and the graphs go flat. An agent that starts answering wrongly produces no signal at all. Throughput looks normal. Latency looks normal. The error rate is zero, because nothing errored.

In April 2025, Cursor's AI support agent told users that the product was limited to one device per subscription. No such policy existed. The Register reported that developers cancelled subscriptions over an invented rule, and co-founder Michael Truell had to deny the policy publicly on Reddit and Hacker News.

Nothing was down for a single second of that. The detection path was a customer posting a screenshot to a forum, which is not a monitoring strategy. Building the signal that would have caught it earlier is covered in the piece on monitoring agents for drift in production.

The cause is not in the deploy diff

Standard root cause analysis starts with what changed. For an agent, often nothing changed. The same prompt can return different answers on consecutive calls.

The Cursor bot did not tell every user the same thing. Some were given the invented restriction and some were not, which is the behaviour that makes reproduction hard and makes a bug report look like user error.

So the evidence set is different. You need the input, the retrieved context, the tool calls and the model version. A git log contains none of it.

The blast radius is whatever you granted

In July 2025, an AI agent on Replit deleted a live production database during a stated code freeze. Fortune reported that records covering more than 1,200 executives and over 1,190 companies were destroyed. Chief executive Amjad Masad called it unacceptable and said it should never have been possible, and committed to automatic separation of development and production databases.

The instruction not to touch anything existed. It sat in the prompt, where the agent could decline it, rather than in the permission set, where it could not. An instruction the agent can ignore is not a control, and that boundary belongs in the authority clauses that define what an agent may do.

Severity tiers for agent incidents

Most severity scales are keyed on how many users are affected. For agents, that is the wrong axis.

Key severity on reversibility instead. An agent that gave 10,000 users a slightly worse answer is annoying. An agent that sent one wrong invoice to one regulator is a different category of problem, and the affected user count does not capture it.

These tiers are a starting point, not a measured standard. No published benchmark maps classes of agent action to response times, so calibrate the boundaries against your own agent's tool list.

The first 15 minutes

The first quarter of an hour decides how large the incident gets. Here is the sequence, and the order is deliberate.

Minute 0 to 2: revoke before you diagnose

The instinct is to open the logs and work out what happened. That instinct is wrong here, because the agent is still running while you read.

Revoke the credential, not the container. Restarting a process gets you a fresh agent with the same permissions doing the same thing. Killing the token stops the tool calls. Which credential, in which system, with which command, is the first line of the runbook, and it belongs beside the notes on how non-human identities are issued and revoked.

This step is where most plans fail in practice. Writer surveyed 2,400 employees and C-suite leaders with Workplace Intelligence in January 2026. 35% of executives said they were not very confident they could pull the plug on a rogue agent. That research is vendor-commissioned and self-reported, so treat it as directional. It still describes a first step a third of respondents cannot take.

Minute 2 to 8: capture what disappears

Agent evidence has a short half-life. Context windows are not persisted by default, vendor log retention is often 30 days or less, and a retry can overwrite the trace you needed.

Capture five things before anything else. The full prompt and assembled context. Every tool call with its arguments and its return value. The model name, version and sampling parameters. The documents retrieved. Any approval a human granted during the run.

Most teams log the output and discard the input. The input is the half that explains the incident.

Minute 8 to 15: decide reversible or not

This is the fork in the runbook. If every effect is reversible, you go to rollback and the incident stays technical. If any effect is not, you go to communication, and the incident becomes commercial and possibly regulatory.

Write the decision down with a timestamp. That timestamp is what starts the reporting clock, and you will be asked for it later.

Rollback comes in three layers

Teams say they can roll back and mean one specific thing. There are three separate capabilities behind that phrase, and they fail independently.

The third layer is the one nobody has

An email cannot be unsent. A payment needs a reversal path that somebody has to have built in advance. A message posted into a customer's shared channel needs a correction, and a correction is a communication decision, not a technical one.

The practical rule is simple. Every time you grant an agent a tool, write the compensating action for that tool on the same line. If you cannot write one, the tool needs an approval gate rather than a rollback plan, which is the argument in the piece on designing human approval into agent architecture.

There is a second rule, learned the hard way at Replit. The agent told its user that rollback was impossible and that all database versions were gone. That was not true. Never accept the agent's account of what is recoverable.

Confidence about all of this runs far ahead of practice. The backup vendor Keepit surveyed more than 300 IT decision makers in April 2026. 94% felt assured their recovery plans covered agentic AI systems, 32% tested those plans monthly, and 52% doubted the plans covered agentic scenarios at all. The same respondents were both assured and doubtful, which tells you how much of that assurance is untested.

Who owns the agent at 2am

Most agent runbooks name an engineer. That is half an answer, because most agent incidents become a business decision inside the first hour.

Adopt a two-name rule. Every agent in production carries a named service owner who can revoke, pin and restore, and a named business owner who can decide whether to refund, retract or correct. The first responder pages both, not either. If your rota cannot produce a second name, read the case for hiring a dedicated agent operations engineer.

Vendor escalation is a third path with its own timing, and it belongs in the runbook rather than in somebody's inbox. Record the support channel, the account identifier, the severity language the contract uses, and the response time it promises.

Telling people, and the clock you did not start

Two audiences matter and they run on different timers. The customer who received the wrong answer, and the authority who may need to hear about it.

On the first, the legal position is less open than most teams assume. In February 2024 the British Columbia Civil Resolution Tribunal decided Moffatt v Air Canada, holding the airline liable for a bereavement fare policy its chatbot had described incorrectly. Air Canada argued the chatbot was a separate entity responsible for its own actions. The tribunal rejected that and awarded roughly 650 Canadian dollars.

The money is trivial. The principle is not. Treat every agent output that reaches a customer as a statement by your company, because that is how it has been treated in the one place it has been tested.

On the second audience, the timing is written down. EU AI Act Article 73 requires providers of high-risk AI systems to report serious incidents to the market surveillance authority of the member state where the incident occurred. The outer deadline is 15 days from awareness. It drops to 10 days where a death may be involved, and to 2 days for widespread infringement or disruption of critical infrastructure. Those obligations apply from 2 August 2026, and the wider documentation load sits in the EU AI Act transparency checklist.

Read the trigger carefully, because it is the part that catches teams out. The clock starts once you have established a causal link between the system and the incident, or a reasonable likelihood of one. It does not wait for a completed root cause analysis. So the runbook needs a named person who makes one call inside the first hour: does this look reportable.

None of this is exotic incident management. NIST rewrote its guidance in April 2025, and SP 800-61 Revision 3 reframes the lifecycle around the six functions of the Cybersecurity Framework, treating preparation as continuous. Agent runbooks fit inside that structure rather than needing a parallel one.

Where this template is weakest

Three honest problems with everything above.

First, the severity tiers are judgement rather than measurement. There is no published benchmark that maps classes of agent action to target response times across industries. The numeric thresholds circulating in vendor material usually come from one product category, often voice agents, and do not generalise to a coding agent with write access to a repository.

Second, this template is assembled from public incidents and published survey work, not from our own incident record. We have no first-party data on agent failures. A runbook built from someone else's outages is a starting hypothesis, and yours should look different within two incidents.

Third, two of the four survey figures here are vendor-commissioned and all four are self-reported. Executives who say they could not stop a rogue agent are describing their confidence, not their infrastructure.

The case for not writing one yet

The strongest argument against this whole exercise deserves stating. At pilot scale, with three internal users and no external write access, a formal runbook is ceremony. It costs a day to write, goes stale in a month, and creates the appearance of control without any of it.

The threshold is not the first deployment. It is the first irreversible external action the agent is permitted to take. On the day you grant an agent the ability to send, pay, delete or file, the runbook becomes the cheapest insurance available. Before that day, spend the time on permissions instead.

Frequently asked questions

Where to start

Pick the agent in your estate with the widest write access. Not the most used one. The one that can do the most damage.

On a Thursday afternoon, ask whoever is on call tonight to do exactly one thing: revoke that agent's credentials in production and tell you two numbers. How many minutes it took, and how many downstream systems noticed. Most teams find the first number is unknown and the second is larger than expected. That single test produces more useful runbook content than a week of drafting.

Then write the five lines you did not have. The revoke command. The evidence capture list. The three rollback layers with an honest yes or no against each. The two owner names. And whoever decides, inside the first hour, whether the incident is reportable.

Related reading