From pilot to operations: the readiness test an AI agent must pass

An agent is not production-ready because it performs well in a demo; it is ready when every decision has defined authority and every failure mode has an owner who can intervene. The test is concrete: after a wrong output, if the team cannot say who stops, corrects, communicates, and authorizes restart, it still has an experiment.

A demo validates capability; production requires accountability

A pilot usually proves that a model can understand a request, query a tool, or propose an action. It answers, “Can this work at least once?” Operations needs a different question: “Can we accept, detect, and recover from its failures throughout its life cycle?”

The gap is not closed by a better average accuracy score. An agent may be right most of the time and still fail on the case that moves money, changes access, or damages a customer relationship. Control therefore belongs at the decision level: what evidence enables it, how far authority extends, and who inherits the case when it crosses a boundary.

Eight production-readiness gates

Google SRE’s Production Readiness Review verifies operating standards, prepares service owners, and transfers responsibility progressively. AWS describes its Operational Readiness Review as both process and checklist, to be run before launch and repeated through the lifecycle. Adapted to agents, those ideas yield eight gates:

1. Decision right. The agent’s permission to recommend, execute, approve, or never perform each action is documented.
2. Failure owner. Every failure class has an accountable team, contact path, and target for first action.
3. Input boundary. Allowed sources, required data, maximum age, and conflict behavior are explicit.
4. Evaluation. Representative and adversarial cases, acceptance criteria, and tested version are recorded.
5. Observability. Business object, decision, tool, result, latency, exception, and evidence are traceable.
6. Human escalation. Queue, delivered context, priority, and reviewer authority are defined.
7. Rollback and pause. The team has tested how to stop actions, reverse where possible, and enter a degraded mode.
8. Cost and continuity. Operating budget, consumption limits, external dependencies, runbook, and retirement condition exist.

A verbal “yes” does not open a gate. It needs a link to a policy, test, dashboard, runbook, or record.

Decision-right matrix: proposing is not approving

Assign authority by action and consequence rather than a blanket “autonomy level.”

Action	Agent	Operator	Business owner
Classify a reversible request	executes within bounds	reviews samples and exceptions	accepts policy
Recommend a financial adjustment	proposes with evidence	validates data	approves or rejects
Publish, pay, or revoke access	prepares case	checks requirements	authorizes irreversible action
Change a policy	not authorized	analyzes impact	approves change and effective date

The NIST AI RMF Core calls for clear roles and communication lines, distinguishes human responsibilities in human-AI configurations, and places responsibility for AI risk decisions with executive leadership. The matrix turns that expectation into enforceable permissions.

Walk a failure path, not another happy path

Strictly illustrative example: an agent prepares a response to an invoice dispute. It finds the document, but the ERP amount differs from the PDF. A demo may select the newer source and continue. An operational review injects that conflict on purpose.

The approved behavior would be to flag a data exception; make no ERP change; attach both permitted references; assign a reason code; route the item to the accounts-payable owner; and start an escalation clock. When the operator resolves the conflict, the record preserves who decided, which evidence was used, and which policy version applied. If the queue breaches its internal threshold, the service enters its agreed degraded mode: classification only, no downstream action.

This exercise reveals more than a hundred fluent responses. It tests the boundary, context transfer, human authority, and recovery path.

Approval rule and minimum evidence

An agent passes only when all eight gates have current evidence and no irreversible action lacks a named approver. The process owner must also accept residual risk explicitly: failures that remain possible after controls.

For each version, retain at least the dated evaluation set; enabled configuration and tools; results by case type rather than only an average; known exceptions; decision-right matrix; runbook; owners; approvals; and the launch, limit, or reject decision. Do not indiscriminately retain sensitive data or entire conversations. Evidence should be sufficient and proportionate.

What this looks like in Quantum

Quantum Automation Center can represent the agent in a catalog, show execution status and timelines, associate permitted artifacts and logs, and apply controlled permissions or human approval. The useful operating view connects those elements: from an affected business object to the decision, exception, and responsible person. See the AI agent pattern and Automation Center.

The platform does not make the organizational choices. The business still defines authority; technology defines degraded mode; security defines access; and operations owns incident response.

Counterargument: eight gates can suffocate experimentation

They can, when every gate carries the same burden for every use case. An assistant summarizing internal documents for human review does not need the package required for an agent that can release a payment. Disproportionate review simply moves risk: teams create shadow pilots to avoid it.

The answer is to vary depth, not remove gates. A reversible case may use lightweight tests and business-hours support. A financial action requires separated authority, stronger evidence, and defined response. Both still need an owner and a tested pause control.

When not to use an agent in production

Do not promote it when the process is deterministic and a rule or conventional automation provides a more testable result. Do not promote it without people who can review the exception queue, sources with adequate rights and quality, or somebody willing to accept accountability for the outcome.

The correct decision may be to keep the agent assistive: it proposes but cannot act. If even a proposal could steer a consequential decision without competent review, stop the use case and repair the operating design first.

The meeting that makes the real decision

Bring together the business owner, operations, security, and technical team. Choose one consequential decision and one plausible failure. Walk through who detects it, who receives it, what evidence they see, what they may do, when it escalates, and who authorizes restart. Whenever someone says “the team would look at it,” replace that phrase with a name, queue, permission, and runbook. That precision—not demo fluency—is the line between pilot and operations.

Sources

• Google SRE — Production Readiness Reviews — sre.google
• AWS Well-Architected — Ensure a consistent review of operational readiness — aws.amazon.com
• NIST AI Risk Management Framework Core — nist.gov