What to automate first: prioritize exception cost and reversibility

The best first automation is not the most visible idea; it is the one that removes costly exceptions through reversible actions supported by sufficient evidence. This thesis can be disproved with data: if an executive-sponsored idea beats another only in narrative appeal while failing those three factors, it should wait or remain an experiment.

Why idea lists produce poor decisions

Organizations commonly collect use cases in workshops. Each function describes manual hours, frustration, and “AI potential.” The resulting list contains incomparable units: one idea counts transactions, another counts whole salaries, and a third claims a commercial opportunity with no baseline. The most persuasive sponsor wins.

Technical possibility is also confused with operational suitability. Classifying an email may be easy, but if a wrong classification triggers payment or denies service, exception cost dominates the saving. Conversely, a less visible task may release an expensive queue with actions that are easy to reverse.

Three questions before scoring

Define the object and decision first. Does the automation affect an invoice, order, shipment, or request? Does it classify, recommend, write, or approve? Without those answers, “automate procurement” is not an assessable use case.

Next, measure the current exception: volume, waiting time, rework, loss, or risk. Use observed data and separate per-case cost from total cost. Third, define the reverse path: what happens when the output is wrong, who can stop it, and whether the action can be undone before harm.

The NIST AI RMF Core says risk-management activity should reflect organizational tolerance and risk priority. It does not provide an automation-selection spreadsheet; it supports the principle that controls should be proportional to risk rather than excitement.

A weighted matrix: configurable example, not benchmark

The following scale and weights are strictly illustrative. They do not come from an external study. Portfolio owners should approve them and test them against past decisions. Score each dimension from 0 to 5, with 5 more favorable for an initial implementation.

Dimension	Illustrative weight	Verifiable question
Avoidable exception cost	30	Is there a baseline for rework, delay, loss, or risk that the case reduces?
Reversibility	25	Can an output be stopped or undone before a serious consequence?
Evidence and data quality	20	Are there authorized sources, identifiers, and a reference truth?
Repetition and stability	15	Is the decision repeated across a sufficiently stable population and rule set?
Integration ease	10	Can it be instrumented without fragile access or dangerous direct writes?

The illustrative formula is:

Score = Σ (0–5 rating × weight) / 5

The maximum of 100 ranks this portfolio only. It is not a probability of success or a return estimate. Publish ratings and evidence alongside the total so disagreement remains visible.

Worked example with visible assumptions

Assume, only to demonstrate the method, three internal candidates:

• A: classify and route requests. Exception cost 4, reversibility 5, evidence 4, repetition 5, integration 4. Illustrative result: 440/5 = 88.
• B: automatically approve credit notes. Cost 5, reversibility 1, evidence 3, repetition 3, integration 2. Result: 300/5 = 60.
• C: summarize contracts for review. Cost 2, reversibility 5, evidence 3, repetition 4, integration 5. Result: 355/5 = 71.

Under these assumptions, A deserves an operational test first. B may offer greater gross value, but the action is difficult to reverse; redesign it as a recommendation with human approval. C is safe to explore but needs a stronger outcome hypothesis. Different weights or ratings change the order. That sensitivity is a feature, not a flaw.

The GAO Cost Estimating and Assessment Guide calls for purpose, scope, technical baseline, assumptions, data, sensitivity analysis, and updates with actual costs in a reliable estimate. Apply that discipline: record source, date, range, and owner for each input, then replace assumptions with observations from the pilot.

Veto gates no score can overcome

An idea cannot enter production, however high its score, when any of these conditions holds:

• no legal or contractual right exists to use the data;
• nobody accepts ownership of the outcome and exception queue;
• an irreversible action has no independent approval;
• there is no reference truth for evaluation;
• integration requires shared credentials or non-idempotent direct writes;
• no review capacity exists during the promised service window.

The UK Government AI Playbook recommends meaningful human control at appropriate stages, full lifecycle management, and using the right tool for the job. An idea may therefore be approved as deterministic automation or an assistant rather than an autonomous agent.

Turn the ranking into an evidence-buying experiment

Take the leading candidate and state an object, population, decision, boundary, and outcome hypothesis. Design a reversible test that produces evidence for uncertain dimensions. If data quality scored 2, do not build the polished interface first; measure coverage, conflicts, and age. If reversibility scored 1, convert execution into a proposal requiring approval.

In Quantum Automation Center, the catalog can preserve candidate, owner, status, and decision; executions and timelines show behavior; artifacts and logs gather evidence; and analytics can compare baseline with observed outcomes. The Automation Center should record the decision, not replace the analysis.

Counterargument: the matrix penalizes transformative bets

That criticism is valid. Novel ideas lack historical data and may score poorly precisely because they are different. A portfolio governed only by this matrix converges on small improvements.

Create two lanes. The operating lane uses the score to decide production priority. The exploration lane buys learning with explicit budget, duration, and stop criteria; it promises neither ROI nor autonomy. A strategic option can enter that lane, gather evidence, and return to the matrix. Uncertainty receives room without masquerading as maturity.

When not to use this matrix

Do not use the score as automatic approval. Do not rank work mandated by law, security, or continuity alongside discretionary ideas; mandatory work has a separate intake and still requires an appropriate control design. Do not combine outcomes measured on incompatible time horizons without stating the difference.

When the portfolio has only a few cases and the decision is obvious, a documented discussion may be enough. Preserve at least exception cost, reversibility, evidence, ownership, and the rejection rationale.

The useful decision is not the final number

A good review ends when the team can explain why one idea ranks higher, which assumption could reverse the order, and what evidence it will acquire first. Store the matrix version and rescore after the pilot. If the score never moves when real evidence arrives, it has become ceremony. If it moves with a traceable reason, it is working as a decision instrument.

Sources

• NIST AI Risk Management Framework Core — nist.gov
• AI Playbook for the UK Government — gov.uk
• GAO Cost Estimating and Assessment Guide — gao.gov