Auditing an agent is not saving its prompt: it is reconstructing the decision

Agent traceability is useful only when a reviewer can reconstruct which evidence entered, which policy and version decided, who authorized the action, and what outcome followed without replaying the full conversation. Saving every prompt may look comprehensive, yet it mixes irrelevant context, secrets, and personal data while leaving the central question unanswered: why was a business action allowed?

The auditable unit is the decision

A technical log says that a call ended. A decision record explains the relationship among a business need, available evidence, applicable rule, authority, and effect. That is the difference between knowing an agent invoked a tool and knowing why it changed a bank account, blocked a payment, or recommended an exception.

The audit and accountability control family in NIST SP 800-53 Rev. 5 treats records as evidence with defined content: event type, time, location, source, outcome, and associated identity. That principle is more useful than indiscriminate transcription. A record should answer control questions, not accumulate text by reflex.

A minimum evidence envelope

The envelope accompanies each material decision and retains verifiable references. Its contract can include:

Field	Question answered
decision_id and object_ref	Which decision and business object are under review?
input_refs, hash, and version	Which invoices, orders, events, or data were seen?
observed_at and effective_at	When was each fact known and when was it true?
policy_id and policy_version	Which rule authorized or stopped the action?
model_or_rule_version	Which component produced the proposal?
tool_call_refs	Which systems were read or changed?
proposed_action and reason_codes	What was proposed and for which normalized reasons?
actor, reviewer, and approver	Who proposed, reviewed, and authorized it?
timestamps and state_transition	How did it move through the workflow?
outcome_ref	What actually happened after authorization?
supersedes and appeal_ref	Which correction or appeal changed the record?
classification and retention	Who can see it and how long is it kept?

Not every field contains original data. input_refs can point to a protected version with a hash, access control, and retention policy. A reviewer verifies integrity and retrieves the document only when authorized.

Four evidence classes that must not be confused

Source evidence identifies documents, events, and versions entering the decision. Execution evidence records tools, responses, errors, and retries. Authority evidence shows permissions, segregation of duties, and approval. Outcome evidence confirms whether the change reached the destination and what effect it produced.

An agent can reason correctly and fail to write; it can also execute a technically valid operation without authority. A single “success” state cannot distinguish these cases. The timeline must preserve each transition and link it to the right evidence class.

Provenance without copying the universe

The W3C PROV-O Recommendation supplies a practical grammar: Entity for an invoice or policy version, Activity for a comparison or approval, and Agent for the responsible person or software. Relationships such as used, wasGeneratedBy, and wasAssociatedWith reconstruct the chain without embedding every item of content in the event.

For example, a payment proposal is an entity generated by a matching activity that used specific invoice, purchase-order, and receipt versions. Approval is another activity associated with an authorized person. The executed payment is a later entity derived from the approved proposal. This chain answers who did what with which evidence.

Illustrative example: supplier bank-account change

This scenario is illustrative. An email requests a change to a supplier's bank account. The agent extracts the identifier, links the message to the supplier business object, and proposes an update. Policy requires verification through a registered channel and approval by someone other than the requester.

The envelope stores references to the email and previous profile, their hashes and versions; records the active policy; marks the proposal as prepared but not authorized; and retains the verifier and approver identities. Only then does it link the update identifier from the ERP. If a bank later rejects a transfer, outcome_ref connects that result to the decision.

Saving only the prompt “update this account” would not demonstrate channel verification, separation of duties, or ERP acceptance. Copying the entire email into every log would multiply exposure without improving that control.

Version policy and components separately

policy_version describes the business rule: tolerance, authority, and escalation condition. model_or_rule_version identifies the component that extracted, classified, or proposed. Separation prevents attributing to a model a decision actually made by a rule, or assuming that a prompt change altered an approved policy.

Maintain a deployable registry of policies and components with effective date, owner, tests, and status. The envelope points to immutable identifiers. If a policy changes tomorrow, yesterday's audit retains the version that actually governed the action.

Reasons should also use stable codes with a readable explanation. “PO_AMOUNT_MISMATCH” can be aggregated and measured; free prose does not replace the code, though it can help a reviewer.

Privacy, security, and retention are part of design

The OWASP Logging Cheat Sheet warns against recording access tokens, passwords, secrets, and sensitive personal data directly; they should be excluded, masked, sanitized, or encrypted as appropriate. The envelope requires field classification, role-based access, encryption, integrity protection, and purpose-bound retention.

Minimization does not mean losing evidence. It means retaining sufficient references, fingerprints, reasons, and metadata; keeping sensitive content in its authorized system; and auditing every retrieval. A hash proves a referenced item did not change, but it does not prove the source was correct. Authority, provenance, and validation remain necessary.

Representing the envelope in Quantum

In Quantum Automation Center, the business object can anchor the record. Its timeline links executions and transitions; artifacts and logs contribute protected evidence; permissions and approvals represent authority; and the catalog identifies agent, tool, and policy versions. The Quantum ontology relates those elements without reducing them to a transcript.

The review interface should lead with decision, risk, reasons, sources, and approver. Technical detail remains available on demand. Operations, audit, and compliance can then inspect the same record through views aligned with their permissions.

Counterargument: the envelope also has a cost

A detailed envelope adds storage, latency, and operational burden; the answer is to tier evidence by risk instead of applying equal depth to a reversible suggestion and a financial action. Define levels: an informational recommendation may need sources and version; a reversible write adds authorization and outcome; a sensitive action requires segregation, stronger evidence, and review.

Compare cost with the question the control must answer. If a field will never be reviewed and serves no investigation, measurement, or appeal, it probably does not belong. If an action can affect money, rights, or compliance, “we have the chat” is probably insufficient.

When not to create an additional record

Do not create a separate case file for low-risk ephemeral recommendations already covered by the system-of-record history, or retain data whose storage is prohibited or lacks a defined purpose. Link existing evidence instead of duplicating it.

Do not confuse the envelope with a legal retention policy, a digital signature, or automatic compliance certification. Each obligation depends on context and jurisdiction. The technical contract helps demonstrate controls; it does not decide by itself what must be retained.

The reconstruction test

Select a material decision and hand it to a reviewer who did not participate. They should identify inputs and versions, reproduce the applicable rule, verify authority, follow the transition, locate the result, and understand any correction without asking for the full prompt. If they can only read a long conversation or trust a screenshot, activity was recorded, but decision traceability is still missing.

Sources

• NIST SP 800-53 Rev. 5 — Security and Privacy Controls — nist.gov
• W3C PROV-O: The PROV Ontology — w3.org
• OWASP Logging Cheat Sheet — owasp.org