Approving an AI agent’s action does not prove it happened

The gap between approved and done

Suppose a support agent proposes a $45 refund on order 1042 and a person clicks approve. A week later the customer says the money never arrived. The approval record is real and correct, and it tells you nothing about what happened next. This is an invented example, but the pattern is common: the person decided well, and the evidence that the action happened correctly was never collected.

Approval answers whether an action may happen. Verification answers whether it did happen, once, with the intended effect. Treating the first as the second is how agent workflows end up with confident records of work that is missing or duplicated.

Four records between a request and a result

Between the proposal and the outcome there are at least four separate facts, and a different party produces each one:

1. Authorization: a person, or a policy, allowed this exact action. It comes from the reviewer.
2. Runner acknowledgement: the process that executes the action received the go-ahead and started. It comes from your runner.
3. The external system’s receipt: the destination accepted the request and returned something, such as a refund ID, a message ID or a created record. It comes from the destination, relayed by your code.
4. Independent verification: a later check against the destination shows the intended state. It comes from a read that does not depend on the agent’s account of events.

Each can exist without the next. An approval can be followed by a crash before the call. A request can time out and leave no receipt although the destination processed it. A receipt can be followed by an asynchronous failure, such as a reversed payment or a bounced email, that only a later check reveals.

Why each one needs its own record

When the four collapse into one success flag, you lose the ability to say what you know. A run marked “succeeded” might mean the agent believed it had finished, that the destination confirmed it, or that someone checked. During an incident those are very different starting points.

Separate records also keep sources honest. The approval belongs to the reviewer, the acknowledgement to the runner, the receipt to the destination and the verification to whoever ran the check. An agent’s own statement that it completed the task is none of these. It is a claim, and a language model can state that it did something it did not do. Store it, label it as reported, and do not let it stand in for a receipt.

Make the approval name the exact action

Verification only works against a precise intent. If the reviewer approved “refund the customer”, there is nothing specific to check afterwards. Approve the concrete parameters instead: operation, target, amount, recipients and an idempotency key.

Bind the approval to those parameters so the runner cannot execute something different. A common technique is to hash a canonical form of the action and store the digest with the decision, so the runner can prove it is executing what was approved. Make approvals single use and give them an expiry, so one yes cannot authorize a second refund next week.

How to verify AI agent actions with read-back, reconciliation and idempotency keys

Choose the check according to the consequence. For example:

• Read-back. After a write, query the destination for the record and compare it with the approved parameters. Where you can, read through a different path or credential from the one the agent used to write.
• Reconciliation. For batches, regularly compare what was authorized with what the destination shows: approved refunds against settled refunds, or scheduled emails against the delivery log. Differences in either direction are findings.
• Idempotency keys. Where the destination supports them, send a unique key with each consequential request, so a retry after a timeout returns the original result instead of repeating the action. Put the key in the approved parameters so it is part of what was authorized.
• Receipt capture. Store the destination’s identifier for the action, not the agent’s summary of it. The identifier is what makes a later read-back possible.

Verification does not have to be exhaustive. A daily read-back of a sample catches systematic failures cheaply, while consequential one-off actions deserve a check every time.

What to do when verification fails or cannot run

A failed check and a missing check are different results, and both need a recorded next step. As an example of a reasonable order:

1. If the check could not run, record the state as unknown rather than failed. Do not treat a timeout as proof that nothing happened.
2. Before any retry, query the destination using the idempotency key or receipt ID if you have one.
3. If the action did not happen, retry with the same key, so a hidden success cannot turn into a duplicate.
4. If it happened incorrectly, correct it with a new, separately approved action such as a reversal, rather than editing the record of the first one.
5. If the same check fails repeatedly, open an incident and add a test that exercises that failure path.

What Toolcaise records when an action is approved

In the Python and Node.js SDKs, guarded_action (guardedAction in Node) asks a person to approve an exact action before your code runs it. The SDK hashes a canonical JSON form of the action with SHA-256 and sends that digest and your title, plus only the fields of the action you choose to show the person deciding, redacted. The rest of the parameters stay in your process. Once approved, the SDK consumes the approval and calls your function with a frozen copy of the action that was hashed.

from toolcaise import Toolcaise

client = Toolcaise()

with client.run("refund-agent"):
    refund = client.guarded_action(
        "Refund order 1042 in full",
        {"operation": "refund", "order": "1042", "amount_cents": 4500,
         "idempotency_key": "refund-1042"},
        lambda approved: issue_refund(approved),     # your code calls the payment API
        show_fields=["order", "amount_cents"],       # what the person deciding sees
    )
    with client.span("verify refund", kind="tool", tool="payments_api"):
        confirmed = read_back_refund(refund["id"])   # your code, a separate read
    client.outcome("refund order 1042", status="succeeded" if confirmed else "unknown")

• An approval is single use. The runner consumes it once, the server marks it consumed, and consumption is never retried.
• It is bound to the run and the digest. A denial, an expiry, a mismatch or any uncertainty raises and nothing runs, and controls fail closed when Toolcaise cannot be reached.
• A pause or cancel issued after the approval stops it from being used.
• The request, the decision with who made it and when, and the consumption are written to the control audit trail.
• Pause, resume and cancel commands are tracked separately as requested, delivered and acknowledged, with the state the runner says it applied.

For MCP tools run through toolcaise-connect mcp with --require-approval, the hold is bound to the tool and its arguments, hashed. The arguments never leave the machine, and the approver sees the tool name.

Agent-reported outcomes and human review

After the action, the agent can report an outcome with a status of succeeded, failed or unknown, and optionally the SHA-256 digest of an artifact. Toolcaise stores it as agent-reported, and a later report cannot rewrite it. A person then accepts or rejects the outcome in outcome review, and the decision is recorded with the reviewer and the time, separately from the agent’s status.

The provenance labels carry the distinction. Metrics and outcomes that an agent’s own code sends are marked agent-reported, figures Toolcaise Connect records while it runs the agent are marked measured by Connect, and records in the supervision workflows carry an evidence origin that separates agent-reported, human-reported and human-reviewed evidence. In the example above the read-back runs in your code, so its result still reaches Toolcaise as the agent’s report.

What Toolcaise verifies, and what it does not

A job can have an independent check: Toolcaise reads the destination through a read-only connection you set up, such as the CRM's API, on the job's schedule, and records whether the expected result is there beside what the agent reported. That is the only time Toolcaise contacts a destination system.

Otherwise it records that an approval was granted and consumed, what the agent reported and what a person decided. It does not confirm that a refund settled, an email arrived or a record changed, and acceptance in outcome review records a human decision, not proof of delivery.

• SDK approvals are cooperative. Code that calls the destination without going through guarded_action is not gated.
• The reviewer sees your title and the fields you choose to show, as your agent reported them. Toolcaise cannot check either against the digest, so take the fields from the action and write titles that state what matters.
• The MCP gateway gates by tool name. An approval hold is how to put a person in front of calls with specific arguments.

Where a job cannot have a check, checking the destination remains your code’s job, and the verification design above is where to put it.