How to design human approval for AI agent actions

What human in the loop means for an agent

For an AI agent, human in the loop usually means one specific thing: before a particular action executes, the agent stops and waits for a person to allow it. That is different from reading transcripts afterwards or reviewing the finished output. Both are useful, but only an approval sits in front of the side effect.

An approval is a control in your code, not an instruction in the prompt. A system prompt that says ‘ask before sending email’ is a request the model may or may not follow. A gate inside the function that sends email is something the model can’t talk its way past.

Which actions need approval

Approval costs a person’s attention, so spend it where a mistake is expensive. Four kinds of action usually qualify:

• Irreversible: deleting records, sending messages, merging to a protected branch, issuing refunds. Anything you can’t undo.
• External: actions that leave your system and reach customers, partners or the public, such as emails, posts, tickets or writes to a third-party API.
• Costly: actions that spend money or start expensive work, such as purchases, large paid API batches or new infrastructure.
• Sensitive: actions that touch personal data, permissions, credentials or anything with legal weight.

Most agent steps are none of these. Searching, reading, drafting and writing to a scratch location can run freely. A good test is whether you would be comfortable explaining the action to the affected person if it had gone wrong and nobody had looked at it first.

Approve the exact action, not the intent

The weakest form of approval is a yes to a description, such as ‘OK to send the weekly digest?’ Between that yes and the send, the agent can change the recipient list, pick a different draft or retry with new parameters, and the approval still appears to cover it. The person approved an intent, and the system executed something else.

A sound approval binds the decision to the concrete operation:

• Parameters: every value that affects execution (recipients, amounts, record IDs, the draft) is part of what is approved.
• Digest: the parameters are serialized in a canonical form and hashed, so any change produces a different request that needs its own decision.
• Execution from the snapshot: the code runs with the exact values that were hashed, not with variables that may have changed since.
• Single use: the approval is consumed when the action runs, so a retry or replay can’t reuse it.
• Expiry: an unanswered request lapses, so a decision made tomorrow doesn’t authorize a stale action.

The request also has to be readable. The approver should see what matters for the decision, for example ‘Refund 48.00 USD on invoice INV-2291’, without secrets or personal data the decision doesn’t need.

Where the checkpoint sits in code

Place the gate immediately before the side effect, after every parameter has been computed. If the agent still has work to do after approval that could change what gets sent, the approval came too early.

Put it in the function that performs the action rather than in each caller. If three code paths can send an email, a gate in the send function covers all three, while a gate in one planning step covers one. For tool-using agents, the natural place is the tool implementation or a proxy in front of it, because every decision the model makes to use that tool passes through there.

What happens on deny, timeout or error

Design the unhappy paths first, because they decide whether the control is real. The rule is to fail closed: if the system can’t confirm an approval, the action doesn’t run.

• Denied: the action is skipped, the denial is recorded with who made it, and the agent follows a defined branch, such as ending the run or moving to the next item.
• Timed out: treat it exactly like a denial. Silence is not consent.
• Approval service unreachable: also a denial. A network error must never be a way through.
• Run cancelled while waiting: the request should lapse with it, so a late approval can’t authorize an action nothing is waiting for.

Then decide who picks up the pending work. A denied or expired action is often still needed in some form, and it needs an owner. Be careful with automatic retries: a loop that files a fresh request until someone clicks approve turns a control into noise.

Approval fatigue and how to scope approvals

If every action needs approval, people stop reading the requests and start clicking approve. At that point the control records consent without providing any. Keep approvals rare enough that each one gets real attention. Some ways to scope them (examples, not rules):

• Use thresholds: require approval for refunds over a set amount, or for email to recipients outside your own domain, and let the rest run.
• Approve a batch as one unit: a single decision on an exact list of 30 records, hashed together, beats 30 separate prompts.
• Watch the pattern: if a type of request is approved every time, consider replacing it with a rule; if it is usually denied, fix the agent.
• Keep requests short and specific, so a decision takes seconds rather than an investigation.

Who should approve

The approver should be the person accountable for the consequence and able to judge it, which is often not the engineer who built the agent. For example, a finance lead approves refunds and a support lead approves messages to customers.

Two more rules help. The agent, and anything acting with its credential, must not be able to approve its own requests. And requests need cover: decide who approves when the usual person is away or in another time zone, and set expiry times that match how quickly someone will realistically respond.

How to test the approval path

An approval path that has only ever been approved is untested. Use a harmless action, such as posting to a test channel, and check each of these:

1. Approve once: the action runs exactly once, with the approved values.
2. Deny: nothing runs, and the denial is recorded.
3. Let the request expire: nothing runs.
4. Change a parameter after approval: the change needs a new decision.
5. Replay a used approval: it is refused.
6. Block the approval service: the action fails closed.
7. Cancel the run while a request is waiting: the request lapses and nothing runs.

How Toolcaise handles action approvals

In the Toolcaise Python and Node.js SDKs, the gate is guarded_action (guardedAction in Node). You pass a short title, the action as a JSON value and a function to execute. The SDK hashes the action with SHA-256, asks for approval, waits, consumes the approval once and only then calls your function with a frozen copy of the exact action it hashed. Toolcaise stores the digest, your title and what you choose to tell the person deciding: why approval is needed (reason), what the action writes to (destination), how many things it affects (affected_count) and the fields of the action to show (show_fields). The SDK takes those fields’ values from the snapshot it hashed and replaces credential-like text and email addresses with [redacted] before anything is sent. The rest of the action never leaves your process.

from toolcaise import Toolcaise

client = Toolcaise()

with client.run("invoice-refunds"):
    refund = {"operation": "refund", "invoice": "INV-2291", "amount_cents": 4800, "currency": "USD"}
    client.checkpoint()                  # waits here while a person has paused the agent
    try:
        client.guarded_action(
            "Refund INV-2291: 48.00 USD to the original card",
            refund,
            lambda approved: payments.refund(
                approved["invoice"], approved["amount_cents"], approved["currency"]
            ),
            timeout=1800,                # the request expires after 30 minutes
            reason="Refunds over 25 USD need a finance lead",
            destination="Payments: refunds",
            show_fields=["invoice", "amount_cents", "currency"],   # the person sees these values
        )
    except RuntimeError:                 # denied, expired, cancelled or unconfirmed
        client.event("refund.not_issued", "Refund not approved; left for follow-up", level="warning")
        raise
    client.outcome("refund issued", status="succeeded")

Workspace owners and admins approve or deny the request in the dashboard. A denial, an expired request, a digest or run mismatch, or any failure to reach Toolcaise raises, and nothing runs. The request expires after the timeout you pass (one hour by default, between one minute and 24 hours), and a pending request lapses when its run ends. If someone pauses the agent after approving, the action waits at a checkpoint until it is resumed, and a cancel ends it. Requests, decisions and each use of an approval are written to the audit trail.

Beside each request the dashboard shows what the agent reported, labelled as reported by the agent. When the agent’s last decided request with the same title and destination had a different digest, it says the parameters changed since that approval or denial and shows which of the shared fields differ. When the same digest was approved before, it says so and links to it. The person deciding can leave a note with the decision, and alert destinations can be told when a request arrives.

checkpoint() is the related control for pause and cancel: call it between safe steps, and it waits while a person has paused the agent and raises AgentCancelled when the run is cancelled.

Limits of these approvals

SDK approvals are cooperative. They protect the actions your code routes through guarded_action; a code path that calls the side effect directly is not stopped. For a hard limit on runtime, cost or tokens, run the agent under toolcaise-connect run, which enforces limits outside the agent’s code.

The approver sees your title and the fields you choose to show, as your agent reported them. The digest guarantees that what runs is exactly what was requested. It doesn’t guarantee that the title or the shown fields describe it honestly: Toolcaise cannot check them against the digest. Take the fields from the action, as show_fields does, and review the code that builds the title.

An approval is not proof that the action succeeded. It records that a person allowed it. The payment provider or mail server can still fail or partly apply the change, so report the result as an outcome and confirm it where it landed.