What an AI agent audit trail should record

What an AI agent audit trail is for

An audit trail exists so that someone who was not there can reconstruct an event later: what the agent did, who allowed it, and what evidence they had when they decided. That person might be a client asking why an email went out, an engineer writing a post-mortem, or an internal auditor sampling last quarter’s decisions. The trail has done its job if they can answer those questions without interviewing the people involved.

It is not a replacement for application logs. Logs are for debugging, so they are verbose, short-lived and mostly readable by the engineers who wrote them. An audit trail is smaller and more deliberate: it records decisions and state changes in a form that stays meaningful for months and holds up when the person reading it is skeptical.

Record the run and its steps

The unit of work is the run: one execution of an agent against one task. Give each run a stable identifier, and record the agent that ran it, the configuration it ran with, when it started and ended, and its final state. Inside the run, record the steps that matter for accountability, such as each tool call that reads or changes an external system and each model call that costs money.

Step records do not need the full payload. For an example agent that files support tickets, a step record might hold the tool name, the ticket ID it created, the status, the duration and the token count, which shows what happened and in what order without copying the customer’s message into a second system.

Separate what the agent claimed from what was measured

Most of what an agent tells you about itself is a claim. When an agent reports that a run succeeded, cost $0.40 and sent one email, those figures come from the agent’s own code, which can be wrong or unaware of what happened downstream. Some facts can be established independently, for example by a proxy that meters model calls or a supervisor that watched the process exit.

Label every record with where it came from, and keep the label when you display or summarize it. A reviewer should be able to tell at a glance whether a figure was reported by the agent, measured by something outside it, or entered by a person. One combined success badge hides exactly the uncertainty a review is looking for.

• Agent-reported: run status, step names, token counts and costs the agent sends about itself.
• Independently measured: usage metered by a gateway the agent’s calls pass through, or a process exit seen by a supervisor.
• Human-recorded: approvals, denials, outcome reviews and incident acknowledgments.

Record who approved an AI action, and exactly what they approved

An approval record that says “Dana approved the refund” is weak evidence. It does not say which refund or for how much, or whether the refund that went out matched the one Dana saw. A useful record names the person, the time, the decision and the exact action, and binds the decision to that action so it cannot be reused for a different one.

The practical way to bind them is a digest. The agent computes a hash over a canonical form of the action (the tool, the target and the arguments), sends it with the approval request, and must present the same digest when it executes. If the arguments change, the digest changes and the approval no longer applies. Add an expiry and make each approval single use, and the record means what it says.

{
  "type": "approval.decided",
  "approval_id": "apr_7f3c",
  "run_id": "run_0412",
  "agent": "refund-assistant",
  "title": "Refund order 18842 for $64.00",
  "action_digest": "9b1e4c0d...e07a",
  "decision": "approved",
  "decided_by": "dana@example.com",
  "decided_at": "2026-09-22T04:14:09Z",
  "expires_at": "2026-09-22T04:44:00Z",
  "consumed_at": null
}

Record configuration, policy, credential and incident changes

Many incidents trace back to a change nobody connected to them: a raised spending limit, a new tool on an allowlist, a credential that was rotated but never revoked. Record every change to what an agent is allowed to do as a new numbered version, with its contents or a digest of them, who made it and when. Also record what the runner says it enforces, since the saved version and the version in force can differ.

Credentials need their own lifecycle records: created, rotated and revoked, each with who did it. Incidents need theirs too, from detection through acknowledgment to resolution, with the person or process behind each change. An acknowledgment shows that someone was paying attention; it does not show that the problem was fixed, so keep the two as separate entries.

What to leave out of an AI agent audit trail

An audit trail is kept longer and read by more people than most data, which makes it a poor place for anything sensitive. Leave out prompts and model outputs that contain personal data, raw documents the agent processed, and anything that works as a secret, such as API keys, tokens and passwords. If a secret does land in the trail, treat it as exposed and rotate it.

Minimization is easier to design in than to clean up afterwards. Record identifiers and digests instead of contents, and leave the contents in the system that already governs them. For example, record the ticket ID and a digest of the reply rather than the reply text; the digest still lets you confirm later that the text you are shown is the text that was approved.

Retention, deletion and tamper evidence

Decide how long each kind of record is kept, and write it down. Step-level detail is usually needed for weeks, while approvals, policy versions and credential changes may be needed for as long as the contract or client relationship they relate to. Regulated teams should take their periods from their own requirements and confirm that deletion actually runs.

Tamper evidence means that if a record were changed or removed, someone could tell. The plainest forms are permissions that let the application add records but not edit or delete them, versioning where a change adds a new version instead of overwriting the old one, and copies held where the same credentials cannot reach. Stronger forms chain or sign records cryptographically. Know which of these you have; a careful reviewer will ask.

Making the trail useful for a review

Before you need it, pick one real run and check that you can answer these from the records alone:

• Which agent ran, and which configuration and limits were in force at the time?
• Which steps changed an external system, and in what order?
• Who approved each consequential action, when, and was the approved action the one that ran?
• Which figures came from the agent, and which were measured or checked independently?
• Which incidents were open during the run, and who acknowledged them?

Make the trail searchable by run, by time and by person, because reviewers come in from different directions: a client from an outcome, a post-mortem from a time window, and an internal audit from a sample of decisions checked against the policy in force at the time.

What Toolcaise records

Toolcaise supervises AI agents, and its records follow the split described above. Each event, step, usage sample and output check is labeled with who vouches for it, decided by the credential that sent it and never by the record itself: “Reported by the agent” for anything an agent’s own code sends, “Measured by Toolcaise Connect” for what Connect records on your machine while it runs the command or fronts the MCP server (its model calls, tool calls and resource use), “Observed by Toolcaise Connect” for a discovered agent whose process Connect only watches, and “Recorded by Toolcaise” for Toolcaise’s own entries. A report is the agent’s claim, not independent proof, and Connect’s figures are only as trustworthy as the machine it runs on.

• Credentials: creating, rotating and revoking an agent’s credential are recorded with who did it, and so are approving and revoking a Connect device.
• Controls: each pause, resume or cancel command records who issued it, when the runner received it and acknowledged it, and how it was applied when that is known: by the agent’s own code at a checkpoint (cooperative) or by Toolcaise Connect on the agent’s path (enforced). A command replaced before it was acknowledged is recorded as replaced.
• Approvals: the agent sends a digest of the exact action and a short title, not the action itself, and may add a reason, a destination, a count and the fields it chooses to show, redacted and labelled as reported by the agent. The record keeps who approved or denied it, when, and any note they left. An approval expires, can be used once, only by the same run for the same digest, and not while the agent or run is paused or canceled.
• Policy: every change to an agent’s limits creates a new numbered version with its digest and author. The application’s database role can add versions but cannot change or delete them. The version the runner reports it applied, or rejected, is recorded too.
• Incidents: each moves between open, acknowledged, resolved and recovered, with a history of who or what changed its state and when.
• Outcomes: the agent’s reported status is kept apart from a person’s accept or reject review, and a retry cannot rewrite an outcome once it has been reported.

Run history is kept for 7 days on the Free plan, 30 days on Team and 90 days on Business. Once a run is older than that, it becomes eligible for deletion, and its events, steps, outcomes, control commands and approvals go with it. The decision ledger is kept: every approval request and decision (title, run, action digest, who decided, when, the expiry and whether it was used), every control command from request to delivery, acknowledgement or replacement, and every recovery authorization and claim, recorded as it happens in a table the application can add to but not change. Those entries outlast the run they belonged to, as do agents, credentials, policy versions, incidents and workspace-level audit entries. Deleting an agent removes its records, its ledger entries and policy versions included.

What Toolcaise does not cover

Toolcaise does not prove that an external action succeeded. An approval record shows that a person allowed a specific action and that the runner used the approval once; whether the refund reached the customer is a fact in your payment system, and you should check it there. Agent-reported runs, costs and outcomes are only as accurate as the code that sends them.

Approvals and remote controls through the Python and Node.js SDKs are cooperative: they work at checkpoints your code implements, and code that skips a checkpoint is not stopped. Toolcaise does not sign or hash-chain its records, and it does not claim any compliance certification. If you work under a regulatory regime, treat these records as one input to your evidence and confirm the requirements with your compliance owner.