Instrument the agent you already run

Install

Both SDKs have no dependencies: Python 3.10 or later with the standard library, or Node.js 20 or later. They implement the HTTP API, so anything they do can also be done with plain requests.

The toolcaise packages for PyPI and npm are prepared but not published yet. Until they are, download the SDK as one file and keep it beside your agent; it is the same code the packages will contain.

curl -fsSLO https://toolcaise.com/downloads/toolcaise_agent.py   # then: from toolcaise_agent import Toolcaise

curl -fsSLO https://toolcaise.com/downloads/toolcaise-agent.mjs   # then: import { Toolcaise } from "./toolcaise-agent.mjs"

Then check that Toolcaise accepts the agent's credential:

export TOOLCAISE_AGENT_TOKEN=...   # from Dashboard, Agents
python toolcaise_agent.py check    # or: node toolcaise-agent.mjs check

Configure

A client is one agent: the credential decides which workspace and which agent every report belongs to. Create one client per process and reuse it.

A missing credential does not stop your agent: the client warns once, reports nothing, and its checkpoints and guarded actions refuse to run. In strict mode it raises instead.

Report a run

Wrap each unit of work in a run. The run reports its start, its end (completed, failed, or canceled when a person cancelled it) and a heartbeat every 60 seconds. Inside it, spans time the steps; nested spans record their parent automatically; events and outcomes need no run ID. Exceptions are recorded as a failed status, never with their text, and are raised to your code unchanged.

from toolcaise_agent import Toolcaise
from openai import OpenAI

client = Toolcaise()                                   # reads TOOLCAISE_AGENT_TOKEN
openai = client.instrument_openai(OpenAI())            # optional: record model calls

with client.run("nightly-research") as run_id:
    with client.span("collect sources", kind="tool", tool="web_search"):
        sources = collect_sources()
    with client.span("write report"):
        openai.chat.completions.create(model="gpt-4.1", messages=[...])
    client.outcome("research report", status="succeeded")

import { Toolcaise } from "./toolcaise-agent.mjs";
import OpenAI from "openai";

const client = new Toolcaise(); // reads TOOLCAISE_AGENT_TOKEN
const openai = client.instrumentOpenAI(new OpenAI()); // optional: record model calls

await client.run("nightly-research", async (runId) => {
  const sources = await client.span("collect sources", () => collectSources(), {
    kind: "tool",
    tool: "web_search",
  });
  await client.span("write report", () =>
    openai.chat.completions.create({ model: "gpt-4.1", messages: [/* ... */] }),
  );
  await client.outcome("research report", { status: "succeeded" });
});

The current run follows your code: through async/await and asyncio tasks in Python (contextvars), and through promises, timers and callbacks in Node (AsyncLocalStorage). A plain Python thread does not inherit it; pass run_id= explicitly there, or start the thread with contextvars.copy_context().run. In async Python, use async with client.run(...) and async with client.span(...).

Values known only when a step finishes, such as token counts from a provider Toolcaise does not instrument, can be added to the open span with client.update_span(input_tokens=..., output_tokens=..., model=...) (client.updateSpan({ inputTokens, outputTokens, model })).

When Toolcaise is unreachable

Reporting is fail-open. Runs, spans, events, outcomes, heartbeats, metrics and output checks are queued in memory and delivered in the background (a thread in Python, the event loop in Node), so a slow or unreachable Toolcaise never raises into your code and never makes it wait. Consecutive reports travel together in one request when the server would treat them exactly as it would separately, which keeps a busy agent inside its 60 requests a minute.

Warnings go to the toolcaise logger in Python and are process warnings of type ToolcaiseWarning in Node (visible on stderr unless you run with --no-warnings). client.delivery_stats() (deliveryStats()) returns how many reports are queued, sent and dropped, and client.flush(timeout) (await client.flush({ timeoutMs })) waits for the queue to drain.

Controls are fail-closed in every mode. checkpoint() and guarded_action() talk to Toolcaise directly and raise (reject) if they cannot confirm the decision, so an unapproved action never runs because Toolcaise was down. Before asking, they wait up to one request timeout for queued reports, because Toolcaise refuses controls for a run it has not heard of.

Names in Python and Node

The two SDKs have the same methods and options, each spelled in its language's style: Python uses snake_case and seconds, Node uses camelCase and milliseconds with an Ms suffix. Positional arguments are the same; Python takes the rest as keywords and Node as a final options object.

Method reference

Signatures are Python first, then Node. Methods marked reporting are fail-open (they return None / resolve null once queued, or the server's answer in strict mode); controls always raise on failure.

Runs and steps

Reports

Delivery

Controls, instrumentation and measurements are described in their own sections below.

Controls and approvals

A person can pause, resume or cancel an agent or a single run from the dashboard, and approve or deny actions you guard. The agent applies these at checkpoints you choose, so nothing interrupts a step already in progress.

with client.run("weekly-digest"):
    client.checkpoint()                      # waits here while a person has paused the agent
    client.guarded_action(
        "Send the weekly digest",
        {"operation": "send_digest", "list": "subscribers", "draft": draft_id},
        lambda approved: send_digest(approved["list"], approved["draft"]),
        reason="Sends to more than 1,000 people need a person",
        destination="Newsletter: subscribers",
        affected_count=1240,
        show_fields=["list", "draft"],
    )

await client.run("weekly-digest", async () => {
  await client.checkpoint();
  await client.guardedAction(
    "Send the weekly digest",
    { operation: "send_digest", list: "subscribers", draft: draftId },
    (approved) => sendDigest(approved.list, approved.draft),
    {
      reason: "Sends to more than 1,000 people need a person",
      destination: "Newsletter: subscribers",
      affectedCount: 1240,
      showFields: ["list", "draft"],
    },
  );
});

Put every parameter that matters in the action and use the snapshot you are given: Toolcaise approves exactly the SHA-256 digest of that action. The forms checkpoint(run_id) and guarded_action(run_id, title, action, execute) from before 1.0 still work.

What the person deciding sees

The title, and anything you choose to add. The rest of the action never leaves your process.

Before anything is sent, the SDK replaces credential-like strings (bearer tokens, API keys, private keys, password= pairs) and email addresses with [redacted] in the title and in everything above, and shows [redacted] for any field whose name suggests a secret (password, token, credential, cookie, or a name ending in key, such as api_key or sshKey), at any depth. Toolcaise applies the same filter again when the request arrives. The dashboard shows these details as reported by the agent: Toolcaise cannot check them against the digest, which is what the approval is bound to. These options need SDK 1.1.0 or later: an older Python SDK refuses them with a TypeError, and an older Node.js SDK ignores them.

When the agent's last decided request with the same title and destination, made in the 30 days before, had a different digest, the dashboard says the parameters changed since that decision and shows which of the shared fields differ. So keep the title the same each time the agent asks for the same action, and show what can change as fields. When the same digest was approved in those 30 days, it says so and links to it. Whoever decides can leave a note, which is kept with the decision and never sent to the agent. Alert destinations that take warnings (email, Slack, Teams, Discord and webhooks, never PagerDuty) are told when a request arrives.

Model-call instrumentation

instrument_openai(client) and instrument_anthropic(client) (instrumentOpenAI, instrumentAnthropic) wrap an OpenAI or Anthropic client you created and return it. Each call to chat.completions.create, responses.create or messages.create made inside a run is then recorded as a model span under the current span, plus a call-scope usage sample, with:

• the model the provider answered with (else the one requested);
• input and output tokens from the provider's usage fields (for Anthropic, input includes cache reads and writes);
• latency, whether it streamed, and success or failure.

Prompts, messages, output, tool arguments and error text are never read. Sync, async and streamed calls are covered: a stream is recorded when you finish iterating it, and its token counts come from its final events. OpenAI chat streams report usage only when you pass stream_options={"include_usage": True}; without it the span has no token counts rather than zeros.

Usage, resources and output checks

client.metrics({"contextTokens": 18000, "contextCapacity": 128000}, scope="call", call_id="summarize-1")
client.metrics(client.sample_resources(output_path="report.json"))
client.output_checks("report.json", "Daily report", max_age_seconds=3600,
                     schema={"type": "object", "required": ["summary"]})

The accepted metric names and bounds are in the API reference.

Upgrading from the single file

Version 1.0 replaces the downloadable single-file SDK and keeps every method name and call form it had. What changes:

• Reporting no longer raises or waits. report(), heartbeat(), event(), outcome(), metrics() and output_checks() return None (resolve null) once queued, instead of the server's answer. run() starts your code even when the start could not be reported, and a failed heartbeat no longer raises after a successful run. Pass strict=True for the previous behaviour.
• Inside a run, event() now attaches to that run when no run ID is given.
• A 429 waits for the server's Retry-After instead of a fixed 60 seconds, in every mode.
• check is unchanged: it sends one heartbeat and fails loudly.

What is sent

Only operational metadata: run and span names, statuses, timings, model names, token counts and costs, event types and the messages you write, outcome labels, measurement values and check results. Never prompts, model output, file contents, exception text or tool arguments; guarded actions send a digest of the action and only the fields you choose to show, redacted. Labels are cut to their limits and credential-like strings are redacted, but free text you write is stored as written, so keep secrets and personal data out of names, titles and messages.

Everything an agent reports is stored and shown as agent-reported: a claim, not independent proof.