Skip to content

Supervise Vercel AI SDK agents

Export the AI SDK's OpenTelemetry spans to Toolcaise: every generation step is a model call with token counts, and every tool call is recorded by name.

Set it up

Recommended path: OpenTelemetry, with AI SDK 7's @ai-sdk/otel. You need an agent credential from Dashboard, Agents: one credential per agent.

1. Install the AI SDK's OpenTelemetry integration and an OTLP exporter

Shell
npm install @ai-sdk/otel @opentelemetry/sdk-node @opentelemetry/exporter-trace-otlp-proto

2. Register telemetry once, before the first model call

In AI SDK 7, registering telemetry turns it on for every call. Turn off input and output recording per call; Toolcaise never reads them either way.

TypeScript
import { NodeSDK } from "@opentelemetry/sdk-node";
import { OTLPTraceExporter } from "@opentelemetry/exporter-trace-otlp-proto";
import { generateText, registerTelemetry } from "ai";
import { OpenTelemetry } from "@ai-sdk/otel";
import { openai } from "@ai-sdk/openai";

const sdk = new NodeSDK({
  serviceName: "research-agent",
  traceExporter: new OTLPTraceExporter({
    url: "https://toolcaise.com/api/v1/otlp/v1/traces",
    headers: { Authorization: `Bearer ${process.env.TOOLCAISE_AGENT_TOKEN}` },
  }),
});
sdk.start();
registerTelemetry(new OpenTelemetry());

const { text } = await generateText({
  model: openai("gpt-4.1"),
  prompt: "Summarise today's incidents",
  telemetry: { functionId: "incident-summary", recordInputs: false, recordOutputs: false },
});

await sdk.shutdown(); // flush the last spans before a short script exits

Then run the agent as usual. Its runs appear in the dashboard as soon as their first report arrives.

What Toolcaise receives

How it arrives
RunsEach trace is one run. It is created when the first span arrives and completed, or failed, when the trace's root span ends.
Model callsEach chat step becomes a model span with the provider's model and its input and output tokens. The run's aggregate usage on the invoke_agent span is not counted again.
Tool callsexecute_tool spans become tool spans with the tool's name.
StepsThe invoke_agent span that wraps a call becomes a step and, as the root, completes the run.
FailuresA failed span in the trace adds an event that carries only the exception type, never its message.
Agent detailsservice.name, service.version and host.name describe the agent.

What this path does not give you:

  • Pause, cancel and approvals: OpenTelemetry only reports. Use the SDK's checkpoint() and guarded_action() for decisions.
  • Cost: Toolcaise does not price tokens on this path, and stores cost only when the instrumentation reports llm.cost.total.

Everything here is reported by your agent or its framework and stored as agent-reported. Prompts, completions and tool arguments are never read: the OpenTelemetry endpoint reads only an allow-list of attributes, and the SDK sends metadata only.

Other ways

Next.js

Register in instrumentation.ts with @vercel/otel: registerOTel({ serviceName, traceExporter: new OTLPHttpProtoTraceExporter({ url, headers }) }), then registerTelemetry(new OpenTelemetry()).

AI SDK 6

Telemetry is built into the ai package and enabled per call with experimental_telemetry: { isEnabled: true, recordInputs: false, recordOutputs: false }. Those spans use the older ai.* attributes (ai.usage.promptTokens, ai.toolCall.name), which Toolcaise reads according to the AI SDK's documentation; we have not captured them from a live AI SDK 6.

Approvals: the Node SDK in your tools

Wrap the call in client.run() and guard a tool's execute with client.guardedAction(). This reports a run of its own next to the trace's run.

TypeScript
import { Toolcaise } from "./toolcaise-agent.mjs";
import { generateText, tool } from "ai";
import { z } from "zod";

const toolcaise = new Toolcaise();
await toolcaise.run("refunds", () =>
  generateText({
    model: openai("gpt-4.1"),
    prompt,
    tools: {
      refund: tool({
        inputSchema: z.object({ orderId: z.string(), amount: z.number() }),
        execute: (input) =>
          toolcaise.guardedAction(`Refund ${input.amount}`, input, (approved) => refunds.issue(approved)),
      }),
    },
  }),
);

Caveats

  • Registered telemetry records inputs and outputs unless each call sets recordInputs: false and recordOutputs: false.
  • A process that exits without sdk.shutdown() can lose the spans still waiting in the exporter's batch.

What was checked

Checked on 2026-09-22 against the AI SDK 7 telemetry and migration documentation (ai 7.0, @ai-sdk/otel 1.0). Toolcaise's OTLP ingest is tested against spans captured from a real AI SDK 7 with @ai-sdk/otel.

Reference: SDKs, HTTP API, and the OpenTelemetry endpoint's limits and answers.