Where an agent’s cost number comes from
Before you trust a cost figure, find out who produced it. For an AI agent there are usually three sources, and they can disagree about the same run.
- Provider-reported usage. Each API response carries token counts, and the provider’s billing records and invoice follow later. The invoice is what you reconcile against, but it arrives late and is rarely broken down by agent.
- Measured at a gateway. A proxy between the agent and the provider reads the usage on every response and prices it. It sees calls the agent did not choose to report, but only calls that pass through it.
- Reported or estimated by the agent. The agent’s code, or its framework, adds up tokens and multiplies them by rates it holds. This is convenient and often close, but it is a claim: a missed code path or a stale rate changes the number without anyone noticing.
Label every figure with its source, so that when a dashboard total and the invoice differ you can tell where each came from.
Input, output and cached tokens, and why the price table matters
Providers bill tokens by type, and each type has its own rate for each model:
- Input tokens: the prompt, instructions, tool definitions and conversation history sent with every call.
- Output tokens: what the model generates. On some models, internal reasoning tokens are billed as output even though you never see them.
- Cached input tokens: input served from a prompt cache, usually billed below the fresh input rate.
- Cache writes: input stored in a cache for reuse, which some providers bill above the plain input rate.
Providers report these differently. Some count cached tokens inside the input total and break them out in a detail field; others report cache reads and writes as separate fields. If your tracking prices all of it at the full input rate, an agent with a good cache hit rate looks more expensive than it is.
Cost is tokens multiplied by a rate, so the price table is half of every number. With made-up example rates, 400,000 input tokens at $2 per million and 50,000 output tokens at $8 per million come to $0.80 plus $0.40, or $1.20. If the table holds an old rate, or has no entry for a model the agent switched to, the token counts stay correct and the cost is wrong. Keep the table dated, versioned and owned by someone.
Cost per run, per agent and per month
These answer different questions. Cost per run tells you what one unit of work costs and makes outliers visible. Cost per agent tells you which workload is responsible and who should look at it. Cost per month is what finance compares with the budget and the invoice.
Build the larger figures from the smaller ones without counting spend twice. Suppose a workflow reports three model calls costing $0.01, $0.02 and $0.03, then a cumulative run total of $0.06. Adding all four gives $0.12, which is wrong: the call sum and the run total describe the same $0.06. Pick one level as the source of truth for each run.
Attributing cost to agents and teams
Attribution usually fails because several agents share one provider key. The provider bills the key, not the agent, so the split has to come from your own records. Give each agent its own identity, ideally its own credential, and record which team owns it. Tag runs with agent version, prompt version and model, so a rise after a release can be compared like with like.
Track coverage next to totals. A run that reported no usage is unknown, not free, and a total that treats missing values as zero looks better than reality. Show how many runs are missing cost data beside the total they would change.
The cost of retries and loops
A retried model call sends its full input again, so a step that fails twice before succeeding costs about three times what you planned. Loops are worse: each turn usually re-sends the growing conversation, so input tokens per call climb and the total grows faster than the number of turns.
Give each model invocation a stable call ID, so a report re-delivered after a network error is recognized as the same call while a second paid request counts as a new one. Record retries and steps per run as numbers you can chart: a run that normally takes 12 steps and suddenly takes 60 is usually a loop. Monitoring tells you a loop happened; only a limit on the call path stops one.
How to read a cost spike
When a daily total jumps, work through the causes in an order that separates real spend from measurement problems. For example:
- Check whether there were more runs or more expensive runs. A backlog replay can double spend with every run normal.
- Break the day down by agent and find the one whose cost per run moved.
- Separate tokens from price. Flat tokens with rising cost means the price table or the model changed.
- Separate input from output. Rising input points at context growth, larger retrieved documents or a lost prompt cache; rising output points at a prompt or model change.
- Look at retries and steps per run for loops and failure storms.
- Rule out measurement artifacts: call and run totals counted twice, a code path reporting the same call twice, or an edit to the price table.
Setting cost budgets and alerts
Turn the monthly budget into numbers you can alert on earlier. Divided by the days in the month it gives an expected daily spend, and your median cost per run gives a per-run expectation. Alert when a day passes a clear multiple of the expected value, and when a single run passes a per-run ceiling, so a runaway run is noticed while it is still running.
Relative alerts catch regressions that fixed thresholds miss: an agent whose cost per run rises 40 percent after a prompt change may still be under every absolute limit.
An alert tells a person; it does not stop anything. If a run must not exceed a figure, you need a limit enforced on the path the model calls take, which our guide to AI agent spending limits covers.
How Toolcaise records model usage and cost
Toolcaise gets usage data in two ways. With the Python or Node.js SDK, you wrap the provider client you already use. Each chat.completions.create or responses.create call inside a run is recorded as a model span with the model, input and output tokens from the provider’s usage fields, latency and success. Prompts and output are never read.
from openai import OpenAI
from toolcaise import Toolcaise
client = Toolcaise() # reads TOOLCAISE_AGENT_TOKEN
openai = client.instrument_openai(OpenAI()) # records model, tokens and latency
with client.run("nightly-research"):
openai.chat.completions.create(model="gpt-4.1", messages=[...])The SDK records tokens, not dollars, because it holds no price table. To see cost from this path, compute it in your code and report it as reported_cost_usd on a span or as run totals with client.metrics(..., scope="run"). OpenAI chat streams report usage only when you pass stream_options={"include_usage": True}; without it the span has no token counts rather than zeros.
With Toolcaise Connect, the measuring happens on your machine. toolcaise-connect run starts a loopback-only gateway for the command, points the OpenAI and Anthropic SDK base URLs at it and reads the usage of every model call itself; --gateway meters without setting a limit. Cost comes only from prices you supply in a policy file, in US dollars per million tokens, with optional rates for cached input and cache writes.
toolcaise-connect run --agent "Nightly research" --gateway --policy prices.json -- python agent.py
{
"prices": {
"your-model-name": { "inputPerMTok": 2, "outputPerMTok": 8 },
"your-model-family-*": { "inputPerMTok": 2, "outputPerMTok": 8, "cachedInputPerMTok": 0.5, "cacheWritePerMTok": 2.5 }
}
}A call whose response carries no usage, or is cut off, is counted as unknown, never as zero. A call that used cache tokens without cache rates in the table is priced at the plain input rate and flagged approximate.
Daily totals and cost alerts in the dashboard
Under Policies & controls, the Usage tab shows daily activity per agent: runs, calls, reported cost and tokens in and out, by UTC day. When some runs reported no cost, it shows the known subtotal and the number of incomplete runs instead of a partial sum, and it does not add call samples to run totals.
The Rules & alerts tab turns those figures into alerts. A rule on reported cost with a daily total fires when an agent’s day passes your threshold, and fires as a reported minimum when the total is incomplete but the known subtotal is already over. The response can be an incident and notification, a suggested pause or a request for a cooperative pause. An optional percentage change from a tagged baseline catches regressions once there are three baseline observations.
What these cost figures do not tell you
A cost is only as good as the price table behind it. Toolcaise does not ship provider prices: the SDK records tokens, and Connect prices calls from the table you give it, so a stale rate produces a wrong figure and a missing one produces no figure.
- Figures an agent sends about itself are labeled agent-reported. Figures from a run Toolcaise Connect supervises are labeled measured by Connect, which counts each model call at its gateway from the provider’s usage fields; Connect vouches for them, but they are not tamper-proof.
- Calls that go around the instrumentation are not counted, such as an uninstrumented client method or an agent that hard-codes a provider endpoint.
- It is not an invoice. Reconcile monthly totals against the provider’s billing records.
- Metric samples are kept for 30 days.
Frequently asked questions
How do I track token usage per run for an AI agent?
Group every model call under a run ID and record the input, output and cached token counts from each response’s usage field. Sum them per run, or report a cumulative run total, but not both. Streamed responses often omit usage unless you ask for it, and missing usage should be recorded as unknown rather than zero.
How do I calculate OpenAI cost per agent?
Record each agent’s token counts per response, split into input, cached input and output, and multiply them by the current rates for that model from OpenAI’s price page. Attribution is easier when each agent has its own API key, because the provider’s usage view then lines up with your records. Reconcile monthly totals against the invoice.
Why does my agent cost dashboard not match the provider invoice?
Common causes are a stale or incomplete price table, cached tokens priced at the full input rate, calls that bypassed instrumentation, runs that reported no usage, and call and run totals counted twice. Time zones also matter: a dashboard grouped by UTC day and an invoice in local time split days differently.
Is a cost alert the same as a spending limit?
No. An alert notifies a person after a threshold is crossed, and the agent keeps running until someone acts. A spending limit is enforced on the path the model calls take and refuses or stops work when the budget is used. Use an enforced limit when a run must not exceed a figure.
Do cached input tokens cost less than regular input tokens?
Usually, though rates differ by provider and model, and writing to a cache can cost more than plain input on some providers. For monitoring, what matters is that your records keep cached tokens separate and your price table has a rate for them, or an agent that uses the cache well looks more expensive than it is.