How to set a spending limit on an AI agent

Why a provider budget is not a spending limit for one agent

Most model providers let you set a monthly budget or a usage alert on your organization or project. That is worth doing, but it makes a poor limit for a single agent. It is account-wide, so a runaway agent spends the same money as your production app and every other agent, and when the cap is reached they all stop together.

Provider usage figures also tend to appear some time after the calls were made. An agent stuck in a loop, making several calls a second, can spend well past the point you would have chosen before any account-level check notices. Some provider settings are alerts rather than caps: you get an email and the calls keep succeeding. Check which kind you have, keep it as the backstop for the whole account, and put a narrower limit closer to the agent.

Per-run, per-agent and per-month limits

Before choosing a mechanism, decide what the number is attached to. The common scopes answer different questions:

• Per run: one execution may spend at most this much. For example, a nightly research agent gets $5 per run. This catches loops and runaway retries, the usual cause of a surprise bill, and it is simple to enforce because the counter starts at zero every time.
• Per agent: every run of a named agent gets the same limits, set in one place, so a new deployment or a teammate’s script cannot start it without them.
• Per month or per day: the total across many runs stays under a figure. This needs a counter that survives between runs, and a decision about what happens to the run that crosses it.

In practice you want a per-run limit on every agent, a per-agent default so nobody forgets it, and a provider budget as the outer boundary. A $5 per-run cap on an agent scheduled once a night bounds its month at roughly $150 even if every run misbehaves, which is often a good enough monthly limit on its own.

Cost is computed from a price table you supply

Model APIs report tokens, not dollars. A response carries usage fields (input tokens, output tokens, and on some APIs cached or cache-write tokens), and cost is those counts multiplied by the rates for that model. The rates live in your configuration and they change, so a cost limit is only as correct as the price table behind it.

A few rules keep that calculation honest:

• Price each model, and match model names carefully. A dated model version may need its own entry or a wildcard.
• Price cached input and cache writes separately where the provider bills them differently, or mark the figure as approximate.
• If a model has no price and a cost limit is set, refuse the call. Counting it as free silently switches the limit off.
• If a call reports no usage, for example a stream that was cut off, record it as unknown rather than zero.

The figure you compute is an estimate for enforcement, not an invoice, so reconcile against the provider’s billing when the money matters. The guide on AI agent cost monitoring explains how to read reported cost numbers; this one is about making the spending stop.

Where to enforce a spending limit

A limit can live in three places, and each fails in a different way.

• Provider limits. Nothing to build and nothing to bypass, but account-wide, coarse and delayed, as described above. Good as a backstop, not as a per-agent control.
• A gateway or proxy in the call path. The agent’s model traffic goes through a proxy that reads the usage on each response, keeps the counters and refuses calls past the limit. A bug in the agent cannot skip it. The trade-off is one more component on the path, and it only sees traffic that is actually routed through it.
• Counters in your own code. A wrapper around your model client adds up usage and raises an error at the limit. It is quick to write and needs no infrastructure, but it only works if every call goes through that wrapper, including calls made by libraries and sub-agents, and a limit implemented by the agent is a limit the agent can get wrong.

For agents that run unattended, a proxy is usually the right choice, because the code being limited is not the code doing the limiting. The official OpenAI and Anthropic SDKs, and most agent frameworks, read their base URL from an environment variable, so routing through a local proxy is a configuration change rather than a code change.

What a hard limit must do when it is hit

Refusing the next model call is not the same as stopping the agent. An agent that gets an error may retry, fall back to another model, or carry on with work that does not need a model, such as tool calls and writes to other systems. A hard limit should do four things:

• Stop the whole process tree. Agents start shells, browsers, language servers and sub-agents, and killing only the parent can leave children running and spending. Send a polite terminate, wait a short grace period so the agent can flush what it has, then force-kill whatever remains.
• Refuse with an error the client will not retry. Many SDKs retry rate-limit responses automatically, so answering a blocked call with a 429 only delays the stop. A permission error such as a 403 is treated as final.
• Accept a bounded overshoot. Usage is known only when a response finishes, so calls already in flight will complete. The overshoot is limited by how many calls can run at once; a limit on the number of calls can be exact.
• Record why. The run should end with a distinct exit code and a status that names the limit, so the next person reads “stopped at the cost limit” rather than a generic failure, and does not simply rerun it.

How to test a spending limit safely

A limit you have never seen fire is a guess. Test it on a task with no external side effects, with a budget far below anything real, before you rely on it:

1. Run the agent once with metering and no limit, to learn what a normal run uses in calls, tokens and cost.
2. Set a limit the first call or two will cross, for example a limit of one model call or a cost limit of one cent.
3. Confirm the process and all its children are gone, the exit code is the one you expect, and the recorded reason names the limit.
4. Remove the price for the model you use and confirm the call is refused rather than counted as free.
5. Set the real limit with headroom above a normal run, so it catches runaways rather than ordinary variation.

Setting a spending limit with Toolcaise Connect

Toolcaise Connect takes the proxy approach. toolcaise-connect run starts your agent command behind a gateway that listens only on 127.0.0.1, points the command’s OpenAI and Anthropic base URLs at it, and measures every model call from the provider’s own usage fields. The limits are flags on the command:

toolcaise-connect run --agent "Nightly research" --max-runtime 45m --max-total-tokens 2000000 -- python agent.py
toolcaise-connect run --agent "Refactor bot" --policy policy.json --max-cost-usd 5 -- node bot.js
toolcaise-connect run --agent "Batch job" --gateway -- python job.py        # meter only, no limits

The limits are --max-cost-usd, --max-total-tokens (with --max-input-tokens and --max-output-tokens), --max-model-calls and --max-runtime, plus --allow-model to refuse any model you have not approved. --gateway meters without limiting, which is the first step of the test above. A cost limit needs prices, and they go in a policy file that can also hold the same limits:

{
  "maxRuntime": "45m",
  "maxCostUsd": 5,
  "maxTotalTokens": 2000000,
  "allowModels": ["claude-*", "gpt-4o*"],
  "prices": {
    "gpt-4o": { "inputPerMTok": 2.5, "outputPerMTok": 10 },
    "claude-*": { "inputPerMTok": 3, "outputPerMTok": 15, "cachedInputPerMTok": 0.3, "cacheWritePerMTok": 3.75 }
  }
}

With a cost limit set, a model that has no entry in the price table is refused rather than assumed free, and a call that used cache tokens without cache prices is flagged as approximate. A call that reports no usage is counted as unknown, never as zero.

What happens when a Connect limit is reached

By default Connect terminates the command and everything it started: a polite terminate, a grace period set by --stop-grace (10 seconds unless you change it), then a hard kill, including processes started while it waited. The run exits with code 124, as timeout(1) does, and is recorded as canceled with the limit as its error code, for example guard_max_cost_usd. With --on-limit block, further model calls are refused with a 403 in the provider’s own error format and the command is left to finish on its own, which suits an agent that can write up partial results.

Enforcement happens on your machine, so a run is limited even with no network path to Toolcaise. Add --remote-control and you can also set limits per agent in the dashboard. They are merged with the machine’s own and the stricter value wins, so the dashboard can tighten a run but never loosen it. A new dashboard limit applies to a live run at once, counting what the run has already used, and the last policy received is kept on the machine so a run that starts while Toolcaise is unreachable is still held to it.

What Connect does not cover

• Only calls to an OpenAI-compatible or Anthropic API made through the OPENAI_BASE_URL, OPENAI_API_BASE or ANTHROPIC_BASE_URL settings are metered. If your agent already sets one of these, for example to an internal gateway, Connect keeps that endpoint as the upstream.
• An agent that hard-codes a provider endpoint, or speaks another protocol, bypasses the metering, so cost, token and model limits do not reach it. The runtime limit and the process-tree stop still do, which is a good reason to set --max-runtime on every run.
• Limits apply to a run. Connect does not keep a total across runs, so a daily or monthly budget still belongs at the provider or in your own accounting.
• Cost comes from your prices, not from an invoice, and calls in flight when a limit is reached still complete.
• A stop sent from the dashboard can take up to one ten-second poll to arrive, and up to two minutes after failed polls. Limits enforced on the machine are not affected by either.
• Connect is currently a preview release, and its builds are not yet code-signed.