Cost control

Where the tokens actually go in an agentic workload, and the four levers that move the bill.

If you have run a coding agent for a full working day you have seen the bill. Agentic loops are token-hungry in a way ordinary chat never is: every file read, every tool result and every diff lands in context and is resent on the next turn.

Output tokens are the expensive ones

On every frontier model, output costs 4–5× input. A model that thinks out loud in its final answer burns money faster than one that answers tightly. Cap max_tokens deliberately and prefer concise models for routine work.

The four levers

1. Route by task. Classification, extraction and routing do not need Opus — Haiku does them at roughly a seventh of the price. Reserve the flagship for reasoning-heavy work.
2. Prompt caching. Stable system prompts and tool definitions are re-read from cache at a fraction of the input price. For agents this is the single biggest lever — see Prompt caching.
3. Trim context each turn. Do not resend an entire transcript; keep recent turns plus a running summary. Unbounded context growth is what turns a $2 session into a $20 one.
4. Cap the loop. Set a hard step limit on autonomous runs so a stuck agent cannot spin forever.

Estimating before you run

CopyIN_RATE  = {class="s">"claude-opus-4-8": 3.0,  class="s">"claude-sonnet-4-6": 0.9, class="s">"claude-haiku-4-5": 0.4}
OUT_RATE = {class="s">"claude-opus-4-8": 15.0, class="s">"claude-sonnet-4-6": 4.5, class="s">"claude-haiku-4-5": 2.0}

def cost(model, tok_in, tok_out):
    return tok_in / 1e6 * IN_RATE[model] + tok_out / 1e6 * OUT_RATE[model]

print(cost(class="s">"claude-opus-4-8",   20_000, 2_000))   class=class="s">"c"># 0.09
print(cost(class="s">"claude-sonnet-4-6", 20_000, 2_000))   class=class="s">"c"># 0.027
print(cost(class="s">"claude-haiku-4-5",  20_000, 2_000))   class=class="s">"c"># 0.012

The same 20K-in / 2K-out task costs 7.5× more on Opus than on Haiku. If the task is classification, that multiple buys nothing.

Measuring after

Every response carries a usage object. Log it against a request id and a task type — without that, cost optimisation is guesswork. The gateway dashboard shows per-model usage, but only your own logs know *why* a request was made.