Two bills, two different shapes

A hosting bill is capacity. You add CPU, bandwidth or storage, and the meter follows the traffic curve. Once you have seen a month, you can predict the next one within a few percent.

A token bill is behaviour. Two teams with identical traffic can differ by 20× because one sends 12,000 tokens of retrieved context per request and the other caches a 400-token system prompt and sends nothing else. Same users, same features, wildly different invoices.

That is not a pricing trick, it is the shape of the meter: you pay per token, and the number of tokens is a design decision.

The three effects that make spend jumpy

1. Input usually dwarfs output — and they are priced differently. In most chat and retrieval features, input tokens outnumber output tokens by five to twenty times. Output tokens are typically several times more expensive per token than input. So “we only generate a short answer” tells you nothing about the cost.

2. Cached input has its own rate. Providers offer a cheaper rate for input that repeats between calls — a system prompt, a stable document, a long instruction block. Ignoring caching means paying full price for the same 3,000 tokens on every request.

3. Context per request swings by an order of magnitude. A summarisation job that appends conversation history every turn re-sends the whole transcript each time. Cost grows faster than the number of turns: turn 20 is expensive because turns 1 to 19 are on the bill again.

Retries and tool calls add a fourth multiplier that is invisible in every usage chart: a failed call that is retried is billed twice, and an agent that calls four tools per turn is billed five times per turn.

The levers, in the order they pay off

Teams usually reach for the model dropdown first. That is rarely the biggest lever. Try these in order:

  1. Trim the context. Cut retrieved documents that do not change the answer, cap the history you replay, and cache the part that never changes. This is usually the largest single win.
  2. Cap the output. Ask for the shape you want (“three bullets, no preamble”) instead of a generous token limit. The model generating 900 tokens of politeness is on your invoice.
  3. Batch what is not urgent. Nightly enrichment, backfills and evaluation runs can often take a slower, discounted path.
  4. Then choose the model. Downgrading a summariser to a smaller model is a real saving — but it is a saving applied to a number that the first three steps should already have reduced.

Estimate before you ship

You need two numbers per request: input tokens and output tokens. If the feature exists, read them from the API response — do not estimate. If it does not exist yet, you need a guess:

monthly cost ≈ requests/day × 30 × (input ÷ 1M × rate_in + output ÷ 1M × rate_out)

Two traps when guessing: averages hide the expensive request (a p95 request with a long document can cost 30× the median, and it is the one that sets your bill), and multi-turn features make the average meaningless because every turn carries the whole history.

The AI cost calculator does that arithmetic with an editable rate table: paste a usage log to get a per-model total, or describe a workload (tokens per request, requests per day, cache hit rate, batch discount) to get per-request, daily, monthly and yearly figures. The comparison tab runs the same workload across every rate you entered, which is the fastest way to see whether switching models is even worth the migration.

Subscription or per-token?

This is a volume question, not a plan preference. If your measured API cost per month is below a subscription price, the subscription is a convenience you are paying extra for; if it is far above, the plan usually wins — until you hit its usage limits, which is a separate constraint that the break-even number does not capture.

The useful output is the break-even volume: given your own per-request cost, how many requests per day must you exceed before a flat plan wins. Compare that with your actual traffic, and the decision stops being a feeling.

One honest caveat about the numbers

Prices change, plans change, and a calculator that ships with a hard-coded price list is wrong the moment someone updates a pricing page. So the tool ships with example placeholders and asks you to enter your own rates from your invoice; your edits stay in your browser and nothing is fetched, uploaded or cached on a server.

The arithmetic is the part that can be exactly right. The rates are the part only your account can answer.

If AI output also feeds reports, two related tools cover the neighbouring problems: the AI number checker flags figures with no traceable source, and the guardrails generator writes the rules file and permission lists that keep a coding agent inside its lane.