How AI API Pricing Works: Tokens, Caching and Batch Explained

Almost every large language model API charges by the token, but the price sheet has more moving parts than a single number. Once you understand six concepts — tokens, input, output, caching, batch and long-context tiers — you can estimate any provider’s bill in a couple of minutes.

1. Tokens

A token is a chunk of text the model reads or writes: often a whole short word, sometimes part of a longer word, a number or a punctuation mark. Prices are quoted per 1 million tokens. Each provider uses its own tokenizer, so the same text can be a different number of tokens on different models.

2. Input tokens

Input is everything you send: the system prompt, the conversation history, retrieved documents, tool definitions and the user’s message. In chat apps the history grows with every turn, so input usually makes up most of the token count.

3. Output tokens

Output is what the model generates. It costs several times more than input — for example, Claude Sonnet 5 charges $2.00 per 1M input tokens and $10.00 per 1M output tokens. If a model “thinks” before answering, those reasoning tokens are billed as output too. Google’s pricing page states this explicitly for Gemini.

4. Cached input

When requests share a long identical prefix, such as a big system prompt or a document, providers can serve it from a cache at a steep discount — typically 10% of the normal input price. Some providers charge extra to write the cache in the first place, and Google also bills cache storage per hour.

5. Batch, flex and priority tiers

If you don’t need an instant answer, batch APIs usually cut the price in half. OpenAI also sells Flex (half price, slower) and Fast mode (double price). Google’s priority inference costs 1.8x on its Gemini 3.x Flash models, and DeepSeek halves its prices outside peak hours.

6. Long-context tiers

Some models switch to higher rates when a prompt gets very long. Gemini 3.1 Pro and Grok 4.6, for example, both charge more once a prompt reaches about 200K tokens. If you stuff whole codebases or books into a prompt, check this line of the price sheet first.

The formula

cost per request = (input tokens × input rate + output tokens × output rate) ÷ 1,000,000

Multiply by your monthly request count, then adjust for caching, batch discounts and long-context tiers.

A worked example

Say your app sends 100,000 requests a month to Claude Sonnet 5, each with 2,000 input tokens and 500 output tokens:

• Input: 2,000 × $2.00 ÷ 1M = $0.0040
• Output: 500 × $10.00 ÷ 1M = $0.0050
• Per request: $0.0090
• Per month: $900.00

If 75% of those input tokens came from the prompt cache, the monthly bill would drop to about $630.00 before cache-write charges.

Hidden costs to budget for

• Tools. Built-in web search is billed per call on top of tokens: $10 per 1,000 calls on OpenAI and $5 per 1,000 on xAI. Google includes 5,000 free grounded searches a month on Gemini 3.x, then charges $14 per 1,000.
• Retries and failures. Every failed or retried request is billed like a successful one.
• Data residency. OpenAI adds 10% for regional processing on newer models, and Anthropic charges 1.1x for US-only inference on Claude 4.6 and later models.

Ready to run the numbers? Try the cost calculator or compare two models side by side.