Tokens & Pricing
Cached tokens
Input tokens the provider recognizes from a previous request and serves at a steep discount instead of full price. Distinct from a full cache hit — here the model still runs, it just doesn't re-charge full rate for the repeated part of your prompt.
Example
Your app sends the same 2,000-token system prompt on every call. With provider prompt caching, those 2,000 tokens bill at a fraction (often ~10%) of the normal input rate after the first call — so the repeated setup stops being the biggest line on your bill.
Related terms
Input tokens (prompt tokens)
The tokens you send into the model — your prompt, the instructions, any documents or history you include. You pay for these whether or not the answer is any good.
Prompt caching
A provider feature that stores the repeated front portion of your prompt so you don't pay full price to send it again. You mark the stable part (instructions, examples, a document); the provider bills it at a discount on subsequent calls. See cached tokens.
Semantic caching
Serving a stored answer when a new request means the same thing as a previous one — even if the wording is different — so the model never runs and the provider is never billed. Unlike prompt caching (a provider discount), a semantic cache hit costs nothing at the provider.