Optimization
Prompt caching
A provider feature that stores the repeated front portion of your prompt so you don't pay full price to send it again. You mark the stable part (instructions, examples, a document); the provider bills it at a discount on subsequent calls. See cached tokens.
Example
A coding assistant sends the same 10,000-token codebase context on every question in a session. Prompt caching drops the cost of that context by roughly 90% after the first call, so a 20-question session costs a fraction of what it would without caching.
Related terms
Cached tokens
Input tokens the provider recognizes from a previous request and serves at a steep discount instead of full price. Distinct from a full cache hit — here the model still runs, it just doesn't re-charge full rate for the repeated part of your prompt.
Semantic caching
Serving a stored answer when a new request means the same thing as a previous one — even if the wording is different — so the model never runs and the provider is never billed. Unlike prompt caching (a provider discount), a semantic cache hit costs nothing at the provider.
Prompt compression
Shrinking a prompt to fewer tokens without losing what the model needs — trimming boilerplate, deduplicating context, summarizing history — so you pay for less input.