Optimization
Prompt compression
Shrinking a prompt to fewer tokens without losing what the model needs — trimming boilerplate, deduplicating context, summarizing history — so you pay for less input.
Example
An agent's prompt carries a 6,000-token conversation history, much of it redundant. Compressing it to a 1,500-token summary of the salient facts cuts input cost by 75% on every subsequent step, and often improves answers by removing noise.
Related terms
Input tokens (prompt tokens)
The tokens you send into the model — your prompt, the instructions, any documents or history you include. You pay for these whether or not the answer is any good.
Context window
The maximum number of tokens a model can consider at once — input plus output combined. It's the model's working memory for a single call. Go over it and the request fails or silently drops the oldest content.
Prompt caching
A provider feature that stores the repeated front portion of your prompt so you don't pay full price to send it again. You mark the stable part (instructions, examples, a document); the provider bills it at a discount on subsequent calls. See cached tokens.