Optimization
Batch API
A cheaper, slower processing mode where you submit many requests at once and accept results within a delay window (often up to 24 hours) in exchange for a large discount, commonly around 50%.
Example
You need to categorize 500,000 archived documents. There's no rush, so you submit them to a batch API overnight at half price instead of paying full real-time rates — turning a $2,000 job into a $1,000 one.
Related terms
Model routing
Automatically sending each request to the cheapest model that can handle it, instead of defaulting everything to one expensive model. The routing decision happens before the call, based on the task.
Prompt compression
Shrinking a prompt to fewer tokens without losing what the model needs — trimming boilerplate, deduplicating context, summarizing history — so you pay for less input.
Cost per 1K tokens
The standard way pricing is quoted: dollars per 1,000 tokens, listed separately for input and output. Providers increasingly quote per million tokens, but the per-1K figure is still the mental unit for estimating a call.