Optimization
Provider arbitrage
Choosing among providers (or hosts) that serve comparable models at different prices, and sending traffic to the cheaper one. The same open model is often available from several vendors at meaningfully different rates.
Example
An open 70-billion-parameter model is offered by three hosting providers at $0.90, $0.60, and $0.40 per million tokens. Routing steady traffic to the $0.40 host instead of the $0.90 one cuts that workload's cost by more than half — for an identical model.
Related terms
Model routing
Automatically sending each request to the cheapest model that can handle it, instead of defaulting everything to one expensive model. The routing decision happens before the call, based on the task.
Model tier
The rough class of a model by capability and price. Providers offer a range — small/fast/cheap up to large/slow/expensive — and the price gap between tiers is often 10–30×.
BYOK (Bring Your Own Key)
An arrangement where you keep your own provider account and credentials, and the governing layer uses your keys on your behalf rather than reselling access. Your provider relationship, spend, and data path stay yours.