The AI Cost & Governance Glossary

How to use this glossary

AI spend has its own vocabulary, and most of it was invented in the last two years. "Reasoning tokens," "semantic caching," "the MCP," "agent identity" — the terms move faster than anyone's onboarding doc, and the gap between the people who use AI and the people who pay for it is largely a gap in language. A CFO can't govern a line item they can't name.

This is a plain-English reference for that vocabulary. It's for anyone who touches AI spend: the engineer wiring up a model, the finance lead reconciling the invoice, the security owner asked whether customer data can leak, the operations lead trying to keep a runaway agent from posting a five-figure surprise.

It's organized into six sections that follow the money from the smallest unit outward — Tokens & Pricing, then Optimization, Attribution & Finance, Identity & Security, Agents & MCP, and Governance & Compliance. Every term has a one- or two-sentence definition and a concrete example — because the fastest way to understand "provider arbitrage" or "cost per successful task" is to see it happen with real numbers.

A few terms are Tokenality concepts (Virtual AI Key, Token Ledger, binding key, spend control plane). Those are defined the same way as everything else — neutrally, so you can judge them on the merits. Where a definition leans on another term, that term has its own entry; this is a glossary, so cross-references are a feature.

You don't have to read it front to back. Skim the section you need, or use it to settle the next "wait, what does that actually mean?" in a planning meeting.

1. Tokens & Pricing

Token

The unit AI models read and write, and the unit you're billed in. A token is a chunk of text — often a word, a piece of a word, or a punctuation mark. Models don't see letters or words; they see tokens.

Example: The sentence "Make every token count." is about 5 tokens. A rough rule of thumb for English: 1,000 tokens is roughly 750 words, or about a page and a half.

Input tokens (prompt tokens)

The tokens you send into the model — your prompt, the instructions, any documents or history you include. You pay for these whether or not the answer is any good.

Example: You paste a 4,000-word support-ticket thread and ask the model to summarize it. Those 4,000 words are roughly 5,300 input tokens — billed before the model writes a single word of the summary.

Output tokens (completion tokens)

The tokens the model generates back to you. Output tokens almost always cost more per token than input tokens — often 3–5× more — because generating text is the expensive part.

Example: The model replies with a 200-word summary (~270 output tokens). If input is $3 per million tokens and output is $15 per million, this one call costs about $0.016 for input plus $0.004 for output — small, until you make it a million times a day.

Reasoning tokens

Extra output tokens some models generate as internal "thinking" before they produce a final answer. You're billed for them, but you usually never see them — they're the model working through the problem.

Example: You ask a reasoning model a hard math question. It answers in 50 visible tokens — but consumed 3,000 reasoning tokens getting there. Your bill reflects 3,050 output tokens, not 50. This is why a "short" answer from a reasoning model can cost more than a long answer from a standard one.

Cached tokens

Input tokens the provider recognizes from a previous request and serves at a steep discount instead of full price. Distinct from a full cache hit — here the model still runs, it just doesn't re-charge full rate for the repeated part of your prompt.

Example: Your app sends the same 2,000-token system prompt on every call. With provider prompt caching, those 2,000 tokens bill at a fraction (often ~10%) of the normal input rate after the first call — so the repeated setup stops being the biggest line on your bill.

Context window

The maximum number of tokens a model can consider at once — input plus output combined. It's the model's working memory for a single call. Go over it and the request fails or silently drops the oldest content.

Example: A model with a 200,000-token context window can hold roughly 150,000 words in mind at once — a small book. A long agent run that keeps appending to the conversation can fill that window and start paying to re-send the same history on every step.

Model tier

The rough class of a model by capability and price. Providers offer a range — small/fast/cheap up to large/slow/expensive — and the price gap between tiers is often 10–30×.

Example: A "flagship" tier model might cost $15 per million output tokens; the same provider's "mini" tier might cost $0.60. If a task (like classifying a support ticket) works fine on the mini tier, running it on the flagship is paying 25× for no benefit.

Cost per 1K tokens

The standard way pricing is quoted: dollars per 1,000 tokens, listed separately for input and output. Providers increasingly quote per million tokens, but the per-1K figure is still the mental unit for estimating a call.

Example: A model priced at $0.003 per 1K input tokens and $0.015 per 1K output tokens. A call with 1,500 input and 500 output tokens costs (1.5 × $0.003) + (0.5 × $0.015) = $0.012.

2. Optimization

Prompt caching

A provider feature that stores the repeated front portion of your prompt so you don't pay full price to send it again. You mark the stable part (instructions, examples, a document); the provider bills it at a discount on subsequent calls. See cached tokens.

Example: A coding assistant sends the same 10,000-token codebase context on every question in a session. Prompt caching drops the cost of that context by roughly 90% after the first call, so a 20-question session costs a fraction of what it would without caching.

Semantic caching

Serving a stored answer when a new request means the same thing as a previous one — even if the wording is different — so the model never runs and the provider is never billed. Unlike prompt caching (a provider discount), a semantic cache hit costs nothing at the provider.

Example: One user asks "What's your refund policy?" and another asks "How do I get my money back?" A semantic cache recognizes these as the same question and returns the stored answer to the second user at zero provider cost. On repetitive production traffic, a well-tuned semantic cache commonly removes a large share of provider calls.

Model routing

Automatically sending each request to the cheapest model that can handle it, instead of defaulting everything to one expensive model. The routing decision happens before the call, based on the task.

Example: Simple "summarize this" requests route to a $0.60-per-million mini model; complex "analyze this contract for risk" requests route to a $15-per-million flagship. If 80% of traffic is simple, routing can cut the model bill by more than half without users noticing.

Provider arbitrage

Choosing among providers (or hosts) that serve comparable models at different prices, and sending traffic to the cheaper one. The same open model is often available from several vendors at meaningfully different rates.

Example: An open 70-billion-parameter model is offered by three hosting providers at $0.90, $0.60, and $0.40 per million tokens. Routing steady traffic to the $0.40 host instead of the $0.90 one cuts that workload's cost by more than half — for an identical model.

Batch API

A cheaper, slower processing mode where you submit many requests at once and accept results within a delay window (often up to 24 hours) in exchange for a large discount, commonly around 50%.

Example: You need to categorize 500,000 archived documents. There's no rush, so you submit them to a batch API overnight at half price instead of paying full real-time rates — turning a $2,000 job into a $1,000 one.

Prompt compression

Shrinking a prompt to fewer tokens without losing what the model needs — trimming boilerplate, deduplicating context, summarizing history — so you pay for less input.

Example: An agent's prompt carries a 6,000-token conversation history, much of it redundant. Compressing it to a 1,500-token summary of the salient facts cuts input cost by 75% on every subsequent step, and often improves answers by removing noise.

3. Attribution & Finance

Cost attribution

Tying each dollar of AI spend to who spent it and why — the person, team, project, or agent responsible — rather than seeing one lump sum on a provider invoice. Attribution is the difference between "we spent $40,000 on AI" and "the research team's document pipeline spent $40,000."

Example: Two teams share one provider key. The invoice says $18,000. Without attribution, nobody can say whether that's the sales team's chatbot or engineering's test harness — so nobody owns it, and nobody optimizes it.

Chargeback

Billing each internal team for the AI spend it actually caused, moving the cost onto that team's budget. It makes teams feel their own usage, which is the fastest way to curb waste.

Example: Finance produces a monthly chargeback report: Sales $6,200, Support $4,800, Engineering $7,000. Each number lands on that team's own budget line, so the support lead now has a reason to care that a chatbot is over-provisioned.

Showback

The lighter cousin of chargeback: showing each team what it spent without moving the money. It creates visibility and accountability without the friction of internal billing.

Example: Every team lead gets a monthly email — "Your team used $4,800 of AI last month, up 30%." No budget transfer happens, but the number alone often triggers a "why is that up?" conversation that trims spend.

Cost per successful task

Spend divided by successful outcomes, not by calls or tokens. It's the metric that reveals whether AI is actually efficient, because a cheap call that fails and gets retried three times is not cheap.

Example: An agent completes 1,000 tasks a day and costs $200 — but 400 of those "completions" failed and had to be redone. Cost per successful task is $200 ÷ 600 = $0.33, not the $0.20 the raw math suggests. Optimizing the failures is where the money is.

Unit economics

The cost of AI to serve one unit of your business — one customer, one transaction, one support ticket — so you can tell whether a feature makes or loses money as it scales.

Example: Your AI support feature costs $0.14 in tokens per ticket resolved. If you charge customers a plan that nets $0.10 of margin per ticket, the feature loses money on every heavy user — a unit-economics problem you want to catch before you have a million of them.

Run-rate

Your current spend projected forward over a full period, usually annualized, to answer "if today's pace held, what would this cost us for the year?"

Example: You spent $9,500 on AI last month. Your run-rate is roughly $114,000 a year. If last month was up 20% from the month before, the naive run-rate understates where you're heading — which is exactly why run-rate paired with growth rate matters.

GL allocation

Mapping AI spend to the right lines in your general ledger — the accounting system of record — so it lands in the correct cost center, department, or project code instead of one catch-all "software" bucket.

Example: Finance needs the $18,000 AI invoice split across three department GL codes for the quarterly close. GL allocation assigns each attributed slice to its code automatically, so the numbers reconcile without a spreadsheet rebuild.

Invoice reconciliation

Matching your own records of what was spent against the provider's invoice, line by line, to confirm they agree — and to catch discrepancies before you pay.

Example: Your internal ledger says $17,940 for the month; the provider invoice says $18,010. Reconciliation surfaces the $70 gap and lets you trace it — a late-posting call, a rounding difference, or a workload you didn't know about — instead of just paying the number on the bill.

FinOps

The discipline of managing cloud (and now AI) spend as a shared, ongoing practice across engineering, finance, and product — bringing accountability to variable, usage-based costs. The FinOps Foundation maintains the standard framework.

Example: Instead of finance discovering the AI overspend at quarter-end, a FinOps practice puts near-real-time spend in front of the engineers who can act on it — so the people generating the cost are the people watching it.

4. Identity & Security

Virtual AI Key

A governed stand-in for a raw provider API key: a key you issue per team, person, project, or agent that carries its own budget, its own limits, and full attribution — while the real provider key stays hidden. The concept mirrors the virtual credit card, where each card has its own limit and owner. (In Tokenality these are issued as tk_live_… keys.)

Example: Instead of five teams sharing one raw provider key, each gets a distinct Virtual AI Key with a $500 monthly cap. When one team's key leaks or overspends, you freeze that one key — the other four keep working, and you know exactly whose dollar it was.

Binding key

A second factor attached to a virtual key so the key string alone isn't enough to spend. A token that leaks without its binding key is a dead key — exfiltrating the string buys an attacker nothing.

Example: A tk_live_… key gets pasted into a public code repo by mistake. Because spending also requires the binding key, which never traveled with it, the leaked string can't be used. What would have been a five-figure incident is a non-event.

Budget cap (hard cap)

A spending limit enforced before a call is sent, not an alert fired after. When the cap is reached, the next call is refused — no spend occurs. A hard cap is a circuit breaker; an alert is a smoke detector that may or may not wake anyone.

Example: A key has a $500 hard cap. A runaway loop tries to spend $34,000 overnight. At $500 the gateway returns a refusal (an HTTP 402) before forwarding the call to the provider, so the loop stops at $500 instead of $34,000.

BYOK (Bring Your Own Key)

An arrangement where you keep your own provider account and credentials, and the governing layer uses your keys on your behalf rather than reselling access. Your provider relationship, spend, and data path stay yours.

Example: You already have an enterprise agreement with your model provider at a negotiated rate. With BYOK, the control layer sits in front of your existing account — you keep your pricing and your contract, and gain attribution and caps on top.

PII pre-flight

A check that scans a prompt for personal or sensitive data before it reaches the model provider, and blocks or redacts it if found. "Fail-closed" means a fault stops the call rather than letting it through — the safe default when a check can't complete.

Example: A prompt contains a customer's full name, email, and card number. A fail-closed PII pre-flight catches all three before the call leaves your perimeter — if the scan itself errors out, the call is dropped, not waved through. The auditor's question "can PII reach your providers?" now has a control behind it, not a hope.

Anomaly detection

Watching spend or usage for sudden deviations from the normal pattern and flagging them the instant they happen, so a spike is caught on the request path rather than in next month's report.

Example: A workload that normally spends $40 an hour suddenly spikes to $1,600 an hour at 3 a.m. Anomaly detection flags the 40× deviation the moment it starts — while there's still time to freeze the key — instead of surfacing it in a report a day later.

Append-only audit

A record that can be added to but never edited or deleted, so the history of what happened is tamper-evident. It's the difference between a log an insider can quietly rewrite and one an auditor will accept as evidence.

Example: Every budget allocation and policy decision is written to an append-only audit. Six months later, an auditor asks what happened during an incident — and the record is provably unaltered, because no one, including an admin, had the ability to change it after the fact.

5. Agents & MCP

AI agent

An AI system that doesn't just answer once but works toward a goal over multiple steps — planning, calling tools, reacting to results, and looping until done. Because an agent can make many calls per task, it can spend far more than a single prompt, and it spends without a human watching each step.

Example: You ask an agent to "research our top five competitors and draft a summary." It makes 40 model calls and a dozen web fetches on its own before returning one document. The spender is no longer a person clicking send — it's software in a loop.

Subagent

A smaller agent spawned by a parent agent to handle a piece of the work. Subagents let one task fan out into many, which is powerful — and a spend-attribution challenge, because their cost needs to roll back up to the agent that launched them.

Example: A research agent spawns five subagents, one per competitor, each running its own calls in parallel. To see the true cost, each subagent's spend has to roll up to the parent agent, which rolls up to the project — otherwise "the agent" is a mystery cost buried in someone's key.

MCP (Model Context Protocol)

An open standard for how AI agents connect to external tools and data sources in a consistent way — a common "plug" so any compatible agent can use any compatible tool. It's how modern agents reach beyond the model to your systems.

Example: Instead of custom-wiring an agent to your database, your ticketing system, and your file store separately, each exposes an MCP interface. The agent speaks one protocol to all three — and a governing layer can watch and gate what flows through that single, standard path.

Agent identity

Giving an autonomous agent its own distinct identity — its own key, budget, and record — rather than having it act under a human's credentials. Without it, an agent's spend and actions are indistinguishable from the person who launched it.

Example: A nightly reporting agent runs under its own identity with a $50 budget and its own audit trail. When it misbehaves, you can freeze the agent without disabling the engineer who set it up — and the record shows the agent acted, not the person.

Tool call

A single action an agent takes to use an external capability — query a database, search the web, run a function — as opposed to just generating text. Each tool call is a step where the agent reaches outside the model.

Example: To answer "what did we bill Acme last quarter?" the agent makes a tool call to your invoicing system, gets the number back, and folds it into its reply. That one tool call is a governable, recordable event — who called what, when, and why.

AI gateway

A single point that all AI traffic passes through on its way to the providers — the place where keys are resolved, budgets are checked, calls are logged, and policies are enforced. It's the choke point that makes control possible, because everything flows through it.

Example: Every model call in the company routes through one gateway. Because it sees every request before it leaves, it can enforce a hard cap, redact PII, serve a cache hit, or refuse an over-budget call — none of which is possible if traffic scatters directly to providers.

Spend control plane

The layer that governs AI spend across an organization: it issues virtual keys, enforces budgets before calls, attributes every dollar, and produces the audit record. "Observability" tells you what you spent; a control plane decides whether you spend it — the enforcement happens before the call, not after. (This is Tokenality's category.)

Example: A control plane connects the systems you already run — projects, people, and budgets — and issues a governed key against each. A call arrives, draws on the right budget, clears a PII check, and either goes through or is refused at the cap — all before a token is spent, with a record left behind.

6. Governance & Compliance

SOC 2

A widely used security-and-controls audit standard (from the AICPA) that assesses how well a service organization protects customer data across areas like security, availability, and confidentiality. A SOC 2 report is often the first thing an enterprise buyer asks for.

Example: A prospect's security team won't sign until they see your SOC 2 report. Having AI controls — access, audit, data handling — already producing evidence means you answer their questionnaire with artifacts instead of promises.

ISO 27001

An international standard for an information security management system (ISMS) — a structured, certifiable program for managing security risk across an organization. Where SOC 2 is common in the US, ISO 27001 is the global benchmark.

Example: An overseas customer requires ISO 27001 certification before onboarding. Because your AI access controls and audit trail map to its requirements, the AI portion of the assessment is evidence you can hand over rather than a gap to close.

ISO 42001

The first international standard for an AI management system (AIMS) — a certifiable framework specifically for governing how an organization builds and runs AI responsibly, including risk, oversight, and accountability. It's the AI-specific counterpart to ISO 27001.

Example: A regulated buyer asks how you govern AI itself, not just your infrastructure. An ISO 42001-aligned program — with documented oversight of AI decisions and spend — answers a question that a general security certification doesn't reach.

NIST AI RMF

The AI Risk Management Framework from the US National Institute of Standards and Technology — a voluntary, widely referenced guide for identifying and managing AI risks across a system's life cycle. Not a certification, but a common language for AI governance.

Example: Your governance program is organized around the NIST AI RMF functions (govern, map, measure, manage). When a customer or regulator asks how you handle AI risk, you can point to a recognized framework rather than an ad-hoc process.

Audit trail

The connected, time-ordered record of who did what, when, and under what authorization — the evidence you produce when someone asks "what happened?" For AI spend, a good audit trail records intent (the budget authorized) alongside execution (what actually ran).

Example: Someone asks "what did the reporting agent do last weekend, and was it allowed to?" A proper audit trail answers both: it shows the budget the agent was authorized against and the calls it made — intent and execution together, not just raw request logs you have to reverse-engineer.

Keep going

If any term here made you think "…wait, can we actually answer that about our own spend?" — that's the useful discomfort. The fastest way to turn these definitions into your own numbers is to look at real data.

• Run the free AI Spend Audit. Connect a usage export and watch the abstract terms above — attribution, cost per successful task, waste — turn into your actual dollars. audit.tokenality.ai

• Watch a call get governed. See a single call hit the ledger, draw against a budget, clear a PII pre-flight, and — if it's over cap — get refused before a token is spent. tokenality.ai/playground

Make every token count.

Tokenality is a preview product; design-partner customers are awaiting sign-off. This glossary describes shipped capabilities only where Tokenality is referenced, and cites public standards (SOC 2, ISO 27001, ISO 42001, NIST AI RMF, MCP, FinOps Foundation) for the rest.