AI cost terms: tokens and inference
AI is priced mostly by tokens, units of text processed, and the dominant ongoing cost is inference, the act of running the model to answer. Inference is roughly two-thirds of AI compute spend, not training.
Why inference dominates the bill
Training a model is a large one-time cost, but running it, inference, happens on every query and accumulates. That is why inference is about two-thirds of compute spend and why token volume drives your budget.
Leaders often fixate on the cost of “building” AI and miss where money actually goes. Most businesses use existing models, so training is not their cost, inference is. Every question sent to a model processes tokens, and you pay per token, so a high-traffic use case can cost far more than a flashy one-time build. Inference is roughly two-thirds of AI compute spend industry-wide. This is exactly why owned-versus-rented economics matter: at high, steady token volume, running your own model can dramatically undercut per-token API pricing.
Frequently asked questions.
How do I keep AI costs from spiraling?
Measure token usage per use case, use the smallest model that meets the quality bar, cache and reuse results where possible, and monitor inference volume as adoption grows. Costs spiral when high-traffic features run on oversized models without anyone watching token consumption.