The token economics
Around the crossover of roughly 3 billion tokens a month or 80%+ hardware utilization. Above it, owning can be up to 17x cheaper per token; below it, idle hardware makes renting cheaper.
Utilization is everything
Owned hardware only pays off when it runs hot. Below ~80% utilization you pay for idle silicon; above it, amortized cost per token collapses far below API pricing.
Inference is now roughly two-thirds of all AI compute and, unlike training, it is a steady, predictable workload, ideal for amortizing owned hardware. The mistake is buying hardware for peak demand and running it half-idle; that erases the advantage. Size owned capacity to your median steady load, keep utilization high, and burst to rented APIs for peaks, and the token economics work in your favor.
Frequently asked questions.
Why include labor in the cost comparison?
Because it dominates hidden cost. Raw GPU rental understates reality by 3–5x once you add the MLOps engineers, monitoring, power, and cooling needed to run owned infrastructure reliably.