Your CEO said, "We need to be doing AI," looked at you, and walked off. That was the whole brief. Now you're the one who has to explain why the AI line item tripled last quarter, and "the tokens got cheaper" is not an answer the board will accept.
You've watched this movie before. It cost $44.5 billion last time.
A decade ago, cloud infrastructure was supposed to save everyone money. Pay for what you use. Scale down when you don't. No more CapEx hardware rotting in a closet. Then the bill arrived. And kept arriving. Cloud waste alone hit $44.5 billion in 2025 (Harness, FinOps in Focus, February 2025), with 91% of organizations reporting at least some waste (HashiCorp/Forrester, 2024) and an average 31-day lag before anyone spots the spike.
AI is running the same pattern, faster. An agentic workflow can spawn 50 inference calls per user request. Thirty-one days is an eternity when costs compound that fast.
Key Takeaways
Token prices fell 92% in 17 months (GPT-4 to GPT-4o), yet enterprise GenAI spend grew 16x, from $2.3B to $37B (Menlo Ventures, Dec 2025).
Agentic workflows and hidden "reasoning tokens" are the multipliers: reasoning-token use per org rose roughly 320x in 12 months (OpenAI usage data).
The fix isn't waiting for prices to stabilize. It's tagging spend by feature before the mandate becomes a crisis.
Why Doesn't Cheaper AI Mean Cheaper Bills?
Because usage is expanding faster than unit prices are falling, and nobody's tracking the multiplier. GPT-4 launched at $30 per million input tokens in March 2023. GPT-4 Turbo dropped that to $10. GPT-4o opened at $5 and settled at $2.50 by August 2024, a 92% reduction in 17 months. Andreessen Horowitz calls the broader trend "LLMflation": roughly a 1,000x reduction in inference cost per unit of capability over three years. Epoch AI (March 2025) puts the annual decline at 40x to 200x to match GPT-4's original performance.
Meanwhile, total enterprise GenAI spend went from $2.3 billion in 2023 to $37 billion in 2025, a 16x increase (Menlo Ventures, State of Generative AI, December 2025). OpenAI's own compute footprint tracked the same curve, growing from 0.2 GW in 2023 to 1.9 GW in 2025. Prices fell 1,000x. Spend grew 16x anyway. That gap is the entire problem, and it's a usage problem, not a pricing problem.
Jevons Paradox: The 160-Year-Old Warning Behind Your Bill
In 1865, economist William Stanley Jevons noticed that more fuel-efficient steam engines led to more total coal burned, because cheap energy made more uses worth it. Microsoft CEO Satya Nadella named the same effect in January 2025: "Jevons paradox strikes again! As AI gets more efficient and accessible, we will see its use skyrocket." Three specific mechanisms are doing that right now.
The Agentic Multiplier
Cheaper inference didn't just make chat cheaper. It changed how software gets built. A single "fix this bug" request to a coding agent isn't one API call; it's a loop: plan, draft, test, error, refine, verify. One user action can spawn 5, 10, or 50 inference calls. OpenAI's own usage data shows average reasoning-token consumption per organization rose roughly 320x in the 12 months leading into 2025, which is agentic architecture reshaping usage patterns, not a fluke.
What Are Hidden Reasoning Tokens, and Why Do They Cost More?
Reasoning tokens are the internal "thinking" steps models like OpenAI's o1 and o3 generate before producing a visible answer, and they're billed at output rates 3-5x more expensive than input. A prompt might show 100 input tokens and 50 visible output tokens on the invoice, with 5,000 hidden reasoning tokens behind it that you never see. Cost stopped tracking answer length. It now tracks problem complexity, invisibly.
Why Does EU Compliance Add a Real Cost Multiplier?
If you operate in Europe, the cheapest model isn't the one with the lowest token price; it's the one that doesn't trigger regulatory exposure. Under the EU AI Act, high-risk use cases like HR screening or credit scoring carry compliance costs estimated up to €400,000 per system, roughly 17% of total AI investment, against fines up to €35 million or 7% of global turnover for non-compliance. Here's the reframe most teams miss: that governance layer isn't a separate cost bolted onto your AI spend. Built correctly, it's the same infrastructure that gives you the visibility this whole article is about. A human review gate that catches a runaway 50-call agent loop is doing compliance and cost control in the same motion.
What's Actually Costing You AI Budget Visibility?
Four missing decision-making skills, not one generic tooling gap, explain almost every AI budget surprise. Most "control your AI costs" advice reads like a checklist; sorting the failure modes by which skill is actually missing tells you what to fix first.
Missing discipline | What's actually happening | The fix |
|---|---|---|
Diligence | Visibility fragments as usage scales. Prompt chains, RAG stacks, and fine-tuning jobs pile up under one API key, so spend can't be attributed to a feature or team. | Audit hooks and data-flow documentation exist before anyone asks for them, not after. |
Discernment | Your FinOps tooling can't see prompts. Most platforms were built for compute and storage, not prompt-driven workloads, and can't tell you that a poorly written prompt costs 10x more. | A human judgment call on whether a feature or model swap is worth the cost. No dashboard makes this decision for you. |
Description | "Cheap" enables waste at scale. At $30 per million tokens, optimizing a prompt paid for itself immediately. At $0.15 per million, an unoptimized prompt run 10 million times still costs $1,500 a month instead of $150. | Writing a tight, tested prompt once volume is high, the same discipline expensive tokens used to force by default. |
Delegation | The optimization window is closing. Deciding which tasks AI should own gets harder once you've got dozens of features, three model providers, and habits already built around unoptimized prompts. | Decide now, while it's cheap to decide, which tasks stay human and which get automated. |
In the workflows we've mapped with early PromptMetrics clients, the Diligence gap shows up almost every time: nobody can say what a single "agent run" actually costs until someone tags it. That's not a tooling gap. It's a five-minute decision nobody made yet. Cloud taught the same lesson once already: teams that built FinOps discipline early saved far more than the ones who scrambled during a budget crunch. The 4 Ds of AI Budget Control 1 Diligence MISSING Spend hides across chains, keys, and jobs FIX Audit hooks before anyone has to ask 2 Discernment MISSING Tools can't judge prompt quality FIX A human call on model or prompt swaps 3 Description MISSING Cheap tokens kill prompt discipline FIX Tight, tested prompts as the default habit 4 Delegation MISSING Habits calcify as features stack up FIX Decide now what stays human, early Read left to right: where AI budget visibility breaks down, and what closes each gap.
Governance Is a Hedge, Not a Bet
The real question isn't whether this gets expensive. It's when, and that depends on a variable you don't control: where token prices go next.
Bull case: prices keep falling, Jevons Paradox kicks in harder, agents run wilder, total spend keeps rising.
Bear case: subsidies end, energy constraints bite, unit prices rise, your unit economics get worse either way.
Both scenarios end in the same place: rising total spend. The only move that works in both is building cost visibility and governance now, so you can throttle volume in a cheap-price world and optimize unit cost in an expensive one. This is exactly what a governed AI implementation should give you by default, not as an add-on: a human stays in the loop on every high-cost decision, nothing about how a workflow spends money is a black box, you measure the baseline before anything gets touched, and someone stays accountable after launch instead of disappearing after kickoff.
Is This Actually Your Problem Yet?
Maybe not, and it's worth saying so plainly. If you're pre-product-market-fit, buying an enterprise FinOps platform is premature; you have bigger things to prove first. But "not yet" isn't the same as "never," and the same 31-day detection lag that costs mature organizations $44.5 billion a year (Harness, 2025) applies just as much to a two-person team as it does to an enterprise FinOps function. The gap doesn't need enterprise scale to hurt; it just needs enough time to compound unnoticed.
So if your AI spend is anywhere near €1,000 a month, don't ignore this entirely. Start tagging API calls now, before there's anything worth optimizing. Building the habit of tracking cost-per-feature costs nothing today. Retrofitting it later, once you've got millions of unlabeled logs and no record of which feature generated which cost, is expensive, slow, and usually happens right after the bill that finally got someone's attention.
What Can You Do About AI Costs This Week?
You don't need a six-month program. Four things, in order:
Tag every API call with which feature triggered it and which team owns it. This is the foundation everything else depends on.
Calculate cost per core action: what it actually costs to generate one customer response or run one agent workflow end to end. Without this number, model selection is a guess.
Set one alert. Pick your biggest AI cost center, set a daily threshold, and make sure a person gets notified if it's crossed. This alone would've prevented half the cloud cost horror stories of the last decade.
Map your data sensitivity. If GDPR or the EU AI Act applies, know which prompts touch personal or high-risk data before you pick a model or a region. A legal mistake costs more than any amount of token optimization saves.
That first step, tagging calls with a human-reviewed rule for what counts as a feature, is exactly the kind of single governed workflow a First Skill Sprint builds: one workflow, live in one to two weeks, with a human checking it before it ships.
Frequently Asked Questions
Why did enterprise AI spending grow 16x if token prices dropped 1,000x?
Usage expanded faster than prices fell. Enterprise GenAI spend rose from $2.3B in 2023 to $37B in 2025 (Menlo Ventures, Dec 2025), a 16x increase, while per-token cost to match GPT-4's original performance fell by roughly 1,000x over the same period (a16z).
What is the Jevons Paradox, and why does it apply to AI costs?
Jevons Paradox, named for economist William Stanley Jevons in 1865, describes how efficiency gains increase total consumption rather than reducing it, because cheaper access unlocks more use cases. Microsoft CEO Satya Nadella cited it directly in January 2025 to explain rising AI usage as prices fall.
What are hidden reasoning tokens, and why are they expensive?
Reasoning tokens are internal "thinking" steps that models like OpenAI's o1 and o3 generate before producing a visible answer. They're billed at output rates, typically 3-5x more expensive than input rates, and can run into the thousands per query without appearing in the visible response.
How much does EU AI Act compliance add to AI project costs?
High-risk AI systems under the EU AI Act, covering uses like HR screening or credit scoring, carry estimated compliance costs of up to €400,000 per system, about 17% of total AI investment (EU AI Act architecture analysis). Non-compliance penalties run up to €35 million or 7% of global turnover.
What's the first practical step to controlling AI spend?
Tag every API call with the feature and team that triggered it. This single step, done before volume scales, is what makes every other cost-control measure, from model selection to budget alerts, possible later.
The Pattern Is Clear
Cloud waste is a $44.5 billion problem despite a mature FinOps industry. AI is on the same trajectory, moving faster. The people who instrument their AI spend and governance before it's urgent are the ones who actually capture the value of falling token prices. The ones who wait will be writing the same "how we cut our AI costs by 40%" post in 2027 that's been written about cloud for the last five years.
The only question is which one you want to be: the person who got ahead of it, or the person explaining to the board why last quarter's AI bill ran 3x the forecast.
