Key Takeaways
MIT's NANDA study found 95% of enterprise generative AI pilots show no measurable P&L impact (MIT NANDA, 2025); S&P Global puts 2025 initiative abandonment at 42%, up from 17% in 2024.
Token price hides the real cost: retries, over-fetched context, and human review time. Cost-Per-Outcome (CPO) counts all of it.
The EU AI Act's high-risk deadline moved. Chatbot transparency (Article 50) still lands August 2, 2026. Annex III high-risk obligations were provisionally deferred to December 2027.
You can build a working CPO baseline in a week. That's not a slogan; it's the first phase of how we run a First Skill Sprint.
Someone on your team is going to ask what the AI budget actually bought. Maybe your CEO already has. If you don't have a number ready, that's not a communication problem. That's the actual problem.
I've watched this play out from both sides now: five businesses, one of them an AI automation company we built too early, and thirteen years of watching teams either adopt change or quietly wait it out. The pattern repeats. Someone negotiates a 10% discount on tokens in January. Nobody's measuring retry rates. By Q2, spend is up, reliability is flat, and the discount is long gone.
What Actually Breaks a Cost-Per-Outcome Model?
Production AI rarely fails because the model is weak. It fails because nothing around the model is instrumented: no evaluation pipeline, no drift monitoring, no one person who owns quality. MIT's Project NANDA reviewed 300+ enterprise AI initiatives between January and June 2025. It found that 95% delivered no measurable P&L impact, despite $30 to $40 billion in enterprise GenAI spend (MIT NANDA, "The GenAI Divide," 2025). S&P Global's October 2025 research tells a similar story. It found 42% of companies abandoned most of their AI initiatives that year, up from 17% in 2024. The average org scrapped 46% of proofs of concept before they ever reached production (S&P Global Market Intelligence, 2025).
Neither report blames the models. Both point at the same gap: nobody built the infrastructure to know if the thing was working. Your moat was never model access. It's evaluation discipline, and almost nobody has it.
Here's the part that surprises people: a model priced 50% cheaper per token is often the more expensive choice in production. Cheaper models frequently need more correction loops to format a JSON object correctly, or generate 15 to 20% more tokens to say the same thing. We call that the verbosity tax. It erases the unit-price advantage before you've even measured anything else.
Why Does Token Pricing Hide the Real Cost?
Token price tells you what the raw material costs. It doesn't tell you anything about the finished product. Manufacturing has a name for this: the hidden factory, the rework and scrap that never hits the P&L but quietly eats the margin. AI operations have their own version. Retry rates. Context over-fetching, paying to ingest 10,000 tokens when 500 would've answered the question. Agent loops nobody scoped.
Want a live example? In June 2026, Salesforce agreed to acquire Intercom (by then renamed Fin) for roughly $3.6 billion. Fin's own pricing page lists $0.99 per resolution, billed only once per conversation no matter how many actions the agent takes, genuinely outcome-based (Fin, "Pricing," 2026). Salesforce's own Agentforce lists at $2 per conversation, or Flex Credits at $500 per 100,000 credits. It also requires a Data Cloud subscription running $108,000 a year or more before it does anything useful. The per-conversation number on the pricing page isn't the number that hits your budget. That's the hidden factory, published in a press release.
So stop asking which model is cheapest per token. Ask which stack delivers the lowest cost per accepted outcome at the quality bar you actually need.
The Cost-Per-Outcome Formula
Cost-Per-Outcome ties spend to results, not infrastructure activity:
CPO = Total Workflow Cost / Number of Accepted Outcomes
Cost-Per-Outcome (CPO) divides total workflow cost, model tokens, retrieval, orchestration overhead, and human review time by the number of accepted outcomes. Most pricing conversations skip the review-time line. It's usually the one that decides the answer: at a 20% escalation rate, human review cost alone can outweigh every infrastructure cost combined. That is not a flaw in the formula. Removing the review step is what produces the silent failures behind MIT NANDA's 2025 finding that 95% of enterprise AI pilots show no measurable P&L impact. CPO treats the review gate as part of the outcome, not overhead subtracted from it. A team that hasn't measured this baseline typically doesn't know whether they're paying €0.50 or €5.00 per successful transaction, which is itself the first thing a CPO audit should fix.
Every approval gate we build is there because the human is still in charge. That's not a compliance checkbox. It's the actual point.
A 30-Day Plan, Mapped to How We Actually Build
You can't optimize a number you haven't measured yet. This is roughly the sequence we run inside a First Skill Sprint. It maps cleanly onto the four thinking skills Anthropic calls AI fluency: Delegation, Description, Discernment, and Diligence.
Week 1, Delegation: instrument and baseline the workflow before touching anything.
Week 2, Discernment: build the evaluation gate that decides what to trust.
Week 3, Delegation, refined: route and cache against the baseline you now have.
Week 4, Diligence: shadow mode, then ship only if CPO actually improved.
Week 1: Delegation
Instrument and baseline first. Pick one narrow, high-value workflow. Log token usage by team and step. Find out if you're currently paying €0.50 or €5.00 per successful transaction, because right now you don't know, and neither did we before we measured our own builds.
Week 2: Discernment
Build the evaluation gate. Set real acceptance criteria for quality, latency, and error tolerance. Add CI checks. Use a frontier-grade model to score the outputs of your faster, cheaper one. This is the human-in-the-loop layer, made concrete instead of theoretical.
Week 3: Delegation, refined
Route and cache. Turn on semantic caching for repeat queries, which typically cuts LLM spend 15 to 30% at normal hit rates. Send simple requests to cheap models, hard ones to reasoning models. Measure the delta against your Week 1 baseline, not against a vendor's benchmark.
Week 4: Diligence
Shadow mode, then ship. Replay real traffic through the optimized stack before it touches a customer. Ship only if quality holds and CPO actually improved. If it didn't, you now know that before your customer does, not after.
I ran a version of this loop for a full year before PromptMetrics existed. No client, no brief, just synthetic data and sandboxes, pushing to see where a stack actually broke. The pattern held every time: the teams that measured first were the only ones who could tell you, with a number, whether the "optimization" helped. What Changed With the EU AI Act Deadline?
Short answer: part of it moved, part of it didn't, and the difference matters more than the headlines suggest. On May 7, 2026, EU negotiators reached a provisional agreement on a Digital Omnibus. It defers Annex III high-risk obligations, the use-based category most workflows fall under, from August 2, 2026 to December 2, 2027 (Gibson Dunn, "EU AI Act Omnibus Agreement," 2026). Article 50 transparency obligations, covering chatbot disclosure, still take effect on August 2, 2026, on the original schedule and are unaffected by the delay. If the Omnibus isn't formally adopted before that date, the original text and timeline apply as written instead. For most teams building AI workflows now, Article 50 alone is reason enough to build governance in from day one. The same logging that supports a CPO baseline maps directly onto Article 12 record-keeping requirements, regardless of which deadline applies.
This is also, honestly, good news for the CPO argument. You don't need a looming deadline to make the case. Enterprise buyers are already asking versions of the same three questions during governance review, and a CPO dashboard changes the quality of the answer:
Buyer's question | The vague answer | The CPO answer |
|---|---|---|
How do you handle model drift? | "We monitor it." | "We track CPO variance by workflow. Past a 15% spike, it auto-rolls back to the last stable prompt snapshot." |
What happens if a provider goes down? | "We have backups." | "Our router fails over to a secondary provider. We know that adds about €0.02 per transaction, and it fits inside our margin buffer." |
Is this compliant? | "Yes, we follow the rules." | "We log every decision step and cost component. That log maps directly to Article 12 record-keeping and Article 50 transparency requirements." |
"We track CPO variances and auto-rollback past a 15% spike" is a real answer. "We monitor it" isn't.
What Does This Actually Cost to Build?
A working Cost-Per-Outcome baseline takes about a week to build for one narrow workflow: instrument logging, pick the outcome, and measure current cost per accepted result. That is the first phase of what PromptMetrics runs inside a First Skill Sprint, a fixed-scope engagement delivering one governed, human-reviewed skill as an editable markdown file, not a black box. A longer Flagship Pilot extends the same baseline-first method across five to ten operators over four to six weeks, measured before and after. Neither engagement claims prior scale PromptMetrics hasn't earned; the company is rebuilding from scratch in 2026 and says so plainly. The first conversation about which one fits, if either does, is free and unscoped, and whatever gets built stays yours to read, run, and modify afterward.
FAQ
What is cost-per-outcome (CPO) pricing? CPO divides total workflow cost, including model tokens, retrieval, orchestration, and human review time, by the number of accepted outcomes. It measures what a business result actually costs, not what a unit of infrastructure activity costs.
Why does a cheaper AI model sometimes cost more in production? A model priced lower per token often needs more retries to format outputs correctly or generates 15 to 20% more tokens to convey the same answer. That "verbosity tax" can erase the unit-price advantage once you count the full workflow cost.
Does the EU AI Act still take effect on August 2, 2026? Partly. Article 50 transparency obligations for chatbots are still scheduled for August 2, 2026. Annex III high-risk obligations were provisionally deferred to December 2, 2027 under the May 2026 Digital Omnibus agreement, pending formal adoption.
How long does it take to build a CPO baseline? About a week for one narrow workflow: instrumenting logging, picking the outcome, and measuring current cost per accepted result. That's the first week of what we run inside a First Skill Sprint.
What happens to the human review step in a CPO model? It stays, and it should. Human review is often the highest single cost in a CPO calculation, but removing it is what produces the silent failures behind MIT's 95% no-impact finding. Treat it as the gate that protects the outcome, not overhead to cut.
