You bought the licenses. You thought you were buying AI.
What you actually bought was access.
That distinction is going to matter a lot more in the next twelve months than it did in the last twelve.
An AI consultant told Axios in May 2026 that one of their clients spent $500 million in a single month on Claude licenses. The cause was simple: no usage limits on employee access.
That’s not a story about a reckless company. It’s a story about a normal company that did what most companies did — deployed AI broadly, moved fast, showed adoption numbers to the board, and didn’t build the architecture underneath it.
The same month, Microsoft canceled most of its internal Claude Code licenses. Uber’s COO said AI costs were getting harder to justify.
The era of “turn on AI for everyone and see what happens” is ending. What comes next requires building something most enterprises skipped: an operating architecture between employees and AI models.
The Scale of What’s Coming
Token prices are falling. Enterprise AI consumption is rising. The gap between them is the problem.
In June 2026, the Linux Foundation announced the Tokenomics Foundation at FinOps X — a signal that managing AI spend has moved from a niche concern to a core operating discipline.
This is no longer just an IT cost-management problem. It’s an operating-model problem. And it’s going to get harder before enterprises get control of it — unless the architecture changes.
BCG Named the Levers. Most Enterprises Can’t Pull Them Yet.
BCG’s framework on managing AI token costs is worth studying carefully. (link) Their RoAI framework makes an important point: the unit of management isn’t tokens. It’s cost per successful outcome. Their four levers — eliminate unnecessary model use, route by task complexity, reuse context and components, train for token discipline — are right.
The problem isn’t knowing what the levers are. It’s having the infrastructure to pull them consistently.
Pulling those levers requires someone to make the routing decision, enforce the context discipline, and measure the outcome — workflow by workflow, team by team. In most enterprises, nobody is doing that systematically. Not because they don’t know they should. Because the architecture that would make it automatic doesn’t exist yet.
The cost difference between routing every request to a frontier model versus routing by task complexity isn’t marginal — it’s structural. Not because one company negotiated a better contract, but because one company made a better architectural decision.
BCG tells you which levers to pull. This piece is about building the architecture that pulls them systematically.
A Clarification Before We Go Further
This architecture addresses one specific problem: enterprise employees using AI to do their work.
The problem is particularly acute in services firms, where delivery and AI consumption are increasingly the same thing — making governance not optional, but structural.
Systems your engineering team builds with LLMs for customers or products — customer-facing applications, embedded agents, product features — have their own cost architecture managed at the application layer. That’s a different problem and deserves its own treatment.
Everything that follows is about the humans inside your organization and the AI they use to do their jobs.
Why Seat Licenses Aren’t Enough
Seat licensing solved the access problem. It didn’t solve the control problem.
And that’s not a criticism of the companies that deployed broadly. The pressure to show AI adoption was real. Procurement is fast. Architecture takes longer. When the board wants to see AI in the hands of employees by Q2, you buy licenses. You figure out the architecture later.
Later has arrived.
When every employee has a Claude, ChatGPT, or Gemini license, every employee is making their own decisions: which model, how much context, how long the response should be, whether a task needs a frontier model, whether an agent should run three steps or thirty. One CTO told Axios that employees were using AI models to check the weather.
The individual decision seems harmless. Multiply it by thousands of employees, millions of interactions, increasingly sophisticated agents, and increasingly large context windows, and you have a cost structure that nobody is really managing.
Traditional SaaS FinOps wasn’t designed for this. A SaaS seat has a predictable cost. AI inference has a variable cost influenced by prompt length, context, model choice, reasoning effort, tool calls, and agent loops. The cost only becomes visible once you move past pilots into real volume — and by then the architecture is already locked in.
The seat license isn’t wrong. It’s incomplete. Access without architecture creates consumption without control.
The Architecture: Three Layers
The answer isn’t to take AI away from employees. It’s to stop treating individual licenses as the primary enterprise AI operating model.
Most enterprises today route AI like this: Employee → AI Vendor → Model. What’s needed is: Employee → Enterprise Control Plane → Governed Workflow → Model. And for the people building those workflows: AI Expert → Authoring Layer → Governed Workflow → Control Plane → Model.
Three distinct layers. Three distinct jobs.
The first is the employee. Employees should still be able to experiment, explore, brainstorm, and solve one-off problems. Open-ended access has real value — don’t eliminate it. But experimentation shouldn’t automatically become the architecture for scaled production work. When a task becomes repeatable, valuable, and widely used, it should become a workflow.
The second is the authoring layer. A smaller group of AI practitioners, workflow designers, and domain experts builds those workflows. They choose models deliberately, design prompts and context, connect enterprise data and tools, test approaches, evaluate quality, establish guardrails, measure cost, and publish approved workflows. Their output isn’t another prompt. It’s a parameterized, tested, governed tool that anyone in the organization can use. This is where organizational intelligence starts becoming reusable — and compounding.
The third is the enterprise control plane. The control plane sits between the organization and its AI providers — frontier models and open weight models alike. It handles model routing, authentication and authorization, policy enforcement, cost attribution, usage limits, context management, caching, observability, and workflow access. The architecture moves the decision from the individual user into the system.
What This Looks Like in Practice
Consider a consulting firm with 2,000 consultants.
Today, consultants independently use AI to analyze interview transcripts, summarize research, draft proposals, and synthesize client material. Some use a lightweight model. Some use a frontier model. Some paste enormous amounts of context into every request. Some build elaborate agents. Some repeat the same workflow manually every week. The firm has little visibility into which approach produces the best result for the lowest cost — and no way to transfer what one consultant figures out to anyone else.
Now imagine a senior consultant and an AI practitioner build a governed Client Interview Synthesizer. The consultant uploads a transcript. The workflow extracts structured information using a lower-cost model, retrieves the firm’s approved methodology and reference material, applies standardized instructions, escalates ambiguous synthesis to a more capable model, produces a consistent output, records the cost and quality metrics, and captures where human intervention was required.
The next consultant doesn’t need to know which model to select. They don’t need to understand prompt caching. They don’t need to decide how much context to include. They use the firm’s tool.
The workflow can now be evaluated against a benchmark set of interviews: Was the extraction accurate? Did the synthesis meet the firm’s quality standard? How much did each run cost? How often did a consultant have to intervene? That feedback loop makes the tool better over time.
Subsequent uses can become cheaper, faster, and more consistent. The firm’s knowledge compounds. Individual heroics become institutional capability.
That’s the real value of the authoring layer. And it’s completely invisible without the control plane underneath it.
How the Control Plane Pulls BCG’s Levers
BCG’s four levers become far more powerful when enforced through architecture rather than left to individual behavior.
Routing by task complexity becomes an architectural decision rather than a user decision. A workflow can start with a lower-cost model and escalate when confidence or complexity requires it. Summarization and extraction go to lightweight models. Complex reasoning goes to frontier models. No user decision required.
Reusing context and components becomes systematic. Prompt caching, shared context, stable prefixes, and common policies dramatically reduce repeated processing. The infrastructure handles it — users never have to think about it.
Training for token discipline becomes structural. Output constraints, context limits, and stopping rules are designed into the workflow rather than left to individual habit. Brevity becomes a default, not a best practice someone has to remember.
BCG frames these as levers to pull. The control plane makes them self-executing — and a human expert in the loop makes the execution progressively smarter. The architecture captures which routing decisions produce the best outcomes. The expert uses that evidence to refine the tools, adjust the thresholds, and improve the quality bar over time. Automation handles the routine. Judgment handles the edge.
The Missing Layer: Evaluation
There is one more capability that becomes essential as AI moves from experimentation to infrastructure: evaluation.
If the objective is cost per successful outcome, you need to know what successful means. For each important workflow, enterprises should be able to see cost, latency, quality, human intervention rate, and business outcome. That creates a continuous feedback loop: build, deploy, observe, evaluate, improve, redeploy.
This is where architecture and human expertise compound together. The infrastructure provides the data. The experts interpret it. The workflow gets better. The organization accumulates knowledge about which models, prompts, contexts, and approaches actually work — and that knowledge doesn’t walk out the door when someone leaves.
That’s very different from giving everyone a license and hoping usage patterns evolve into something productive.
The Tools Exist. The Decision Is Architectural.
None of this requires inventing new technology. The gateway and control plane market is now substantial, with mature options across the build-versus-buy spectrum.
For organizations with engineering capacity to self-host, open-source options are production-ready — providing enterprise governance, multi-provider routing, and lightweight proxies that support a wide range of model providers with minimal latency overhead.
For organizations that prefer managed infrastructure, commercial platforms handle the operational burden — combining routing, fallbacks, caching, guardrails, and compliance controls in a single managed service. Some have been acquired into broader security platforms as AI governance has become a security concern. Others extend existing API management deployments or suit edge-first architectures.
Open source or commercial, self-hosted or managed — the architecture is the same. A control plane between your organization and your models, with routing, governance, attribution, observability, evaluation, and a governed workflow catalog sitting on top.
What most enterprises are missing isn’t the vendor. It’s the architectural decision.
The Economics
The more important economic argument is control, not unit cost.
Seat licensing gives you a relatively predictable access cost, but limited visibility into the economics of the work happening through it — and no structural way to govern it. A control-plane model makes AI consumption attributable and governable — tracked by workflow, team, business unit, and outcome.
You can’t measure return on AI at the workflow level without workflow-level attribution. You can’t get workflow-level attribution without a control plane that sees the workflow.
The cost advantages of tiered routing, prompt caching, and batch processing for asynchronous workloads are real and available today. None of them require negotiating a better contract. They require making a better architectural decision.
What to Build First
The sequence matters — and most enterprises get it wrong by starting with the tools catalog before the infrastructure exists to govern it.
Start with the control plane. Get routing, identity, governance, attribution, observability, and evaluation in place. That’s what gives you the visibility BCG describes in their “See” phase. Without it, you’re flying blind.
Then build the authoring layer. Identify your power users. Give them the environment to build, test, evaluate, and publish workflows. Establish the quality bar. Put human experts in place who are accountable for what gets built and for improving it over time.
Then open the catalog. Users access the tools. The control plane routes requests to the appropriate model. Usage and outcomes are captured. The organization learns.
Then optimize continuously. Models change. Prices change. Workflows change. The architecture should allow you to swap the underlying model without forcing thousands of employees to change how they work. The employee sees a useful tool. The enterprise sees a governed, measurable, improvable system.
You bought the licenses. Now build the architecture.
Does your organization have workflow-level visibility into what your AI is doing, what it costs, and what it produces — or just a token bill at the end of the month?
I write about operationalizing AI in services businesses. If this resonates, I’d like to hear what you’re building — or struggling to build.