AI tokenomics: Balancing open-weight and frontier models for optimal ROI

Why airlines don't fly all-first-class planes and what it teaches us about AI strategy

Airlines learned very early that they can't optimize a profitable route selling every seat as first class. Instead, the aircraft has multiple cabins, and the airline sizes each according to the route, the season, and the customers who use it. Operational flexibility is what makes the route economics work.

Now picture the alternative: a plane with only first class seats available, which few of the passengers can afford. Either the planes fly with vacant seats, or the airline operates at a loss by offering a first-class experience at a discounted price. Worse, passengers keep arriving faster than the fares can drop. Worst of all, imagine if the manufacturer only built the aircraft one way—no configuration option, no choice of layout. The airline is stuck with an economics problem it can't engineer its way out of.

When it comes to AI models, most enterprises are flying that metaphorical plane today. A simple use case like checking a bank account balance and a complex case like a multi-agent fraud investigation both travel first class, at the same price per token. But do all your AI use cases deserve a first-class seat? Or would cabin classes give you better economics and, just as importantly, the control and governance that come from deciding who sits where?

An AI gateway control plane gives you that choice. By putting open-weight and frontier models, agent interactions, and Model Context Protocol (MCP) tool calls behind a single open hybrid cloud control plane, routine AI traffic stops paying frontier prices, lowering total cost of ownership (TCO). The frontier budget concentrates on the use cases that truly move the business, which maximizes the return on investment (ROI) of AI. An AI gateway built with Red Hat AI provides routing, guardrails, and observability across both open-weight models on your own infrastructure and frontier APIs.

We've modeled this end-to-end: the workload mix, infrastructure costs, operational elements, and compliance costs on both sides to see what the business value looks like. This article shares what we found, including the inputs you'll need to build the same case for your own environment.

What has changed: The lower-fare cabins now provide better value

An AI gateway gives you the ability to choose between frontier and open-weight models, and to map the right model to the right use case.

Four years ago, the objection to model tiering was fair—the cheaper cabins just had little to offer. That objection no longer holds true.

In August  2026, the LLM Token Expenditure Index—Silicon Data's daily gauge of what the market is paying per million tokens—fell to 97 cents. This is its lowest reading since launch late last year, and less than half its summer high.

This number matters for 2 reasons. It confirms frontier pricing is under real competitive pressure, and it tells us where the pressure is coming from. Reporting on the same index, an analyst described the moat shifting away from raw model capability, where the open-weight gap is now measured in months.

The market is making the same bet with capital. In September 2026, NVIDIA entered a definitive agreement to acquire Hugging Face, the main distribution point for open-weight models, for approximately $12.9 billion (subject to regulatory approval). The company selling the hardware is investing at scale on the expectation that models organizations host themselves will run a growing share of enterprise inference.

Open-weight models—whose parameters are published so you can download, inspect, modify, and run them yourself—have closed most of the practical gap on the bulk of enterprise traffic. A well-chosen 8 billion token (8B) model handles classification, extraction, and routine chat at a quality users can't distinguish from frontier output. Models in the 30 billion token (30B) to 120 billion token (120B) range handle domain reasoning that previously required a frontier endpoint. The frontier still leads on the most complex agentic AI work, which is why it should be  reserved for those sorts of jobs rather than lower grade tasks that can be handled by smaller, more efficient models.

This maturity is what enables flexibility: route each class of request to a model sized for it, and customize those models with your own knowledge, instead of sending everything to an expensive endpoint at whatever price the market sets.

One caution before you plan around less expensive tokens. A falling index makes each call cheaper, but it doesn't change how many calls you make. Enterprises running AI at scale are still spending more this quarter than last, because volume is rising faster than price is falling. Cheaper tokens are the tailwind. Routing is the lever.

Why the bill rises when the price falls

Google provides the hyperscaler view: 9.7 trillion tokens a month in May 2024, roughly 480 trillion by May 2025, and over 3.2 quadrillion by May 2026. The enterprise view is arriving at the same place, just a step behind. Bank of America's Erica virtual financial assistant has passed 3 billion cumulative client interactions and now runs at tens of millions a month. Wells Fargo's Fargo assistant logged 245.4 million interactions in 2024, more than double the bank's own projection. Meanwhile, JPMorgan has put in-house generative AI tools in front of more than 200,000 employees.

None of these firms publishes token counts, but the arithmetic is straightforward: at a conservative 1,000 tokens per interaction, a single customer-facing assistant at Erica's scale consumes hundreds of billions of tokens a year. Add internal copilots at JPMorgan's seat count, where knowledge work carries far longer context windows, and a large institution can easily cross a trillion tokens a year.

Agents are further accelerating that curve. One human request used to mean 1 model call. An agent breaks a request into steps, calls a model for each, carries full context on every hop, and retries whatever fails. So even as token prices halve, volume multiplies, and the overall platform cost continues to climb.

Applying FinOps solutions to AI tokenomics

None of this is purely a new discipline. It's workload management and financial operations (FinOps) applied to a new unit of consumption and hybrid cloud infrastructure.

The FinOps Foundation notes in FinOps for AI that cost-per-token, volatile pricing, and GPU scarcity bring real challenges. The response is familiar—track costs regularly, set quotas, tag resources, and optimize GPU allocation. Their guidance on model selection is the part most enterprises skip:

"It's crucial to avoid using the most complex and expensive models for every task, as this often leads to unnecessary costs."

They attach a key performance indicator (KPI) to it, LM Model Choice Quality Score Alignment, measuring the gap between the quality a task requires and the capability of the model to which it was routed. That gap represents measurable platform waste.

From a governance and control perspective, practices like showback and unit economics carry directly into this platform architecture. Once teams see their own consumption data, showback changes developer behavior at the API boundary before you charge anyone anything. Unit economics reframes the conversation from total spend to cost per business outcome, the version C-level executives can act on.

How we calculated it: The inputs, the assumptions, and the results

The parameters a frontier API lets you ignore are the same ones a credible business case often depends on. And the frontier API approach asks almost nothing of you: take a key, accept a price, and start building. That simplicity is why shadow AI spreads so quickly.

Model tiering and routing is more complicated. You have to know your real request mix, tokens per request, throughput per GPU, utilization, concurrency at peak, and AI performance and sizing metrics. An open hybrid cloud AI control plane architecture didn't invent those parameters, but a metered endpoint lets you avoid measuring them.

This trade cuts in your favor. The inputs the frontier approach allows you to skip are the same inputs that give you control. You can't right-size what you never measured, attribute cost to a team you never tracked, or show a regulator a routing decision you never made.

Here's what we measured, and what it produced.

Framework and inputs

1. Request classification: Sort traffic by what the task genuinely demands.

2. Load distribution and tokens per request: In our reference scenario, we used 20M requests a month on a 30/30/20/20 split across the easy, medium, hard, and complex tiers with 1,000/5,000/8,000/10,000 tokens per request, a 20% Year-over-Year (YoY) growth rate, and a 20% rate limiting issues buffer. That asymmetry reflects real-world scenarios and gives us roughly the requirement of handling 9 trillion tokens over 5 years.

3. Infrastructure fit: We used token requirements to size the GPU types and volume. Tokens per second is the primary sizing input, with time to first token (TTFT), latency, and concurrency shaping the result. We base our sizing on industry benchmarks, tailored to specific use cases. 

4. Utilization: On-premise capacity planning varies from an on-demand frontier model, so we include utilization as a sensitivity assumption in the model. We run at 75% utilization to more accurately reflect real-world idle capacity.

What each side of the comparison carries

Our directional findings

Treat these findings as directional—we built a reference scenario on published list pricing, but you need to validate it against your own telemetry before putting it in a presentation to the board. Also, all calculations include a margin of accuracy of roughly 40% to temper the benefits.

Using only a frontier API: Token spend is the dominant and only growing line, roughly doubling over 5 years at 20% annual query growth, before adding DLP tooling, audit readiness, and support that scale with spend.

Using only on-premise, multi-model: Capital is front-loaded, then costs flatten. We observed a near 80% savings against a top-tier frontier model, around 70% against a mid-tier model, and under 20% against a bottom-tier one. Your baseline model choice, not the architecture, determines cost savings. Also, the cash flow can be spread over multiple years depending on the purchasing choices of the customer.

Figure 1: 5-year TCO, all-frontier versus all on-premise, open-weight models (directional, reference scenario)

Hybrid scenario: Easy, medium, and hard use cases are directed to open-weight on-prem models while complex use cases are routed externally to frontier models. This gave the strongest TCO benefits of the 3 scenarios we looked at (~89% TCO reduction) and was also the least disruptive, because nothing moves before it is ready.

Figure 2: 5-year TCO, all-frontier versus hybrid, with complex requests routed to frontier models (directional, reference scenario)

Benefits of an AI gateway built with Red Hat AI

Red Hat AI provides an AI gateway control plane that makes the following scenario possible:

• A developer picks the right model through 1 gateway and 1 token, without filing a ticket and without going around IT to do it.
• The routine, low-value AI traffic runs at near-zero marginal cost, and the frontier budget is spent only when a frontier model is truly needed.
• Customer data and personally identifiable information (PII) stay entirely within your domain, and every model version, data source, and policy decision is logged in a single place.
• The chief financial officer (CFO) can see ongoing AI spend by team, application, and provider, rather than as a single opaque number after the fact.
• The model itself is something you control. Cost savings are pre-calculated and capability shaped to your domain compounds.

None of that is possible without a control plane. Red Hat AI's gateway will eventually provide a single place to manage model traffic, MCP tool calls, and agent-to-agent traffic, apply role-based access control (RBAC), rate limiting, and guardrails, and emit token-level observability across every destination. Behind it, KServe and llm-d orchestrate vLLM across tiered NVIDIA GPUs, which can provide further performance and economic benefits to this business case.

The bottom line

• Cheaper tokens won't fix your AI bill. Volume is growing faster than price is falling. This is a routing and governance problem, not a procurement problem.
• Open-weight models have closed the capability gap on routine and mid-complexity enterprise work, making tiered routing practical rather than theoretical.
• You can now tune a model with your own data. A smaller, domain-customized model will often beat a general frontier model on your specific enterprise use case.
• Open hybrid cloud is the optimal destination. Model tiering doesn't mean pulling everything back from frontier models; it means establishing 1 set of policies, identities, and cost attribution across on-premise and frontier traffic, so routing becomes an intentional platform decision rather than one each team makes for themselves.

No airline sets its cabin mix by intuition. It measures who flies the route, configures it for them, and reconfigures when the route changes. Your AI traffic is a route you need to study and map.

Every environment produces different numbers. If you'd like to run these inputs against yours, the Red Hat Business Value Practice does this modeling with customers as a structured engagement. Please contact your Red Hat sales team to get connected.

Alternatively, continue your learning journey with our guide to Models-as-a-Service.

Figures are directional, drawn from a reference scenario using published list pricing, and require validation against customer-specific data before use in planning.