Compute is the New Bottleneck for Frontier AI

The visible problem and the invisible one

The hardest problem with frontier AI in 2026 isn’t that prices are going up. They are, and considerably so, the harder problem is that the model you license isn’t the same week to week. 

While the model weights are stable, it’s the harness around the model, i.e., default reasoning effort, system prompts, tool-call routing, cache layers, that’s changing. And, this “harness” together with the model is what you’re running in production. In other words, the AI system you license isn’t always the system you think you’re running. 

While increasing costs are what the CFO sees, the engineering org feels the pain differently; regressions on workflows broken today that worked yesterday, deadlines slipping on changes the model card never disclosed, and no clean way to tell whether the bug is yours or the provider’s. For example, Anthropic shipped three harness changes between March and April that materially affected model performance and caused major interruptions to enterprise deployments. They disclosed the changes weeks later in a post-mortem. Anthropic’s transparency is the only reason any of this is in the public record. The emerging standard across the industry is that harness changes ship without notice.

There are two things happening at once here. First, prices are rising, and, secondly, the underlying product is drifting. They’re connected, and the connection is due to limited compute capacity. In this post we explore both issues in turn, and advise enterprise leaders to invest in observability frameworks to keep track of these emerging trends in modern AI deployments.

The price went up while the technology matured. That’s not what usually happens

The textbook trajectory for any compute-driven technology is monotonic: prices fall as the technology gets more efficient, the buildout matures, and the user base grows. Right now, we’re seeing the exact opposite. 

In March 2023, GPT-4 launched at $30/$60 per million input/output tokens. By August 2025, GPT-5 was at $1.25/$10. GPT-4o to GPT-5 was another 50% drop on the way down. Anthropic cut Claude Opus 4.6 pricing by 67% from Opus 4.1 in a few months. Google’s Gemini 1.5 Flash dropped 78% in three months in late 2024.

Then 2026 happened.

Figure 1: Flagship model blended pricing (assuming 75% input and 25% output tokens)` on a log scale. Closed-frontier providers (Claude, ChatGPT, Gemini) all started November 2023 between $7 and $30 per million tokens, fell over two years, and reversed in 2026. The shaded band is 2026.

On April 23, GPT-5.5 launched at $5/$30 per million tokens, doubling GPT-5.4 on both sides. Opus 4.7 kept the same $5/$25 sticker as Opus 4.6, but a new tokenizer produced up to 35% more tokens per prompt, so the effective rate climbed accordingly. Gemini 3.1 Pro raised both input and output rates well above the 1.5 Pro floor. Qwen 3.6 Max Preview launched at $1.30/$7.80, a clean doubling of Qwen3 Max on output, and Alibaba closed their flagship model for the first time. Even DeepSeek, a lab known for their open weights work, closed their frontier V4 Pro at $1.74/$3.48, a 6X jump over the open V3.2.

Figure 2 presents the current cross-provider snapshot, from least to most expensive on the effective per-token rate, assuming a typical workload of 75% input tokens and 25% output tokens (the Artificial Analysis convention). Open-weight models are less expensive, closed-frontier hosted models are the most expensive.

Figure 2: 2026 flagship model pricing snapshot. 

The 10X difference in price from Kimi K2.6 to GPT-5.5 doesn’t mean the model quality differs by 10X. Kimi K2.6 leads SWE-Bench Pro at 58.6%, beating GPT-5.4 at 57.7% and Opus 4.6 at 53.4%. The difference isn’t quality, it’s what you’re paying for and how you’re allowed to deploy it. Closed-frontier providers sell more than inference. The token rate buys access to a moving target: new tool integrations, longer context windows, new model generations, live in production the moment they ship. That’s the moat, but you can’t self-host these models. Open-weight models work the other way. The weights you downloaded in March are the weights you’re running in November, but you choose where to run them: someone else’s cloud, your own GPUs, or anything in between. And nobody hosting open weights can charge a premium on a model you can self-host.

Anthropic said the quiet part out loud

Anthropic is the right place to start because they’re one of the only major frontier providers that documented what they did in a post-mortem (and it isn’t the first time they did so). 

For about six weeks, power users on GitHub, Reddit, and X reported that Claude Code felt noticeably worse. The narrative the developer community converged on was “AI shrinkflation“, i.e., paying the same rate for a quietly downgraded model. The most cited audit came from Stella Laurenzo, Senior Director in AMD’s AI group, who walked through 6,852 Claude Code session files and over 234,000 tool calls and concluded that the tool was “unusable for complex engineering tasks.” Other external audits told a similar story – third-party benchmark BridgeMind reported that Claude Opus 4.6 dropped from 83.3% to 68.3% in accuracy, knocking it from #1 to #10 in their rankings.

In the April 23 report Anthropic identified three “harness” changes shipped between March 4 and April 16:

1. On March 4, the default reasoning effort on Claude Code dropped from high to medium, framed publicly as “addressing UI latency issues.”
2. On April 16, a system prompt instruction was added telling the model to keep text between tool calls under 25 words and final responses under 100 words.
3. A cache-optimization bug in v2.1.116 introduced silent quality regressions.

The report framed all three as bugs or product missteps. When read as a set (a default effort downgrade, a verbosity throttle, and a cache fix), every one of them conserves compute. They were individually reasonable product decisions. Collectively, they amount to capacity management. SemiAnalysis put the same point bluntly: “When the harness is part of the product, the model gets blamed.”  

Models have public reputations; harnesses don’t. A price increase shows up on every invoice and triggers procurement reviews. While a change to the default reasoning level is invisible to your finance team, the impacts on your engineering org can be substantial and may amount to material impacts downstream. And when the output gets worse, the model gets blamed before the harness does.

Anthropic’s capacity challenges are on the public record. CEO Dario Amodei has repeatedly described Anthropic as compute-constrained. They introduced stricter peak-hour usage limits in March, which, by their own account, affected ~7% of users on Pro tiers. News reporting indicated OpenAI’s revenue chief, in a leaked internal memo, called Anthropic’s compute strategy “a strategic misstep” and said Anthropic is “operating on a meaningfully smaller curve” than competitors. And in mid-April, Anthropic moved its largest enterprise customers to a fully usage-based token pricing model, replacing fixed contracts. That last move is the most illuminating of the bunch. Mature accounts don’t get shifted from fixed pricing to metered pricing unless unit costs are rising and capacity is finite.

To Anthropic’s credit, they explained the post-mortem in detail and reverted the harness changes. Most providers do the same things more opaquely.

OpenAI faced similar challenges, but responded by increasing prices

OpenAI’s GPT-5.5 release on April 23 (the same week as the Anthropic post-mortem) doubled per-token pricing relative to GPT-5.4 and held GPT-5.5 Pro at $30/$180. OpenAI’s public framing was that GPT-5.5 uses ~40% fewer output tokens to complete equivalent Codex tasks, so the workload-effective cost increase is closer to +20% than +100%. The capacity-conservation logic at OpenAI and at Anthropic is identical. The difference is that OpenAI prices it explicitly, and Anthropic shipped it as a quieter harness change.

OpenAI now sells the same model at four different price points (Priority at 2.5x, Standard, Flex at 0.5x, Batch at 0.5x), letting customers self-select on speed-vs-cost trade-offs. That’s price segmentation as a way to manage capacity. Same model, four different rates.

Look at the magnitude of the increases. Pricing went up across the board in 2026, on both input and output, in nearly every flagship model:

 Figure 3: Per-generation input and output price increases on the 2026 flagship model transitions. Every provider raised both sides.

In four of five transitions, output prices rose by 100% or more, and in three of five (Gemini, Qwen, DeepSeek) output rose strictly faster than input. OpenAI raised both sides aggressively, landing GPT-5.5 at the highest price. Anthropic didn’t move the sticker at all. Opus 4.7 launched at the same price as Opus 4.6, with the ~25% effective increase coming from a new tokenizer that produces more tokens per prompt. Same compute capacity constraint. OpenAI put it on the invoice, Anthropic put it in the tokenizer and harness.

Prices increased across the board, but changes were more pronounced with output tokens. Output has always cost more because decode (generating one token at a time) is the slow side of the workload, while prefill (reading the prompt) runs in parallel. More GPU time per token means a higher rate per token. For three years, providers cut prices anyway as hardware improved, batching matured, and KV cache footprints shrank. The 2026 transitions reversed that, with output rates moving 2X to 7X against more modest input increases. The GPU squeeze hits both phases, just harder on the one that takes longer.

Cost per task, not cost per token

OpenAI’s defense of the GPT-5.5 price increase points at the right metric. The number that matters is cost per task completed, not the pricing rate per million tokens. SemiAnalysis made the same point in their April 24 breakdown and put it in the simplest possible form: 

“Mythos may be 5x more expensive than Opus on a per token basis, but much of that price increase is nullified because Mythos can solve the same problem using fewer tokens.” 

That’s the right way to frame the question. Cost per task, and benchmarks like SWE-Bench, capture the real economic value of a more efficient model. A flagship model that finishes the same job in 30,000 tokens instead of 60,000 absorbs a doubled rate card and may leave your bill flat. From the seller’s side, that’s a defensible price increase. But the metric also reveals what the rate card alone hides. For three years, providers delivered both halves of the trade at once: more capable models and lower per-token rates. GPT-4 to GPT-4o was 75% cheaper on output. Gemini 1.5 Flash to 2.0 Flash was 71% cheaper. 

We all got a more capable model in 2026. We just didn’t get the rate cut that came with every previous generation. Cost per task held flat at best. Model providers don’t pass efficiency gains through to price when they need those gains somewhere else, and right now they need them on the GPU. Compute is the bottleneck. The labs can’t build out fast enough to serve the demand, so every token the new model saves goes to another user instead of cutting your bill.

If you’re optimizing your AI fleet on per-token rate pricing, you’re optimizing the wrong number. The thing you actually pay for is cost per task, and the providers know it. Designing your fleet around the token pricing alone is like comparing contractors on hourly rate without asking how many hours the job will take.

Capital is fast. Concrete is slow

Capital raised today turns into served tokens in late 2026 or 2027. Power substations have 12–18 month wait times. HBM supply is constrained. Meanwhile, the shape of demand has gotten worse for capacity. For example, an agentic Claude Code session that fires 1,000 tool calls consumes hundreds of thousands of tokens for a single user task. A year ago, that was a chat. The buildout takes years, but the demand it’s chasing is doubling every few months.

What this means for an enterprise AI fleet

Rising prices you can see. Drifting capability you can’t. Here are three things to follow for anyone running production AI workloads at scale.

First, anything that multiplies effective compute on existing hardware matters more than it did 12 months ago. KV cache reuse is the cleanest version. The KV cache is the working memory a transformer builds while reading a prompt: expensive to compute the first time, cheap to load on a second pass. When capacity is the constraint, persistent KV caching stops being a latency optimization and becomes a capacity multiplier. Our benchmarks at Everpure on KVA show up to 20X time-to-first-token improvements on long-context workloads, which is another way of saying you can fit more agent runs on the same GPU fleet.

Second, the on-prem economics now look different. Enterprises stayed on hosted APIs through 2024 and 2025 because hosted prices fell faster than self-hosting could amortize. That curve broke in Q1 2026. Capacity rationing at the hosted providers is an underrated tailwind for every enterprise that’s been quietly building toward on-prem.

Third, observability stops being nice to have and becomes how you defend your unit economics. The users who caught the Anthropic regression first were the ones running their own evals against production traffic. Without that, you’re flying blind in a market where providers have every reason to quietly tune the model behind your workloads.

The hosted-vs.-on-prem question used to be a sovereignty question. In 2026, it’s a capacity question. The labs that can solve their compute buildout in the next 24 months will reclaim some of the price floor. Until then, the enterprises that win are the ones whose unit economics don’t depend on someone else’s GPU.