The AI compute gap: Enterprises are buying infrastructure faster than they can measure what it costs

What happened

Across 107 enterprises, AI infrastructure spending is accelerating well ahead of the ability to see or steer its economics. Buying decisions turn on integration and total cost of ownership rather than headline token price — which is fortunate, This wave of VentureBeat Pulse Research examines enterprise AI infrastructure and compute: where organizations are in th

Integration with the existing stack (41%) and total cost of ownership (35%) dominate, while the headline metric — cost per million tokens — is the deciding factor for just 8%, dead last. The pattern is coherent: buyers are optimizing for how a provider fits and what it truly costs to operate, not for the advertised unit rate.

It also foreshadows Finding 7 — enterprises say TCO matters most, yet most cannot yet measure it rigorously. The stated priority and the measured capability are out of step. 83% report GPU utilization of 50% or less We asked what share of their GPU capacity enterprises actually utilize. The answer is a well-known but rarely quantified inefficiency.

run at 26–50% utilization 34% run at 10–25% utilization 15% run under 10% utilization 12% run over 50% — the efficient minority don’t measure utilization at all; a further 7% consume via API and run no GPUs of their own Disclosure: Band percentages count every selection against all 107 qualified respondents; 14 respondents selected more than one band, so bands overlap.

Why it matters

What happened Across 107 enterprises, AI infrastructure spending is accelerating well ahead of the ability to see or steer its economics. Most organizations run their AI on a familiar base of hyperscalers and model-provider APIs, yet the next dollar is aimed at specialized compute almost none of them use today; a majority intend to switch or add providers within the year, many within a quarter.

7% plan to change within 6–12 months For a category as foundational as compute, this is a remarkable amount of intended movement. Only 36% have no plans to change, meaning a clear majority (64%) intend to switch or add a provider within twelve months — and 38% within the next quarter alone.

Where that interest points is telling: the providers drawing the most switching consideration are again the incumbents — Microsoft Azure and Google Cloud (33% each), OpenAI (30%), and Gemini (22%) — which suggests much of the near-term movement is reshuffling among the majors and consolidating spend rather than defecting to new entrants.

The neocloud interest in Finding 3 is a 12-month evaluation thesis; the switching in the next quarter is mostly incumbents trading share. ) Integration and total cost of ownership decide — not sticker price We asked what matters most when enterprises select an AI infrastructure provider. Headline price finished last.

integration with the existing cloud and data stack — the top factor 35% total cost of ownership (TCO) 24% performance — latency and throughput each cite security/compliance, autoscaling for spiky workloads, and GPU access/availability 8% cost per 1M tokens — the least-cited factor Enterprises do not buy AI infrastructure on pricing, which is the place vendors compete on hardest.

What to watch

At the respondent level, 83 of the 100 GPU-operating enterprises reported utilization at or below 50% The compute already in place runs cold. Adding the bands at or below half capacity, 83% of enterprises that operate GPUs report utilization of 50% or less, and nearly half (49%) run at 25% or below. Only 12% clear the 50% mark, and a further 8% do not measure utilization at all.

Idle accelerators are expensive accelerators, and this is the clearest single measure of the compute gap: enterprises are planning to buy more