CPU spot pricing is gone — treat compute like a scarce resource

The mental model most small teams rely on for cloud compute is now broken. You spin up what you need, scale down what you don't, and use spot instances to cut costs on bursty or experimental workloads. That model assumed a reliable surplus. There isn't one anymore.

Gergely Orosz's piece on the emerging CPU shortage is worth reading if you run anything at scale. The short version: spot CPU pricing has effectively disappeared, because there is no longer idle capacity to sell at a discount. Reservations now need to be placed months ahead. Some cloud regions are turning away new tenants entirely. Even well-funded companies with cash to spend and willingness to sign long leases are being told no.

The driver is AI workloads — not just inference, but reinforcement learning and agentic pipelines. Agents don't just generate text; they compile code, run tests, execute linters, call APIs. All of that is CPU work. The data center ratio of one CPU to eight GPUs is reportedly shrinking toward 1:4, and could approach 1:1. At the same time, CPU supply is squeezed from the manufacturing side: TSMC is prioritising GPU production, and DRAM makers are shifting wafer capacity toward high-bandwidth memory for AI accelerators. Server order lead times have stretched from one or two weeks to around six months, with prices up 10–20%.

What this actually changes for builders

For teams operating at meaningful scale, the implication is clear: capacity planning for general-purpose compute is no longer optional. Per the published summary, reserving specific CPUs now needs to be done months in advance, and cloud providers will turn down reservations if they lack the right capacity. If you have been autoscaling into on-demand CPU capacity and assuming it will always be there, that assumption needs revisiting now.

For smaller teams, the risk is subtler. You may not feel the shortage today, but decisions you make in the next few months — which cloud provider, which region, which instance families — will constrain you later. Locking into a provider relationship early, even at slightly higher cost, is likely better than discovering the market is thin when you need to grow.

The efficiency argument is now mandatory

When capacity was abundant, leaving headroom on underutilised instances was a reasonable trade-off against engineering time. That calculus has shifted. The article makes the point directly: this is a good moment to audit which services are genuinely CPU-intensive and whether they need to be, and to consolidate workloads running on underused nodes. Efficiency work that was previously nice-to-have now frees up reserved capacity that may be difficult to replace.

The trade-off to name clearly

Committing to reserved compute is more expensive upfront and less flexible. If your usage drops, you eat the cost. That is a real downside, and it favours teams with reasonably predictable workloads over those with highly variable demand.

But the alternative — staying fully on-demand and hoping spot markets recover — now carries its own cost: the risk of not being able to scale when you need to, or paying a premium for capacity that is increasingly scarce. Optionality in cloud compute used to be free. It no longer is.

The practical move is to forecast your growth over the next 12 months with as much honesty as you can manage, reserve what you are confident you will need, and treat on-demand as the exception rather than the baseline.

Sources

• The Pulse: a new trend of CPU shortages (original)
• Found via the TLDR newsletter