NewsMore in News→
Why Memory Is Becoming the Real Bottleneck in AI Infrastructure
Key Takeaways
- High Bandwidth Memory (HBM) can represent 30-40% of an AI accelerator's cost and is growing as a share of infrastructure spending
- Only three companies — SK hynix, Samsung, and Micron — manufacture HBM, creating a supply oligopoly with rising prices
- Inference demand, not training, is the primary driver of memory consumption as AI applications scale to millions of users
- New capacity takes years to build, meaning memory constraints will persist even as GPU availability improves
- Techniques like quantization and mixture-of-experts architectures are direct responses to memory cost pressures
DE
DT Editorial Team··6 min read·via techcrunch.com