What Happens When We Can't Just Build Bigger AI Datacenters Anymore?
The Register, Friday, January 24th, 2025
We stitch together enormous supercomputers from other smaller supercomputers of course
Generative AI models have not only exploded in popularity over the past two years, but they've also grown at a precipitous rate, necessitating ever larger quantities of accelerators to keep up.
Short of a breakthrough in machine learning and with power becoming a limiting factor, AI's continued growth may ultimately hinge on a new kind of supercomputer, one that spans entire countries and potentially even continents.
The idea here is rather straightforward. If it's no longer practical to build bigger datacenters, start stitching together the ones you already have.