What Would You Do With A 16.8 Million Core Graph Processing Beast?
The Next Platform, Friday, September 1,2023
If you look back at it now, especially with the advent of massively parallel computing on GPUs, maybe the techies at Tera Computing and then Cray had the right idea with their 'ThreadStorm' massively threaded processors and high bandwidth interconnects.
Given that many of the neural networks that are created by AI frameworks are themselves graphs - the kinds with vertices with data and edges showing the relationships between the data, not something generated in Excel - or output what amounts to a graph, maybe, in the end, what we need is a really good graph processor. Or, maybe millions of them.
Gasp! Who speaks such heresy in a world where the Nvidia GPU and its wannabes are the universal unguent to solve - salve, surely? - our modern computing problems? Well, we do. While GPUs excel at dense matrix high precision floating point math that dominates HPC simulation and modeling, a lot of the data that underpins AI frameworks is sparse and lower precision to boot. And given this, maybe there are better ways to do this.