In-Memory Computing Could Be The Inference Breakthrough AI Needs
FORTRA, Thursday, February 22nd, 2024
Generative AI is unlocking incredible business opportunities for efficiency, but we still face a formidable challenge undermining widespread adoption: the exorbitant cost of running inference.
You've heard of the staggering expenses incurred during the training of Large Language Models (LLMs): the multitude of GPUs, the enormous electricity bills. Analysts estimate Meta may spend $15 billion on GPUs in 2024. Generative AI demands copious memory and bandwidth for weight calculations and data handling, presenting a major obstacle to running models at scale. Even OpenAI's Sam Altman says, 'there's no way to get there without a breakthrough,' he said. 'It motivates us to go invest more in fusion.'