OpenAI Chief Says Age Of Giant AI Models Is Ending; A GPU Crisis Could Be One Reason Why
VentureBeat, April 17th, 2023
The era of ever-larger artificial intelligence models is coming to an end, according to OpenAI CEO Sam Altman, as cost constraints and diminishing returns curb the relentless scaling that has defined progress in the field.
Speaking at an MIT event last week, Altman suggested that further progress would not come from 'giant, giant models.' According to a recent Wired report, he said, 'I think we're at the end of the era where it's going to be these, like, giant, giant models. We'll make them better in other ways.'
Though Mr. Altman did not cite it directly, one major driver of the pivot from 'scaling is all you need' is the exorbitant and unsustainable expense of training and running the powerful graphics processes needed for large language models (LLMs). ChatGPT, for instance, reportedly required more than 10,000 GPUs to train, and demands even more resources to continually operate.