AI Inference Costs Set To Plunge: Gartner
CIODIVE, Wednesday, March 25th, 2026
But CIOs likely won't see any savings as model sizes go up and functionality becomes more advanced, the analyst firm said.
Performing inference on an AI model with 1 trillion parameters will cost large language model providers more than 90% less in 2030 compared with last year, according to analyst firm Gartner. Over the next four years, LLMs will become up to 100 times more cost efficient than some of the first models from 2022. Improved hardware and model design, as well as inference on edge devices and inference-specialized chips will drive the reduced costs.