A Few Enterprise Takeaways From The AI Hardware And Edge AI Summit 2024
Data Science Central, Tuesday, October 1st, 2024
Enterprises haven't seemed as enthusiastic about generative AI and large language models (LLMs) lately as they have been in previous years. The Kisaco Research event I attended in September provided some reasons why.
Current gen AI processing far too centralized to be efficient or cost effective
If there's a single takeaway that I could point to, it's how overloaded data centers are and how limited edge infrastructure has been when it comes to effectively reducing that data center load for gen AI applications. Ankur Gupta, Senior Vice President and General Manager at Siemens Electronic Data Automation noted during his talk that 'the opportunity for low power needs to be met at the edge.'
Gen AI-oriented data centers must handle an inordinate amount of heat per GPU. Gupta asserted that half a liter of water evaporates with every ChatGPT prompt.