Google Cloud Run Embraces NVIDIA GPUs For Serverless AI Inference
VentureBeat, Wednesday, August 21st, 2024
There are several different costs associated with running AI, one of the most fundamental is providing the GPU power needed for inference.
To date, organizations that need to provide AI inference have had to run long-running cloud instances or provision hardware on-premises. Today, Google Cloud is previewing a new approach, and it could reshape the landscape of AI application deployment. The Google Cloud Run serverless offering now integrates Nvidia L4 GPUs, enabling organizations to run serverless inference.
The promise of serverless is that a service only runs when needed and users only pay for what is used. That's in contrast to a typical cloud instance which will run for a set amount of time as a persistent service and is always available. With a serverless se