Docker, Inc. Makes Invoking LlMs Simpler For Application Developers
Cloud Native Now, Friday, April 11th, 2025
Docker Inc. this week added an ability to make it possible for application developers using its tools to build cloud-native applications to run large language models (LLMs) on their local machines.
Available in beta, Docker Model Runner achieves that by embedding an AI inference engine into Docker Desktop using an open source llama.cpp library to make the LLM available locally using the OpenAI application programming interface (API). As a result, no extra tools or setup is required.
In the future, Docker also plans to add Docker Model Runner to the Windows instance of Docker Desktop.