IBM Announces Availability of Open-Source Mistral AI Model on watsonx, Expands Model Choice to Help Enterprises Scale AI with Trust and Flexibility
IBM News, Thursday, February 29th, 2024
IBM offers an optimized version of Mixtral-8x7B that showed potential to cut latency by up to 75%; Adds to growing catalogue of IBM, third-party and open-source models to give clients choice and flexibility
IBM announced the availability of the popular open-source Mixtral-8x7B large language model (LLM), developed by Mistral AI, on its watsonx AI and data platform, as it continues to expand capabilities to help clients innovate with IBM's own foundation models and those from a range of open-source providers.
IBM offers an optimized version of Mixtral-8x7B that, in internal testing, was able to increase throughput - or the amount of data that can be processed in a given time period - by 50 percent when compared to the regular model.1 This could potentially cut latency by 35-75 percent, depending on batch size - speeding time to insights. This is achieved through a process called quantization, which reduces model size and memory requirements for LLMs and, in turn, can speed up processing to help lower costs and energy consumption.