OpenVINO 2023.3 Brings Full Support For Intel Emerald Rapids, Broader GenAI & LLMs

Intel engineers on Wednesday released OpenVINO 2023.3 as the latest major update to this leading open-source AI toolkit. The OpenVINO 2023.3 brings "full support" for new Emerald Rapids and Meteor Lake processors, other Intel hardware support improvements, and continuing to expand support around generative AI (GenAI) and large language models (LLMs).

OpenVINO 2023.3 rolls out the OpenVINO Gen AI repository for demonstrating native C/C++ pipeline samples for large language models. OpenVINO has also validated additional models including Mistral, Zephyr, ChatGLM3, and others. Torch.compile is also now fully integrated with OpenVINO.

For expanding the Large Language Model support there is now INT4 weight compression model format support on Intel Xeon CPUs and Intel Core and Intel iGPUs. There is also improved performance for transformer-based LLMs on both CPUs and GPUs, easier optimizations for Hugging Face models, and more.

The OpenVINO 2023.3 release also now has full support for 5th Gen Xeon Scalable "Emerald Rapids" processors, further optimizations for Intel Core Ultra "Meteor Lake" processors, improved performance on ARM platforms, a preview JavaScript API, and other improvements.

Downloads and more details on the big OpenVINO 2023.3 release via GitHub.