OpenVINO 2023.2 Released With More Generative AI Coverage & Broader LLM Support

Intel has released OpenVINO 2023.2 as the newest version of their open-source toolkit for optimizing and deploying AI inference. OpenVINO continues in its never-ending quest for maximizing deep learning performance and efficiency while continuing to expand in its model support and features.

With the OpenVINO 2023.2 release they have continued expanding in their generative AI coverage and integration. The new release can handle models used for chatbots, instruction following, code generation, and more. Among the models now supported are LLaVA, chatGLM, Bark, and LCM. There is also improved support for PyTorch models as well as Hugging Face models.

OpenVINO 2023.2 also has accelerated inference for large language models (LLMs) with Int8 model weight compression, expanded model support for dynamic shapes for better Intel GPU performance, preview support for the Int4 model format on Intel CPUs and GPUs, and other LLM support advancements.

The OpenVINO 2023.2 release also is now available via the Conan package manager and has improved performance for running OpenVINO on ARM processors via enabling FP16 model format support.

Downloads and more details on the OpenVINO 2023.2 release this morning via GitHub.