Intel Releases OpenVINO 2023 - Load TF Models Directly, Hybrid CPU Thread Scheduling

Written by Michael Larabel in Intel on 1 June 2023 at 06:00 AM EDT. Add A Comment
INTEL
Intel has released a major update to its wonderful, open-source OpenVINO toolkit for optimizing and deploying AI inference. OpenVINO continues working out great for optimizing and running AI models on a variety of hardware and continues to introduce new features.

The OpenVINO 2023.0 release now allows for TensorFlow and TensorFlow Lite models to be loaded directly into the OpenVINO Runtime and OpenVINO Model Server. Loading the TF/TF-Lite models directly will cause the models to be converted automatically but the developers still recommend converting to OpenVINO IR in advance. There is also experimental support for loading PyTorch models directly too without first having to convert to ONNX.


OpenVINO 2023.0 is also significant in that it now supports the latest Python 3.11 series, ARM processor support in the OpenVINO CPU plug-in including official support for the Raspberry Pi 4 and Apple M1/M2, and broader model support and various new optimizations.

The OpenVINO 2023.0 CPU plug-in also adds thread scheduling support for Intel hybrid CPUs of Alder Lake 12th Gen Core and newer. With this thread scheduling you can choose to run inference on just E or P cores, or both a combination of the energy efficient and high performance cores. OpenVINO 2023.0 also has a new default inference precision where OpenVINO will default to the format that enables optimal performance, such as BF16 on the latest Intel Xeon Scalable CPus or FP16 when dealing with GPUs.

Downloads and more details on the OpenVINO 2023.0 release via GitHub.
Related News
About The Author
Michael Larabel

Michael Larabel is the principal author of Phoronix.com and founded the site in 2004 with a focus on enriching the Linux hardware experience. Michael has written more than 20,000 articles covering the state of Linux hardware support, Linux performance, graphics drivers, and other topics. Michael is also the lead developer of the Phoronix Test Suite, Phoromatic, and OpenBenchmarking.org automated benchmarking software. He can be followed via Twitter, LinkedIn, or contacted via MichaelLarabel.com.

Popular News This Week