TensorFlow Lite Now Supports Tapping OpenCL For Much Faster GPU Inference

Written by Michael Larabel in Programming on 18 August 2020 at 11:21 AM EDT. 7 Comments
TensorFlow Lite for AI inference on mobile devices now has support for making use of OpenCL on Android devices. In doing so, the TFLite performance presents around a 2x speed-up over the existing OpenGL back-end.

To little surprise, the TensorFlow developers are finding their new OpenCL back-end for TFLite to be much faster than their OpenGL back-end for mobile inference. Thanks to better performance profiling abilities, native FP16 support, constant memory, and OpenCL being better designed for compute than OpenGL ES with compute shaders, the TFLite performance is much improved -- and especially so compared to doing inference on the mobile SoC CPU cores.

More insight on the new OpenCL back-end for TensorFlow Lite via the TensorFlow.org blog. "Our new OpenCL backend is roughly twice as fast as the OpenGL backend, but does particularly better on Adreno devices (annotated with SD), as we have tuned the workgroup sizes with Adreno's performance profilers mentioned earlier."
Related News
About The Author
Michael Larabel

Michael Larabel is the principal author of Phoronix.com and founded the site in 2004 with a focus on enriching the Linux hardware experience. Michael has written more than 20,000 articles covering the state of Linux hardware support, Linux performance, graphics drivers, and other topics. Michael is also the lead developer of the Phoronix Test Suite, Phoromatic, and OpenBenchmarking.org automated benchmarking software. He can be followed via Twitter, LinkedIn, or contacted via MichaelLarabel.com.

Popular News This Week