TensorFlow Lite Now Supports Tapping OpenCL For Much Faster GPU Inference

[phoronix]

Phoronix: TensorFlow Lite Now Supports Tapping OpenCL For Much Faster GPU Inference

TensorFlow Lite for AI inference on mobile devices now has support for making use of OpenCL on Android devices. In doing so, the TFLite performance presents around a 2x speed-up over the existing OpenGL back-end...

http://www.phoronix.com/scan.php?page=news_item&px=TensorFlow-Lite-OpenCL-Accel

[bachchain]

I'm not sure that I would trust Android OpenCL drivers to not be a broken buggy mess

[cb88]

I'm not sure that I would trust Android OpenCL drivers to not be a broken buggy mess

Tensorflow is probably inherently resistant to that to some degree... the main thing is its likely some implementations accumulate error but as long as your math takes that into consideration you can completely side step it.... its not like this is being used for scientific computation either where that would be impractical.

I for one am glad to see the TensorFlow developers slowly venturing out of CUDA land... perhaps eventually there may even be a full non CUDA version of Tensorflow.

[peterdk]

Wow, maybe we get opencl support for AMD etc in the future? A few years ago I played with tensorflow and it required basically NVidia.

[bridgman]

We have tensorflow support for ROCm upstream as well.

[dxin]

OpenCL isn't part of Android public API so this is mostly limited to a few white listed vendor apps.

[Aeder]

We have tensorflow support for ROCm upstream as well.

Really? TF documentation seems to only mention NVIDIA for their GPU variant.

[bridgman]

Really? TF documentation seems to only mention NVIDIA for their GPU variant.

Yeah, I'm not sure what it takes to get the core documentation updated, will ask. There have been community builds from upstream with ROCm support for something like a year now... I think they started last September.