Etnaviv NPU Support Coming Together, Mesa Upstreaming Next

The Etnaviv Gallium3D driver within Mesa has long been focused on reverse engineering and supporting Vivante graphics IP found in various SoCs. That driver has worked out well for open-source OpenGL support for Vivante graphics while more recently Etnaviv has begun tackling Vivante neural processing unit (NPU) support that is beginning to be found in various SoCs.

Recently there's been work with the Etnaviv DRM kernel driver and Etnaviv Mesa code for enabling VeriSilicon Vivante NPU cores. Fortunately, enabling the NPU support hasn't been too daunting with the architecture close to existing Vivante graphics cores.

Tomeu Vizoso has been one of the developers leading the Etnaviv NPU charge and announced today "we got there!" He's now successfully running MobileNetV1 with full-accuracy on the NPU cores with the pending Mesa code. Notably, the performance on the NPU cores is around three times faster than running the inferencing on just the CPU cores. Vizoso commented:

"That takes us to a performance level around 3 times faster than running the same inference on the CPUs on the A311D SoC.
...
The 13 ms. that the convolutions take in the NPU is still sensibly higher than the 8 ms. that the blob achieves, but the optimizations mentioned in previous updates in this blog should bring us pretty close."

With the recent successes, Tomeu Vizoso is now focusing on getting the rest of the Vivante NPU support code for the Etnaviv driver into upstream Mesa. This pending merge request has the code for allowing inferencing on the VeriSilicon NPU cores with Etnaviv Gallium3D. More details on the Etnaviv NPU state via this blog post.