Why TensorFlow Lite Has Been Running Slower On Recent Linux Kernels

The Linux 5.0 to 5.9 kernel benchmarking posted this week showed TensorFlow Lite running slower since the Linux 5.5 kernel... On top of looking at the new Linux 5.9 regressions, I also spent some time bisecting and figuring out what happened for TensorFlow Lite last year that has at least for the system under test caused it to run slower for all the kernel releases this year as shown in the aforelinked article.

So with the speedy AMD EPYC box and knowing the regression happened between Linux 5.4 and 5.5, it was on to bisecting the TensorFlow Lite performance drop. TensorFlow Lite is the open-source deep learning framework for on-device inference. TensorFlow Lite is an optimized version of TensorFlow catering to mobile and edge devices. This testing is focused on TensorFlow Lite with not yet evaluating the full TensorFlow performance or other deep learning workloads across the tested kernel versions with regards to this regression.

With there being a clear difference across the 5.5 threshold, the TensorFlow Lite test profile with the Phoronix Test Suite made for straightforward bisecting of the Linux Git state during the 5.5 merge window and set off to bisecting with the sixty-four core AMD EPYC processor.

The bisecting revealed...