Intel Linux Kernel Optimizations Show Huge Benefit For High Core Count Servers

First up was DragonFlyDB that describes itself as a "modern Redis replacement" and aims to be a very speedy memory store compliant with the Redis and Memcached protocols. When running the same Dragonflydb server and software in both environments, there quite clearly shows a difference in performance at 30+ threads. Clear Linux with Intel's latest in-progress kernel optimizations show the performance scaling quite well until 60 threads and then slightly lower when the dual socket configuration comes into play and the memory distribution between the two Xeon Platinum 8490H processors. Ubuntu 23.04 meanwhile shows a huge drop in performance at 120 threads and only slightly recovers in the standard 240 thread configuration... Clear Linux at 120/240 threads shows Dragonflydb still performing much better than at 4~30 threads while under Ubuntu (and other distributions) is that huge performance regression compared to its 60 thread peak.

Running Memcached on this Xeon Scalable Sapphire Rapids server showed nice scaling with Clear Linux's enhancements compared to the default kernel of Ubuntu 23.04. Hopefully these Intel kernel optimizations focusing on scalability will all manage to be mainlined soon.

Redis on this Sapphire Rapids server was showing much better scaling particularly with the GET operations compared to Ubuntu 23.04.