Intel Speed Select Driver Issue Was Hurting Performance In Some HPC Benchmarks

Intel's Speed Select Technology introduced since Cascade Lake for providing more granular power/performance controls was done in the name of performance but it turns out an ISST Linux driver inefficiency could lead to a 10%+ performance hit for some HPC benchmarks.

Public details are scarce on this latest Intel Speed Select Technology Linux driver change but when making use of this ISST code on select systems and for unspecified HPC workloads it could lead to reported 10%+ performance penalties for some high performance computing benchmarks. The issue stems from the CPU to PCI device mapping carrying out a linear search of PCI devices on systems and in particular for massive servers this could prove to be very expensive.

Fortunately, a patch is pending to optimize the CPU to PCI device mapping for the ISST kernel code. The patch noted, "It was observed that some of the high performance benchmarks are spending more time in kernel depending on which CPU package they are executing. The difference is significant and benchmark scores varies more than 10%. These benchmarks adjust class of service to improve thread performance which run in parallel. This class of service change causes access to MMIO region of Intel Speed Select PCI devices depending on the CPU package they are executing."

That slow function dramatically hurt some benchmarks when running on large servers with many PCI devices to search through in a linear manner. The optimization catches the device and function to relieve that overhead and in turn "improves performance of these benchmarks significantly."

It's a narrow scope of impact but if running HPC workloads and making use of Speed Select, it may be of interest. It also is just another recent example of the increasing complexity of CPU power management features/controls having the potential for significant unintended consequences.