Benchmarking The Linux Kernel With An "-O3" Optimized Build

Stemming from last weeks Linux kernel patches suggesting an -O3 experimental option for all CPU architectures and Linus Torvalds rather quickly shooting it down, here are some fresh benchmarks looking at the Linux kernel performance when the kernel image is rebuilt with the -O3 optimization level rather than -O2.

As during past attempts when -O3 optimizing the Linux kernel has been brought up, Linus Torvalds has expressed concern due to the GCC compiler at least historically generating worse code than -O2 and sometimes has even generated incorrect / faulty code especially when dealing with the low-level kernel behavior. Particularly with the huge beast that is the Linux kernel, it can be difficult to debug the kernel when its induced by the compiler generating subtly bad code and may not be immediately noticeable by the user/developer.

GCC documentation on the -O3 optimization level.

In the name of safety and reminiscing from the times when the GCC compiler wasn't always reliable with its optimizations, Linus Torvalds is obviously cautious still around aggressive compiler optimizations for the Linux kernel. Additionally, last week's patches hadn't provided any quantitative reasoning for the proposed experimental "CC_OPTIMIZE_FOR_PERFORMANCE_O3" in broadening that outside of the existing ARC architecture tunable.

The -O3 optimization level can be easily set as well via the "KCFLAGS" environment variable for setting the kernel CFLAGS. With that and since I haven't dabbled with compiler optimization level comparisons for the Linux kernel build itself in years, I ran some fresh benchmarks of a -O3'ed kernel build.

For this article I ran benchmarks of the same Linux 5.19 Git state when building the kernel with KCFLAGS=-O2 (default) set and then again rebuilding the same kernel state and same Kconfig with the KCFLAGS=-O3 set for the more aggressive compiler optimization level. The CFLAGS/CXXFLAGS were not changed for the open-source benchmarks used -- the user-space binaries were 1:1 the same with only the Linux kernel build being changed out during the benchmarking process.

All of this testing was carried out on a mid-range Intel Core i5 12600K "Alder Lake" desktop system. A wide range of synthetic and real-world benchmarks were carried out for looking at the impact of an -O3 kernel build over -O2. So let's see what a -O3 kernel means in 2022 when using Ubuntu 22.04 LTS with the modern (but not latest) GCC 11 compiler version.