Trying To Make Ubuntu 18.10 Run As Fast As Intel's Clear Linux

With the recent six-way Linux OS tests on the Core i9 9900K there was once again a number of users questioning the optimizations by Clear Linux out of Intel's Open-Source Technology Center and remarks whether changing the compiler flags, CPU frequency scaling governor, or other settings would allow other distributions to trivially replicate its performance. Here's a look at some tweaked Ubuntu 18.10 Cosmic Cuttlefish benchmarks against the latest Intel Clear Linux rolling-release from this i9-9900K 8-core / 16-thread desktop system.

In the forum comments and feedback elsewhere to that previous Linux distribution comparison, there were random comments questioning:

- Whether Clear Linux's usage of the P-State "performance" governor by default explains its performance advantage with most other distributions opting for the "powersave" governor.

- Clear Linux is faster because its built with the Intel Compiler (ICC). This is not the case at all with Clear being built by GCC and LLVM/Clang, but seems to be a common misconception. So just tossing that out there...

- Clear Linux is faster because of its aggressive default CFLAGS/CXXFLAGS/FFLAGS. This does certainly help in some built-from-source benchmarks, but that's not all.

About a year ago I tried similar tests to today of tweaking Ubuntu 17.10 to try to run like Clear Linux while in this article is a fresh look. The OS configurations tested were:

Clear Linux - Clear Linux running on the i9-9900K with its Linux 4.18.16 kernel, Mesa 18.3-devel, GCC 8.2.1, EXT4 file-system, and other default components.

Ubuntu - The default Ubuntu 18.10 installation on the same system with its Linux 4.18 kernel, Mesa 18.2.2, GCC 8.2.0, EXT4, and other stock components/settings.

Ubuntu + Perf Gov - The same Ubuntu 18.10 stack but when switching over to the P-State performance governor rather than the default P-State powersave mode.

Ubuntu + Perf + Flags - The P-State performance mode from above on Ubuntu 18.10 but also setting the same CFLAGS/CXXFLAGS/FFLAGS as used by Clear Linux before re-building all of the source-based benchmarks to compare the performance impact of the default tuning parameters.

Ubuntu + Perf + Flags + Kernel - The Ubuntu 18.10 tweaked state of above with the P-State performance governor and tuned compiler flags while also building a Linux 4.18.16 kernel from source with the relevant patches and Kconfig configuration as shipped by Clear Linux. Their kernel configuration and carried patches can be found via clearlinux-pkgs/linux on GitHub.

Ubuntu + Clear Docker - Ubuntu 18.10 with the P-State performance governor, running on the Clear Linux optimized kernel, and using Docker CE to run the latest Clear Linux Docker image for having all of the Clear user-space components running within this container.

The same system was used for all of the testing and was the Intel Core i9 9900K at stock speeds, ASUS PRIME Z390-A motherboard, 16GB DDR4-3200 memory, Samsung 970 EVO 250GB NVMe SSD, and Radeon RX Vega 64 8GB graphics card.

I ran 92 different tests on Ubuntu 18.10 and Clear Linux for a wide look at the performance between these distributions ranging from scripting language benchmarks like PHP and Python to various scientific workloads, code compilation, and other tests. With the 92 test runs, here are the key findings from this large round of testing of Clear Linux compared to Ubuntu 18.10 in five different tuned states:

- When comparing the out-of-the-box Clear Linux to Ubuntu 18.10, the Intel distribution was the fastest in 66 of the benchmarks (72%) with Ubuntu only taking the lead in 26 of these different benchmarks.

- Switching to the P-State "performance" governor on Ubuntu 18.10 only allowed it to win over Clear Linux in an extra 5 benchmarks... Clear Linux still came out ahead 66% of the time against Ubuntu either out-of-the-box or with the performance governor.

- In the third state of Ubuntu 18.10 with using the P-State performance governor and copying Clear's compiler flags allowed Ubuntu 18.10 to enhance its performance relative to the default Ubuntu configuration, but Clear Linux was still leading ~66% of the time.

- When pulling in the Clear Linux kernel modifications to Ubuntu 18.10 and keeping the optimized compiler flags and performance governor, Ubuntu 18.10 just picked up one more win while Clear Linux was still running the fastest in 59 of the 92 benchmarks.

- Lastly, when running the Clear Linux Docker container on Ubuntu 18.10 while keeping the tweaked kernel and P-State performance governor, Clear Linux won now in "just" 54 of the 92 benchmarks, or about 59% of the time it was the fastest distribution.

Going to these varying efforts to tweak Ubuntu for faster performance resulted in Clear Linux's lead shrinking from 72% to 58%... Or about 64% if not counting the run of using the Clear Linux Docker container itself on Ubuntu 18.10 for the optimized Clear user-space.

This data shows that Clear Linux still does much more than adjusting a few tunables to get to the leading performance that it's not as trivial as adjusting CFLAGS/CXXFLAGS, opting for the performance governor, etc. Clear additionally makes use of GCC Function Multi-Versioning (FMV) for optimizing its binaries to use the fastest code path depending upon the CPU detected at run-time among other compiler/tooling optimizations. It also often patches its Glibc and other key components beyond just Linux kernel patches not yet ready to be mainlined. Other misconceptions to clear up about this open-source operating system is that it does not use the Intel ICC compiler, it does run on AMD hardware (and does so in a speedy manner as well), and runs on Intel hardware going back to around Sandy Bridge, just not the very latest and greatest generations.

While the prominent performance numbers are already shared, on the following pages are a look at some of the interesting benchmark results from this comparison.