Debian vs. Ubuntu vs. CentOS vs. openSUSE vs. Clear Linux Post-Meltdown Performance

[Michael]

phoronix ... I would like to see a benchmarking of *lowlatency* & generic kernel now in the era of post-meltdown/spectre. OutOfOrder-Pipelining... is now limited(?), does lowlatency-kernel may have an positive impact? Just wondering...

As you're a new premium member... Thanks for joining and I will work on some tests this weekend!

[Uqbar]

No Arch? No party!
How can you leave Archlinux behind in a head to head contest?
Come on!

[Vistaus]

I think a more likely reason is portability, you don't want to cause performance regressions on a binary distro. On a source distro like Gentoo people are putting a lot of effort into optimizing compiler flags because they're just local anyway.

Solus is using a lof ot Clear Linux's flags, yet portability doesn't seem like an issue there. And Solus is binary and quite popular now.

[geearf]

It seems no one is taking steps to optimize the flags in the popular distros. Maybe it's too much work for them?

I don't know if it's only flags. FDO is probably doing some good too.
But yeah, I definitely wished they would look into it, or follow suit like Solus did.

[linuxgeex]

I think one of the parameters that might affect system performance with the Meltdown patches, due to context switching, is 100hz/250hz/1000hz and pre-emption vs server (no preemption). The more context switches there are, the bigger the drop (especially in older hardware).

Perphaps a test of an identical kernel with preemption on / off, or different timer frequencies could reveal more about it, which would then lead distros to fine tune their distros according to the new data. Of course all these tests require time :P

I think you mean CONFIG_NOHZ, not "pre-emption off".

CONFIG_NOHZ doesn't disable pre-emption. It switched from a scheduled rate of pre-emption to opportunistic pre-emption, where the kernel basically only pre-empts when an interrupt was already occurring rather than introducing an extra 100 interrupts (or more) regular per second.

CONFIG_NOHZ is good for power management, good for cache locality, good for realtime processes, and bad for desktop UI latency. Lots of people swear by CONFIG_HZ 1000 because it makes the desktop buttery smooth, but actual system performance is lower, mostly because it impacts cache locality.

No it won't be affected much by the KPTI/Retpoline patches. Why? A process can make hundreds of context switches inside their timeslice, with hundreds of times more impact than from the regular scheduling. Whether the kernel wakes up 400 times per second with CONFIG_NOHZ or 500 times with CONFIG_HZ=100, makes no measurable difference vs the 500,000 syscalls dbench can make in one second.

[linuxgeex]

It seems no one is taking steps to optimize the flags in the popular distros. Maybe it's too much work for them?

I've had some pretty lengthy discussions with various people about this. Basically it comes down to 3 things:

1) the CFLAGS and makefile produced by the application vendor. These are rarely changed by package managers.

2) Distro CFLAGS policies. Most of the distros have a -O2 policy and rules for when to use other flags, boiling down to something along the lines of "Don't use -O0 or -O1 unless it won't build and operate normally with -O2. By all means enable other optimisations if they are stable and have significant benefits"

which leads to...

3) The package maintainers are prepared to invest so much time for quality assurance and bug fixes. For distros that build on multiple architectures, testing with FDO/PGO or even with -O3 and sets of other flags ie -fno-strict-aliasing, -ffast-math, -fgcse, can easily result in 20-100 times as much build time. The maintainer may not have access to sufficient hardware to perform all that building and testing, they may not have the time personally, they may not be mentoring a new maintainer that they can delegate that work to, and so they may choose to just trust the vendor who says it's stable on XYZ with -O2. Their job isn't to squeeze every ounce of performance out of their packages. It's to ensure there's no release-critical bugs, in a timely fashion.

So, if you are sad about that situation, my all means, find a maintainer for packages you care about, volunteer to help them with testing optimisations. I'm sure your work will be well appreciated by the community.

[_Alex_]

I think you mean CONFIG_NOHZ, not "pre-emption off".

Both... for high throughput I generally disable preemption and decrease hz... the rationale is that the less disruptions there are to the workload, the better it is. Besides, I think desktop responsiveness is not a real issue with modern multi-core processors, but it was definitely an issue in the single-core era when preemption and high hz frequencies were needed for responsiveness. There are loads though that tend to do better with higher hz frequencies...

[Michael]

phoronix ... I would like to see a benchmarking of *lowlatency* & generic kernel now in the era of post-meltdown/spectre. OutOfOrder-Pipelining... is now limited(?), does lowlatency-kernel may have an positive impact? Just wondering...

https://www.phoronix.com/vr.php?view=25860

[drSeehas]

Why is SUSE missing in FS-Mark?