-O3 Compiler Optimization Level Still Deemed Too Unsafe For The Linux Kernel

[zcansi]

Kind of surprised to learn that it's even supported at all to compile different kernel modules with different compiler flags. I wonder what the reason is for supporting that?

[HD7950]

On Archlinux i compile manually some of my most used packages using -O3 and march=native flags, kernel is one of them, and never had a problem.

[CommunityMember]

Kind of surprised to learn that it's even supported at all to compile different kernel modules with different compiler flags. I wonder what the reason is for supporting that?

There will always be certain code constructs for which -O3 makes a significant difference in performance critical code, and for which validation of the results is possible. (De)Compression is the poster child.

[CommunityMember]

LLVM? Even -O3 in its worst days is a lot better and more trustworthy then building the kernel with this unreliable untrustable mess.

What is the technical argument for your claims that LLVM is unreliable and/or untrustable?

[linner]

In all the years I have been writing high-performance code I've not noticed much difference with -O3 anyway. A tiny bit, but nothing worth getting bothered over.

[sdack]

Kind of surprised to learn that it's even supported at all to compile different kernel modules with different compiler flags. I wonder what the reason is for supporting that?

To have it as an option for where it is needed. It makes sense to have compression and encryption functions optimised. A driver and even an entire subsystem, which implements complex algorithms as part of their operation, can sometimes benefit from more aggressive optimisations.

What is trying to be avoided by not using -O3 everywhere is to optimise critical code beyond reason and thereby to introduce undefined and possibly even false behaviour. Some parts of the kernel have to work flawlessly to avoid lockups, hangs, security holes or it could even damage the hardware.

Although the kernel allows modules to have their own flags can one also influence the compiler's behaviour with #pragma's from within the source files, too.

[FireBurn]

I've been using -march=native for a very long time, O3 for a number of years, and now also Clang & LTO without issue

[ms178]

I've been using -march=native for a very long time, O3 for a number of years, and now also Clang & LTO without issue

That, and I even throw Polly into the mix.

On the other hand, I've seen some breakage with certain modules, configs, Full LTO and so forth from time to time. But all could be worked around for my purposes so far.

[perpetually high]

I've been using -march=native for a very long time, O3 for a number of years, and now also Clang & LTO without issue

Same here, kind of wish I didn't see this thread lol

I'm gonna consider dropping down to -O2 for the kernel and others, I'm not nearly smart enough to know the implications of -O3, I just know the kernel has compiled successfully for a couple years now with it and hasn't given me any problems that I can tell. Gonna try the clang LTO and see how that compares as well

[arQon]

Last time this topic came up, I mentioned that in 20+ years of GCC I'd literally *never* had O3 not cause problems in a single real-world project in all that time.

At which point someone threw a temper tantrum and started ranting about how their "high performance, very clever, etc" code was just fine with it and always had been, and how tests would catch all the problems anyway (despite them never noticing such problems in the first place), and so on. I don't remember the name, but it'll be interesting to see if they show up in this thread to argue the same case again.

O3 is "simple": a toy project may well work fine with it, and even a "real" project may APPEAR fine with it - and maybe a miracle happens and the project genuinely IS fine - but like I say, I've never seen one myself. People should understand that O3 is fundamentally as unsafe as race conditions or undefined behavior. But unlike those, it's not about about how "good" the code is, or some magical confirmation bias of your brilliance, or ANYTHING under your control at all: it's Russian Roulette, and just because you've managed to not get shot in the head so far doesn't mean the gun is empty.

As much as anything else though, O3 *certainly* isn't an excuse for writing crappy code in the first place and hoping the compiler will "fix" your sloppiness. It won't, and in most cases O3 will get you very little speedup over O2 anyway. But if you absolutely, desperately, need a tiny bit more performance from a specific piece of code that's genuinely a hotspot, and you can't do a better job than the compiler (which these days is genuinely difficult on x64, given the instruction set sprawl and generational timing differences, etc), then sure: split out that piece of code - but *only* that piece - and build that O3, and see if you get lucky. gl.