"CC_OPTIMIZE_FOR_PERFORMANCE_O3" Performance Tunable Dropped In Linux 6.0

[marios]

just to point out some stuff, because I don't like where this discussion is going:
0. I strongly recommend to everyone the use of -O3 everywhere and submit bug reports if things break. -O3 is supposed to work, if it does not, there is a bug somewhere. As a side note I think that testing should be done with many compilation options including -O0, -O3, -flto and potentially -Ofast (albeit it is OK if it breaks).
1. I think that -O3 should be at the kernel config, just like -O2 and -Os. I would also add O0 and the option to use arbitrary flags. You can keep -O2 as the default (although I prefer -O3) but I find not having the option beyond stupid. Before telling "just use KCFLAGS", tell me why is -Os different and it can be configured using the kconfig.
2. I have no problem with people that compile their kernel using whatever options they want. I have lots of problems with people that try to persuade everyone that their options are the correct ones with 0 (ZERO) arguments.

[dekernel]

BTW, the whole tangent about Clear Linux is a waste of time. Its only relevance to the discussion is in showing that -O3 works and doesn't compromise stability.

No, it is actually quite germane to the conversation because if you want -O3, use that distro. If I remember correctly, you say you have an application that you support using -O3 well, good for you. I bet it is nowhere near as complex as an OS kernel that has to run on countless pieces of hardware and compiled on countless compiler chains so again, I will take guidance from the people who actually are developing the code. Please note that if I ever have to use your code/product, I will take your guidance on how to compile the project. Look, if you believe that you can tell the difference a -O2 and -O3 build then go for it, but I personally believe that only hard-core gamers and HPC environments are ones where the tradeoffs are worth it. I just don't get where you feel you are entitled to tell me that I should be running a kernel ... or even distro compiled in a manor that suites you with typically one set of requirements than myself and others for that matter who have a different set of requirements. Long story short, stop telling the kernel devs how they should release their project unless you want to invest time and money to run through every darn scenario to prove your points.

[CochainComplex]

dekernel unfortunately I have never had the possibility to drive on a real racing track but during my mechanical engineering bachelor's time I have thought a lot about the physics involved in racing. And considering that also mech engineering is tied to real world boundaries means that usually there are a lot of safety or error margins calculated into the maximum - whatever this maximum means torque, speed, temperature...etc.
So my anecdotal picture is based on the assumption that both have a valid point. The engineer taking in consideration all the "what ifs" and the petrolhead not caring about it and just trying to push it to the max in the current situation. The true sweet spot is somewhere in between. Taking the what ifs into consideration but also being aware how likely or how fatal it is to the usecase.

But AFAIK it was never mentioned that o3 should be the default. Just a possible kconfig option. Which then at the end is up to the distro maintainers to risk it or not.

[tildearrow]

/Thread.

Stop pretending you're a moderator when you love derailing threads.

[dekernel]

But AFAIK it was never mentioned that o3 should be the default. Just a possible kconfig option. Which then at the end is up to the distro maintainers to risk it or not.

I would agree you didn't mention making -O3 the default. I think the decision to make it a not so trivial option shows the desire of the maintainers to make sure people tread carefully from the suggested defaults. I brought up the distro topic because it shows that if people do wish to use an -O3 build that there are options available if so inclined. The general tone I get from the people wanting the -O3 option is that they seem to believe that should be the default which I find to be a serious mistake for several reasons. If you poke through the comments, you will see my reasons, but the one major reason is if the maintainers of the code suggest a course of action, it is risky to think one knows more than they do and go a different route especially when we are talking about an OS kernel and not some code that runs at the application level.

[NotMine999]

karolherbst Can I borrow a box of popcorn? I ran out while reading this thread.

[coder]

And now you are just being an ass so I am moving on.

Just because I point out a contradiction in your position, that's no reason to sling insults.

I guess it's easier to blame me than reevaluate your own rationale.

Ah so we have yet another arm-chair warrior here who knows more than the kernel devs themselves.

It's well-established that -O3 is safe, and yet it seemingly remains taboo in the kernel community. I'll bet many of them are a lot more ignorant of it than you presume.

Look, they have far more to take into consideration than just a single release point including supporting multiple platforms and compilers

We're talking about GCC, though. For some other given compiler, maybe even -O2 isn't safe.

so why don't you just worry about your world and let them worry about their world.

Why are you worried about what I'm worrying about? Asserting your own right to have a position on the matter while denying mine is hardly fair.

Kernel performance is relevant to my interests. So, it's a concern to me when compiler optimizations aren't being explored and exploited to their potential.

They have done a really good job so I will take their suggestions over yours if you don't mind.

That's a very arbitrary judgment. Sure, Linux is usable by a lot of people, for a lot of things. So, in some sense, there's a notion of competency. Still, to say they're doing "a really good job" seems like it should somehow account for the amount of engineering hours and testing resources it's taken to achieve its current level of quality. And that quality metric really ought to be based on the normalized defect rate, rather than some vague sense of "this thing works for me".

Furthermore, Phoronix publishes an endless series of news stories about poorly-conceived datastructures and algorithms having to be ripped out replaced, not to mention all the performance bottlenecks that Jens Axboe is tripping over, left and right. So, I think there's room to make an argument that the kernel devs haven't overall had performance as their main focus.

Finally, I don't really have an overall sense of the distribution of opinions about compiler optimizations, among the kernel development community, do you? I think we can say there's not an overwhelming push for > -O2, or else Linus might have caved. Beyond that, I wonder if you're conjuring a broader consensus than what truly exists.

[coder]

if you believe that you can tell the difference a -O2 and -O3 build then go for it, but I personally believe that only hard-core gamers and HPC environments are ones where the tradeoffs are worth it.

I develop commercial products which run 24/7. In such a case, higher energy usage increases operating costs and CO2 emissions.

A couple % might not seem like a lot, but it's not as if -O3 is the end of the story. I already described how one might achieve even higher gains by investigating where it helps/hurts and specifically which sub-options are having a positive or negative impact.

I just don't get where you feel you are entitled to tell me that I should be running a kernel

You said you were concerned about stability above all else. If you believe optimizations are risky, then it stands to reason that you should be using -O0.

I honestly don't care what you use, but it bothers me when people arrive at a justification for their actions after the fact, especially if it's not even logically consistent.

Long story short, stop telling the kernel devs how they should release their project

I didn't. You're projecting views onto me that you oppose, to make me a convenient foil.

The only thing I opposed was the mere notion that -O3 isn't suitable. It's that idea which is holding back progress a lot more than whether or not there's a kernel config option for enabling it.

[arQon]

So my anecdotal picture is based on the assumption that both have a valid point.

Jep, and in that context I don't think anyone's really going to disagree too strongly.

I'm going to extend it a little though, because the reality is that in the next race, it rained the night before and the track is damp; or the crew delays a tyre change for another couple of laps; etc. The driver, having gotten away with his 160kmh cornering last time, does it again this time and finds himself in the crash barrier. THAT'S what -O3 has always meant.
Sometimes you get lucky, but sometimes you don't, and the increased risk of *being* unlucky is what makes O3 a very bad idea unless you only have to worry about a *very* small subset, like a handful of programs on a single platform. That obviously isn't true for the kernel, which obviously also doesn't have anything even remotely close to an adequate AT environment even for just current x86, let alone "legacy" x86 and ARM. How many performance regressions alone does Michael catch each year?

It's fine for people to claim that O3 doesn't randomly explode any more, but that's basically just survival bias, always has been, and would still be even if the O3 problems are only half as common as they used to be. There's just SO much code out there that the compiler generates bad output for, or has, or will in the next release, that it just doesn't make sense to invite those problems unless you have manpower to burn, and very few distros do.
Paradoxically, the kernel is in some ways one of the better places to try it, and I do agree with the sentiment that O3 is potentially being pushed aside for the wrong REASONS, even if doing so is the right OUTCOME. It's likely that O3 *is* fine for most users on x64 - but we also know that it has basically no benefit, and *some* of us know that there's a lot more to the world than the specific CPU in one person's PC and the current release of one compiler.

Even so though, the whining over the implementation specifics is unbelievable. It's not like Linus is going out of his way to make it even *nontrivial* to build -O3, let alone impossible. O3 is a second-class case for good reason, and seeing non-developer forum warriors try to position themselves as somehow better informed than kernel devs is just cringe-inducing. Basement-dwellers struggling with inferiority complexes and trying to imagine they have some sort of secret knowledge that the sheeple lack and a mythical enemy is trying to suppress is the stuff of cults, not software development.

The naivety of the idea that when people run into the bugs with O3 they should burn days or weeks of their time trying to get them fixed is similarly entitled and depressing, and you can tell they've never actually done it themselves. They seem to think the GCC team spend their days just waiting for bug emails to arrive, have no backlog, will understand the bug AND the cause immediately, can fix it immediately, and will release a new version of the compiler the following day. Every step of that process takes at least one order of magnitude longer than it apparently does in their imagination, and most will be closer to two. (Oh, and also that the team never has to deal with incorrect bug reports from people who are still learning the language, which for C++ is a LOT of people with a LOT of "unexpected" behavior).

Over time, code gets littered with "#if (CC_VERSION >= x) && (CC_VERSION <= y)", not for *capabilities* like new language features, but just to workaround compiler bugs. Likewise with assorted pragmas to control optimization level for specific functions, or the same thing in makefiles. All of this comes at a cost, and it's no surprise that those who are always shouting the loudest in threads like this are also overwhelmingly the ones who've never had to bear that cost. Somehow, these "experts" insistent on O3 being the default are the same ones who find having to use KCFLAGS too much of a hurdle. The mind boggles...

All of this noise over a *1%* improvement - in *benchmarks*, not even daily workloads.

Millions of users, and millions of servers supporting billions of users, are not a f**king playground to beta-test with just to satisfy some random child's idea that the world revolves around them.
If someone enjoys rebuilding the kernel on a daily basis with different flags, I am absolutely in favor of that. There are plenty of worse hobbies to have. But from the perspective of "put up or shut up", since they've repeatedly failed to failed to manage the first part we'd all be a lot better off if they at least adopted the second part.

[coder]

during my mechanical engineering bachelor's time I have thought a lot about the physics involved in racing. And considering that also mech engineering is tied to real world boundaries means that usually there are a lot of safety or error margins calculated into the maximum - whatever this maximum means torque, speed, temperature...etc.

So my anecdotal picture is based on the assumption that both have a valid point. The engineer taking in consideration all the "what ifs" and the petrolhead not caring about it and just trying to push it to the max in the current situation. The true sweet spot is somewhere in between. Taking the what ifs into consideration but also being aware how likely or how fatal it is to the usecase.

The way I imagine racing is that designers are dealing with a model of the car which they try to optimize in CAD, wind tunnels, etc. -- trying different approaches and trading off different variables to see what overall configuration works best. Once the thing is built, then the problem becomes more one of data science and dealing with a collection of simpler, more statistical-type models of the car's behavior which they develop and use to optimize specific parameters for a given race track set of weather conditions, tire/fuel strategy, etc.

I have no idea how accurate that is, but when it comes to predicting how fast a car can take a given corner, the driver is going to get an intuitive feel for how the entire package is handling and can feel approximately where the limit lies. This part, you might call the original form of "deep learning".