Fedora's FESCo Rejects The Idea Of "-fno-omit-frame-pointer" As Default Compiler Flag

[coder]

Is this strictly an x86 thing? On AArch64, the performance impact of carrying a frame pointer should be negligible, due to double the general-purpose registers. I would like to see some benchmarks on that, but you'd obviously want to use a more recent core than something like the Pi's A72. Michael, do you have access to an Ampere Altra?

[Weasel]

Also, who do you think makes the software that gets shipped? Developers. And they have to get something nice out of the platform to keep doing it too.

I'm a developer and there's no way in hell I'd tolerate performance drops on stuff I don't debug.

[King InuYasha]

Exactly that!
For them and at datacenter scale, this is worth it.

Now things get a bit murky beyond that because fedora right now isn't the fedora that i used to know.
Fedora that i used to know: desktop distribution! Plain and simple.
Fedora right now: desktop/workstation distribution, cloud, server, ....

For the fedora workstation (aka desktop users) forcing this change is senseless and only causes (slightly) bigger packages with degraded performance. Sure, all manageable and likely in the single percentage points but a degradation nevertheless. On desktop releases they should focus on user efficient releases, not developer efficient.

For their other spins, specifically server and cloud, this change would make total sense!

Usually in these discussion the money side wins. Which in this case would be server/cloud (desktop users don't pay). I'm glad this didn't turn out to be the case here.

I assume you missed the GNOME developers saying they wanted this too? Or that the KDE folks would have benefited from this as well? Both big desktops have tooling that massively benefits from working real-time tracing and profiling.

Also, who do you think makes the software that gets shipped? Developers. And they have to get something nice out of the platform to keep doing it too.

[markg85]

They have to rebuild the entire distro for their internal builds which they can certainly continue to do but they have been attempting to upstream their desired changes which is generally a good thing to do. These internal changes otherwise tend to become tech debt over time. At the data center scale, the net performance and developer efficiency gains are worth it for them. For other folks, the situation is less clear. The desktop team apparently wants it because it saves them time running their tools and several upstream projects are enabling the flags by default but other teams including the tooling team thinks it's not worth the tradeoffs.

Exactly that!
For them and at datacenter scale, this is worth it.

Now things get a bit murky beyond that because fedora right now isn't the fedora that i used to know.
Fedora that i used to know: desktop distribution! Plain and simple.
Fedora right now: desktop/workstation distribution, cloud, server, ....

For the fedora workstation (aka desktop users) forcing this change is senseless and only causes (slightly) bigger packages with degraded performance. Sure, all manageable and likely in the single percentage points but a degradation nevertheless. On desktop releases they should focus on user efficient releases, not developer efficient.

For their other spins, specifically server and cloud, this change would make total sense!

Usually in these discussion the money side wins. Which in this case would be server/cloud (desktop users don't pay). I'm glad this didn't turn out to be the case here.

[Hafting]

They can have a slow distro if they like, and we can use other distros if we like. (Something arch based for convenience, or gentoo based if some extra performance is worth all the extra compiling.)

A frame pointer makes some debugging "easier", but all that debugging is possible anyway. The "hard" parts about figuring out a stack layout without frame pointers, can conveniently be offloaded onto a computer and software. Somehow, anyone doing debugging work has access to a computer! These days computers tend to have many cores, so real-time debugging is helped by running this extra work on some other core. And even in the single-core case, the extra work of real-time stack unwinding will only be a linear slowdown.

So instead of "why do this take 10min on a highly optimized build", you get "find out why this takes 20 min when also executing a ton of debugging hooks. And that is better than "this takes 15 min for everyone".

Finally, lots of software is not such a complete mystery. Lots of debugging happens without debugging options, a few extra printf() and you figure out what is wrong.

[RahulSundaram]

But still not worth a standard just to make it easier for a few million dollar companies....

Sorry for asking another uninformed question. What has upstreaming to do with compiler flags? Shouldn't that be totally independend?

As I have noted, it isn't just large organizations, individual developers depending on the group want this too and any performance improvements done by either can flow to everyone. So the tradeoffs are more complex.

From the distribution perspective, compiler flags aren't independent because the whole change proposal is about changing the distribution default compiler flags and it is the upstream for whatever internal rebuilds that organizations like Meta are using and not the individual projects. Also the individual upstream projects are also often impacted by whatever flags major distributions use as well since they get performance feedback and bug reports from them. Changing a major distribution flags tend to affect the ecosystem pretty broadly.

[Anux]

They have to rebuild the entire distro for their internal builds which they can certainly continue to do but they have been attempting to upstream their desired changes which is generally a good thing to do. These internal changes otherwise tend to become tech debt over time. At the data center scale, the net performance and developer efficiency gains are worth it for them. For other folks, the situation is less clear. The desktop team apparently wants it because it saves them time running their tools and several upstream projects are enabling the flags by default but other teams including the tooling team thinks it's not worth the tradeoffs.

Hm ok. But still not worth a standard just to make it easier for a few million dollar companies. According to Michaels benchmarks there are workloads that get their throughput halfed from just no-omit-F-P.

Sorry for asking another uninformed question. What has upstreaming to do with compiler flags? Shouldn't that be totally independend?

[archkde]

Shouldn't it be possible to identify the stack layout from the debug Info and the instruction pointer? Which should be enough to infer the frame address.

The only issue I can see with this approach is alloca or other variable sized allocations on the stack. But those seem like a bad idea in general to me: either your variable sizes data is small enough to always fit on the stack, in which case why not do a fixed size buffer? Or you risk blowing the stack size and should have allocated on the heap to begin with.

Yes, this approach is very much possible. In fact nearly every tool supports this. The only downside is that when using perf, the stack needs to be copied on every sample because Linux doesn't contain a DWARF parser.

[RahulSundaram]

Yeah sorry, just read it myself. But why make it standard, they allready use those flags, so it's not like it's hard to do

They have to rebuild the entire distro for their internal builds which they can certainly continue to do but they have been attempting to upstream their desired changes which is generally a good thing to do. These internal changes otherwise tend to become tech debt over time. At the data center scale, the net performance and developer efficiency gains are worth it for them. For other folks, the situation is less clear. The desktop team apparently wants it because it saves them time running their tools and several upstream projects are enabling the flags by default but other teams including the tooling team thinks it's not worth the tradeoffs.

[Anux]

A lot of the typical questions include these is addressed in the change proposal. They want to use this in production and not just for debugging, the FESCO ticket linked mentions that Google also does this. https://fedoraproject.org/wiki/Chang...in_production?

Yeah sorry, just read it myself. But why make it standard, they allready use those flags, so it's not like it's hard to do. If they don't care about a little performance loss there is also -Og.

I don't understand why they would make that standard in any -O > 1.