Microsoft Is Exploring LTO+PGO For A Faster Linux Kernel

[phoronix]

Phoronix: Microsoft Is Exploring LTO+PGO For A Faster Linux Kernel

While more and more software vendors are employing link-time optimizations (LTO) and profile-guided optimizations (PGO) for leveraging the compiler to squeeze out performance, it may surprise many that Microsoft engineers are exploring LTO+PGO for the Linux kernel in their own quest of achieving greater Linux performance...

http://www.phoronix.com/scan.php?page=news_item&px=Microsoft-Exploring-PGO-Linux

[chuckula]

Ian Bearman, a softeare engineering manager at Microsoft,

STILL Beat TildeArrow even though he edited that out.
And his ears aren't soft.

[latalante]

Google is already doing this.
Probably the kernels on their Pixels have long been compiled with LTO + PGO.
https://android-review.googlesource....droid-4.14-pgo
https://android-review.googlesource.com/c/kernel/common/+/708394

[CochainComplex]

It is a pitty that this is "now" on display just because MS mentioned it. A lot of other folks pushed that but almost no move because the majority of devs prefer generic (obvious reasons sure). Why does it need Intel, MS or (Google) to get Linux Devs (dispite of gentoo guys) on board?

How often have I read the sentence "(LTO/PGO) is not worth the effort" on stackoverflow and co. edit: in this forum too btw.

[torsionbar28]

It's not surprising at all that Microsoft is pursuing this. After all, Linux is the dominant OS within their Azure cloud platform, powering the majority of Azure instances today. The ability to improve performance on existing hardware is beneficial to Azure's profitability. Remember, every FOSS activity by Microsoft is purely to improve profitability for Microsoft. Not that there's anything inherently wrong with that, but it's important to always examine the utility of their FOSS contributions with this in mind.

[bpetty]

I doubt PGO is as niche as you say, Michael. Many workloads share the same core paths. It would not be hard to verify the impact of different PGO payloads. For example: No PGO, PGO under n-number of offline games, PGO under heavy web server and other network load, PGO with both offline gaming and heavy network load. That is just an example. My hunch is that PGO, generated from some niche scenario, would not negatively impact other scenarios... but still give them a small, but measurable, performance boost.

[Ub42]

There's also been the likes of Clear Linux to take it even further by making use of AutoFDO (Auto feedback directed optimizations) for greater performance of their distribution.

I don't think Clear Linux folks are still using AutoFDO with recent version of GCC. The last version producing faster binaries compare to plain O2 was GCC 6. Google not using GCC anymore, the implementation found in GCC is broken or experimental.

[kcrudup]

Remember, every FOSS activity by a company is purely to improve profitability for that company.

FTFY.

[CochainComplex]

I doubt PGO is as niche as you say, Michael. Many workloads share the same core paths. It would not be hard to verify the impact of different PGO payloads. For example: No PGO, PGO under n-number of offline games, PGO under heavy web server and other network load, PGO with both offline gaming and heavy network load. That is just an example. My hunch is that PGO, generated from some niche scenario, would not negatively impact other scenarios... but still give them a small, but measurable, performance boost.

fprofile-partial-training and fprofile-reproducible should help to prevent such miss optimizations.
espcialy the second one

https://gcc.gnu.org/onlinedocs/gcc-10.1.0/gcc/Instrumentation-Options.html#index-fprofile-reproducible

[torsionbar28]

FTFY.

No, not so much. There are many companies our there who have pet side projects just because their employees love Linux. For example, id software releasing Linux binaries for Doom and other games back in the 1990's. We talk about how little <1% Linux market share on Valve's Steam platform here in 2020 - do you know how many Linux Desktop Gamers there were in the 1990's? There were like 3. Clearly this was a pet project at the time and not driven by profitability even in the slightest. In the FOSS spirit, id software even went on to release the source code to this AAA game.

With Microsoft, we know their hostile anti-competitive practices, we know their cut-throat market behaviors, we know their embrace extend extinguish tactics. In light of their decades long track record which continues to this day, it would be illogical to view any FOSS contributions from MSFT as a no-strings-attached pet project of community good will. We know that unlike many other companies, MSFT's business model and corporate culture do not allow for such things.

I would be happy for MSFT to one day prove me wrong. To do so, they would need to perform one or more items from this list. These things would enable true interoperability and would allow for a healthy competitive business desktop software ecosystem.

1. Release the source code to the obsolete Win9x OS.
2. Release the source code to the obsolete NT3 or NT4 OS.
3. Open source the Office file formats.
4. Open source the Exchange network protocol.
5. Open source the AD network protocol.
6. Open source the NTFS file system.

Mark my words, they never ever will though. Their business model relies entirely on vendor lock-in through these proprietary formats and protocols.