Google Engineers Propose "Machine Function Splitter" For Faster Performance

[phoronix]

Phoronix: Google Engineers Propose "Machine Function Splitter" For Faster Performance

Google engineers have been working on the Machine Function Splitter as their means of making binaries up to a few percent faster thanks to this compiler-based approach. They are now seeking to upstream the Machine Function Splitter into LLVM...

http://www.phoronix.com/scan.php?page=news_item&px=Machine-Function-Splitter

[FireBurn]

I'm guessing this will be part of PGO? Otherwise how does it know what's going to be hot or cold?

[discordian]

I'm guessing this will be part of PGO? Otherwise how does it know what's going to be hot or cold?

Yep.

Seems to be (originally) developed as countermeasure to LTO + excessive inlining creating lotsa rarely accessed code.

[bug77]

Maybe I'm missing something here, but is it ok to add something as complex as this (with the required maintenance and whatnot) to a compiler in return for only a few percent improvement? Does no one do a cost-benefit analysis?

[ed31337]

This doesn't sound all that complex to me. Aside from being able to figure out which code path is hot vs cold, moving the code around is pretty trivial in my opinion. In fact, this perhaps doesn't need to be inside the compiler at all -- this could make sense as a run-time profiling and self modifying code feature.

[cb88]

Maybe I'm missing something here, but is it ok to add something as complex as this (with the required maintenance and whatnot) to a compiler in return for only a few percent improvement? Does no one do a cost-benefit analysis?

It's done all the time. And isn't *that* complex... its just deciding if the code is hot or not based on profiling and hinting the compiler about which sections of code should be in the same cache lines or not. Hot code gets grouped with hot code and cold code with cold...so that when when a cache line is loaded or prefeched its more likely to get entirely hot code.

[Ardje]

Maybe I'm missing something here, but is it ok to add something as complex as this (with the required maintenance and whatnot) to a compiler in return for only a few percent improvement? Does no one do a cost-benefit analysis?

16 to 35% from iTLB and 62..67% of sTLB, I do not call that a few percent. That the total benchmark "only" shows 1.5% of performance increase is a total of other factors too.
But the more threads you have the more important that TLB becomes, or the cache itself.

[brouhaha]

deciding if the code is hot or not

Just do it mechanical Turk style. Put code snippets on a web page that asks "hot or not?"

[skeevy420]

What about Goldilocks code? Code that's not too hot & not too cold; code that's just right.

[zyxxel]

What about Goldilocks code? Code that's not too hot & not too cold; code that's just right.

That's not a problem. Most probably not affected. Unless some cold code got moved out, making your Goldilocks code fit in fewer cache lines. The important thing with code is that it's normally quite few lines that represents a lot of consumed time. So the gains for that code will matter significantly even if the majority of the code wasn't touched by any optimization.

That's also why a developer shouldn't start their coding by trying to optimize - it's generally hard to identify the 5% or 2% most important code sequences early on..