Google Engineers Propose "Machine Function Splitter" For Faster Performance

Google engineers have been working on the Machine Function Splitter as their means of making binaries up to a few percent faster thanks to this compiler-based approach. They are now seeking to upstream the Machine Function Splitter into LLVM.

The Machine Function Splitter is a code generation optimization pass for splitting code functions into hot and cold parts. They are doing this stemming from research that in roughly half of code functions that more than 50% of the code bytes are never executed but generally loaded into the CPU's data cache.

The Machine Function Splitter with splitting of functions into hot and cold paths will then ensure the hot paths are loaded into the CPU cache but the unlikely to be executed portions not, in turn saving the limited cache space for other hot code.

Google engineers with the Machine Function Splitter found this pass for LLVM to offer a 2.33% mean improvement in runtime thanks to a reduction in misses to the L1i cache and TLB miss rates. They found that iTLB misses reduced by 16% to 35% while sTLB misses reduced by 62% to 67%. This function splitter was tested to be of help for both Intel and AMD systems. When testing with SPECInt 2017 they found the int rate geometric mean to improve by 1.6%.

It's a small improvement, but all the work adds up and not bad for just a compiler optimization pass. More details on the proposed Machine Function Splitter for LLVM via the LLVM mailing list.