LLVM's LLD Linker Made Incredible Progress In 2016, Much Faster & Linking More

[phoronix]

Phoronix: LLVM's LLD Linker Made Incredible Progress In 2016, Much Faster & Linking More

LLVM's LLD Linker continues making great strides and with the year coming to an end, developer Rui Ueyama has shared a status update as well as posting some performance benchmarks for the gains made by LLD this year...

http://www.phoronix.com/scan.php?page=news_item&px=LLVM-LLD-2016-State

[geearf]

For those curious.
LLD doesn't work with GCC7: https://llvm.org/bugs/show_bug.cgi?id=28414
and a patch to support -fuse-ld=lld : https://gcc.gnu.org/ml/gcc-patches/2.../msg00088.html

[speculatrix]

has linux kernel compilation with LLVM-CLANG stalled? there doesn't seem to be anything new here:

http://llvm.linuxfoundation.org/index.php/Main_Page

[carewolf]

A 20 core system? That is nice for a benchmark of how well the threading scales (apparently terrible), but what good is that for realistic comparison?

[carewolf]

Btw. If you do a proper test of this at some point. Also please check peak memory use, that is what I am most exited about in lld. Linking big projects like Chromium can really blow the memory use of linkers when they do the garbage collection step.

[name99]

A 20 core system? That is nice for a benchmark of how well the threading scales (apparently terrible), but what good is that for realistic comparison?

The world is larger than just your house...
There are multiple clients of LLD and SOME of them (think Google, think Apple) are perfectly happy to provide developers with 20 core machines if that makes them a few percent more productive.

If you read the full ongoing details of LLD development (they are not secret) you would see that they are trying hard to balance algorithms that only win on serious machines (say 8 cores or more) against performance on one or two core systems. Obsessing over snapshots along the way is just silly --- next week the algorithms will change, then again the week after that. Right now is a very fluid point in development.

You might find reading the LLVM lists educational since you're apparently quite happy to throw out claim about what is or is not feasible in the multi-threading of linking --- you could learn something...

[name99]

Btw. If you do a proper test of this at some point. Also please check peak memory use, that is what I am most exited about in lld. Linking big projects like Chromium can really blow the memory use of linkers when they do the garbage collection step.

I assume you're referring to LTO? There are some up-to-date performance/memory numbers here:

http://llvm.org/devmtg/2016-11/Slides/Amini-Johnson-ThinLTO.pdf

Memory usage (and performance) for debugging info is still far higher than anyone likes, but that is being worked on and slowly improved.

[carewolf]

I assume you're referring to LTO? There are some up-to-date performance/memory numbers here:

http://llvm.org/devmtg/2016-11/Slides/Amini-Johnson-ThinLTO.pdf

Memory usage (and performance) for debugging info is still far higher than anyone likes, but that is being worked on and slowly improved.

No, LTO is basically compiling, not really linking. It is even worse in memory consumption, but it is handled by the compiler not the linker. I was referring to the step that deduplicates symbols, in C/C++ projects there are many duplicated symbols, especially in debug mode, because the same headers are compiled over and over again, and not always all dead-code eliminated or inlined in the compilation step.

[carewolf]

The world is larger than just your house...
There are multiple clients of LLD and SOME of them (think Google, think Apple) are perfectly happy to provide developers with 20 core machines if that makes them a few percent more productive.

That would be rather stupid in this case, since the Xeon E-models are two generations behind and slower per thread than consumer CPUs. Though, it does make them more productive in the compilation phase that is almost 100% parallel, which is why I also like them, but I wouldn't bring up tiny percentages in multiprocessing though, because you already lost a lot more than a tiny percentage in single thread performance.