Linux 5.0-ad1 Patch Lets You Build The Kernel With "-march=native"

[starshipeleven]

Would you “world recompile” from the livecd? It would take longer as the compile tool chain is not running optimised.
I’m not a Gentoo user either, but I would assume you recompile the kernel and build tool chain (libc, gcc, etc); reboot and then world recompile.

The issue is that any other software required for boot and the GUI and the shell and anything else needs to be recompiled too or you can't boot.

About the toolchain not being "optimized", I doubt it has a significant impact on compile speed.

[xiando]

This patch is not at all the same as the experimental patch Gentoo (ab)uses (5010_enable-additional-cpu-optimizations-for-gcc.patch). Read the patch from Alexey and you'll quickly see something isn't right. That patch doesn't only allow you the option of using march=native, it also changes lots of assembly code.

I recommend against using the patch from Alexey Dobriyan and here's why: The whole point of using march=native is to allow gcc to figure out what instructions are available to you on your particular CPU and use them efficiently. Alexey goes a bit further and optimizes the actual source code for you, here's one example:

+#ifdef CONFIG_MARCH_NATIVE_MOVBE
+ movbe 0 * 4(%rdx), RL0d;
+ movbe 1 * 4(%rdx), RR0d;

And that's .. perhaps just fine. I see two sides of this: On one hand, it may give real world performance improvement. That's good. On the other hand there's the real risk of bugs being introduced by checking what GCC sees available in march=native and changing a lot of assembly code depending on ifdef of CONFIG_MARCH_NATIVE_MOVBE, CONFIG_MARCH_NATIVE_POPCNT and so on. Perhaps there are no such bugs and there won't be, just that there is some additional risk compared to just compiling with march=native. If there's bugs with this that aren't there without the patch then it's an open question if it's the result of gcc or someone's hand-optimized assembly.

Overall, the patch is pretty interesting. I guess the reason I overall do not like it is that there's a difference between changing a compiler flag and changing that compiler flag and a whole lot of assembly code.

[gmturner]

I think maybe I recently posted a similar question/concern to a similar thread, so, sorry, I suppose.

But: I always worry that saying, in effect, "hey, gcc, go nuts optimizing my kernel for this or that sub-target" will cause gcc to violate various assumptions in arch-specific kernel code about register coverage, atomicity, cache barriers, etc. For example, couldn't this cause kernel code to try to use obscure vector math registers that would need to be added to kernel context tracking structures, to avoid various disaster scenarios?

Or is that not a valid concern because these patches are doing all of that work, or perhaps the kernel is coded defensively against any such possibilities, or else uses fancy pragmas that should completely defend against any such concerns by design, or... (lots of scenarios I could imagine without taking a deep dive into the source)?

[Weasel]

The issue is that any other software required for boot and the GUI and the shell and anything else needs to be recompiled too or you can't boot.

The wonders of "the Linux Way" right? Some real pasta with spaghetti dependencies.

[starshipeleven]

The wonders of "the Linux Way" right? Some real pasta with spaghetti dependencies.

Try booting Windows ARM with a x86 NT kernel on x86 hardware, and you will realize why you just posted bullshit again.

[Weasel]

Try booting Windows ARM with a x86 NT kernel on x86 hardware, and you will realize why you just posted bullshit again.

Nobody posted this patch for cross compilation, nice cop out.

[starshipeleven]

Nobody posted this patch for cross compilation, nice cop out.

Since when recompiling OS-critical software so it is actually using the current system's instructions only (so it does not crash on start) has any relevance to "The Linux Way" and "spaghetti dependencies", nice trolling.

[Weasel]

Since when recompiling OS-critical software so it is actually using the current system's instructions only (so it does not crash on start) has any relevance to "The Linux Way" and "spaghetti dependencies", nice trolling.

Because you said you have to compile everything or it won't boot.

In a sane "way", mixing an OLD code that doesn't use native instructions WITH code that uses native instructions is perfectly useable, no conflicts etc, just perhaps slower since only half or whatever is using them. But nah, in Linux, you don't get modular sane components with sane specifications. Even building with a different version of GCC and mixing them up can cause a disaster in some cases.

The simplest layman way I can describe this is: every component must be a black box, with inputs and outputs. As long as it satisfies the spec, it must work, no exceptions, period. No hidden dependencies or specific "full system configs" reliance. If it has a dependency it must follow a spec itself and as long as that dep satisfies its spec, it must work, NO EXCEPTIONS.

That's called sane engineering. And it doesn't apply just to software.

[gilboa]

Because you said you have to compile everything or it won't boot.

In a sane "way", mixing an OLD code that doesn't use native instructions WITH code that uses native instructions is perfectly useable, no conflicts etc, just perhaps slower since only half or whatever is using them. But nah, in Linux, you don't get modular sane components with sane specifications. Even building with a different version of GCC and mixing them up can cause a disaster in some cases.

The simplest layman way I can describe this is: every component must be a black box, with inputs and outputs. As long as it satisfies the spec, it must work, no exceptions, period. No hidden dependencies or specific "full system configs" reliance. If it has a dependency it must follow a spec itself and as long as that dep satisfies its spec, it must work, NO EXCEPTIONS.

That's called sane engineering. And it doesn't apply just to software.

As some with considerable experience in the field of kernel based high performance network monitoring (Mostly Linux, but also BSD and Windows), I must confess the either my reading comprehension is officially MIA, are you simply have no idea what you're talking about.

We *regularly* compile my out of tree code (several SMLOC worth, with considerable amount of x86_64 / x32 / ARM7 asm code) using nothing more than the Fedora / RHEL / Ubuntu kernel headers and using completely different gcc flags (-O, --march, etc) and at times, even different GCC version with zero issues.

Again, unless I failed to understand your point, you simply have no idea what you're talking about.

- Gilboa

[Weasel]

As some with considerable experience in the field of kernel based high performance network monitoring (Mostly Linux, but also BSD and Windows), I must confess the either my reading comprehension is officially MIA, are you simply have no idea what you're talking about.

We *regularly* compile my out of tree code (several SMLOC worth, with considerable amount of x86_64 / x32 / ARM7 asm code) using nothing more than the Fedora / RHEL / Ubuntu kernel headers and using completely different gcc flags (-O, --march, etc) and at times, even different GCC version with zero issues.

Again, unless I failed to understand your point, you simply have no idea what you're talking about.

- Gilboa

I was replying to his claim that "it won't boot". Whether that is true or not is irrelevant. If it works for you, then that's good, it means "it will boot" and in this case it's not the "Linux way" he envisions (with every package being built specifically around specific others, aka how most distros are today).