Linus Torvalds Cleans Up The x86 Memory Copy Code For Linux 6.4

In recent years Linus Torvalds hasn't had the time to write too much original new code for the Linux kernel himself with these days mostly managing developers, providing insightful mailing list posts, and reviewing code for merging into the kernel tree along with related tasks. For Linux 6.4 though he did manage to write up some new code.

Last week Linus Torvalds merged his own "x86-rep-insns" branch to the kernel for the Linux 6.4 merge window. The focus on Torvalds' code for Linux 6.4 is on cleaning up the x86 memory copy code. He explained in the Git merge:

"This cleans up a lot of our x86 memory copy code, particularly for user accesses. I've been pushing for microarchitectural support for good memory copying and clearing for a long while, and it's been visible in how the kernel has aggressively used 'rep movs' and 'rep stos' whenever possible.

And that micro-architectural support has been improving over the years, to the point where on modern CPU's the best option for a memory copy that would become a function call (as opposed to being something that can just be turned into individual 'mov' instructions) is now to inline the string instruction sequence instead.

However, that only makes sense when we have the modern markers for this: the x86 FSRM and FSRS capabilities ("Fast Short REP MOVS/STOS").

So this cleans up a lot of our historical code, gets rid of the legacy marker use ("REP_GOOD" and "ERMS") from the memcpy/memset cases, and replaces it with that modern reality. Note that REP_GOOD and ERMS end up still being used by the known large cases (ie page copyin gand clearing).

The reason much of this ends up being about user memory accesses is that the normal in-kernel cases are done by the compiler (__builtin_memcpy() and __builtin_memset()) and getting to the point where we can use our instruction rewriting to inline those to be string instructions will need some compiler support.

In contrast, the user accessor functions are all entirely controlled by the kernel code, so we can change those arbitrarily."

The Linux 6.4 merge window remains open until next weekend with more feature code continuing to be published.