Linux 6.8 Landing A Tantalizing Optimization For Common $PATH-Based Searches

[phoronix]

Phoronix: Linux 6.8 Landing A Tantalizing Optimization For Common $PATH-Based Searches

For the execve() system call to execute a program by pathname, the Linux 6.8 kernel is set to land a new optimization to "dramatically" speed-up PATH searches...

https://www.phoronix.com/news/Linux-6.8-Execve-Faster-PATH

[c117152]

Didn't nfs mounts behave differently with execve() under some specific conditions?

[Congelli501]

likes of Musl libc, Python, Perl, and others tend to use the execve-based approach

Should probably be execvpe-based.

NodeJS uses execvpe too.
Anyway, it's a nice optimization.

[toves]

execvpe(3) is a library routine (as are the other exec*p* routine I believe) so I always assumed those routines stat(2)ed each candidate progname along the PATH and moved on if ENOENT etc or possibly the relevant permissions (r-x) bits were not set. Only where the file existed and had the correct perms that execve(2) would be attempted but of course could fail if the wrong format or architecture or missing interpreter etc.
Unless there are cases where execv(2) is possible but stat(2) is not I would have thought execvpe(3) wouldn't often call execve(2) all that frequently in practice.
Still a nice optimisation that would reduce the impact in the system of malicious storm of failing execs etc.

[fitzie]

funny thing is that search path for executables is shrinking as there's no more /sbin /bin /usr/sbin /usr/X11/bin. On the other hand, anyone using cli python tools knows that searching PYTHONPATH is quite time consuming for simple scripts with that have a bunch of package dependencies.

[curfew]

Oh but the code is bad quality because it contains use of goto. Should've rewritten the whole thing in Rust instead!

[archkde]

execvpe(3) is a library routine (as are the other exec*p* routine I believe) so I always assumed those routines stat(2)ed each candidate progname along the PATH and moved on if ENOENT etc or possibly the relevant permissions (r-x) bits were not set. Only where the file existed and had the correct perms that execve(2) would be attempted but of course could fail if the wrong format or architecture or missing interpreter etc.
Unless there are cases where execv(2) is possible but stat(2) is not I would have thought execvpe(3) wouldn't often call execve(2) all that frequently in practice.

I don't know if this is reasonably possible in practice (of course you can write a seccomp-bpf filter that blocks stat while letting through execve), but one problem with that approach is that the file you have determined to exist using stat may be gone until you come around to the exec.

[Weasel]

Oh but the code is bad quality because it contains use of goto. Should've rewritten the whole thing in Rust instead!

Rust needs to goto offline;

[Volta]

Another great optimization:

https://lkml.org/lkml/2024/1/9/515

Analyze and reorganize core networking structs (socks, netdev, netns, mibs) to optimize cacheline consumption and set up build time warnings to safeguard against future header changes. This improves TCP performances with many concurrent connections up to 40%.

Linux 6.8 is going to be in great shape.