New Patches Aim To Reduce Memory Use While Compiling The Linux Kernel

[phoronix]

Phoronix: New Patches Aim To Reduce Memory Use While Compiling The Linux Kernel

Updated patches were sent out today that aim to reduce the maximum memory usage while compiling the Linux kernel. In turn for memory constrained systems that attempt to compile the kernel this should lead to less swapping and faster build times...

https://www.phoronix.com/news/Linux-Kernel-Build-Less-RAM

[caligula]

How much memory does it need? Shittiest rpi servers have 8 gigs of ram.

[NobodyXu]

In particular, switching from malloc() to calloc() in an objtool loop yielded a reduction of 3.7% in peak memory use while processing vmlinux.o.

I was so confused by why this can reduce memory usage.
After checking the commit https://lore.kernel.org/lkml/2022121...eissschuh.net/ , it turns out that it does two things:
- Move allocation out of the loop
- Replace `malloc` + `memset` with `calloc`

Michael I believe it will be better phrased as "In particular, switching from malloc() in an objtool loop to calloc() out of the loop yielded a reduction of 3.7% in peak memory use while processing vmlinux.o."

[elatllat]

git would eat 4 GB RAM before building the kernel would eat 100 MB RAM ...

[neoe]

I was so confused by why this can reduce memory usage.
After checking the commit https://lore.kernel.org/lkml/2022121...eissschuh.net/ , it turns out that it does two things:
- Move allocation out of the loop
- Replace `malloc` + `memset` with `calloc`

Michael I believe it will be better phrased as "In particular, switching from malloc() in an objtool loop to calloc() out of the loop yielded a reduction of 3.7% in peak memory use while processing vmlinux.o."

I'm not sure, I guess it comes from
1. overhead of malloc structure.
2. the internal functions called by malloc ( like 'brk' 'mmap' ) not like to cross memory pages
3. fragments
also the change should bring a little speed improvements.
the malloc algorithm in this case is already 96.3% perfect, no so bad.

[cripsy]

I'm not sure, I guess it comes from
1. overhead of malloc structure.
2. the internal functions called by malloc ( like 'brk' 'mmap' ) not like to cross memory pages
3. fragments
also the change should bring a little speed improvements.

He merged multiple allocations into one big one, heap only has to keep track of one allocation, plus data is probably closer together (idk if the heap on linux stores its info intrusively).

[Mitch]

How much memory does it need? Shittiest rpi servers have 8 gigs of ram.

Well, one thing about the mem usage is that if we lower it, we could compile more kernels at once on any given machine. So if I want to try to see what the kernel would be like with various patches and changes, I might just do concurrent compiles to save time and make it simpler.

[rogerx]

Well with the advent of Rust requiring SSE2 for the past several years and subsequently everything else depending on Rust, I have already tossed all of my old computers (that would have utilized this patch) in the trash.

(Not... I merrily boxed the old computers up, for the day Rust is tossed in the trash for something better.)

[oleid]

Well with the advent of Rust requiring SSE2 for the past several years and subsequently everything else depending on Rust, I have already tossed all of my old computers (that would have utilized this patch) in the trash.

(Not... I merrily boxed the old computers up, for the day Rust is tossed in the trash for something better.)

Huh? What are you talking about? The official rust complier supports i586 just fine. i486 could probably work as well, it's just a matter of passing the correct flags to llvm.

Or are you talking about running the pre-compiled binary on an ancient machine? If so, you should consider cross-compiling. It's easy and saves time and energy.

[darkbasic]

How much memory does it need? Shittiest rpi servers have 8 gigs of ram.

This is the highest end model which is almost never available.