Announcement

Collapse
No announcement yet.

New Patches Aim To Reduce Memory Use While Compiling The Linux Kernel

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • New Patches Aim To Reduce Memory Use While Compiling The Linux Kernel

    Phoronix: New Patches Aim To Reduce Memory Use While Compiling The Linux Kernel

    Updated patches were sent out today that aim to reduce the maximum memory usage while compiling the Linux kernel. In turn for memory constrained systems that attempt to compile the kernel this should lead to less swapping and faster build times...

    https://www.phoronix.com/news/Linux-...Build-Less-RAM

  • #2
    How much memory does it need? Shittiest rpi servers have 8 gigs of ram.

    Comment


    • #3
      In particular, switching from malloc() to calloc() in an objtool loop yielded a reduction of 3.7% in peak memory use while processing vmlinux.o.
      I was so confused by why this can reduce memory usage.
      After checking the commit https://lore.kernel.org/lkml/2022121...eissschuh.net/ , it turns out that it does two things:
      - Move allocation out of the loop
      - Replace `malloc` + `memset` with `calloc`

      Michael I believe it will be better phrased as "In particular, switching from malloc() in an objtool loop to calloc() out of the loop yielded a reduction of 3.7% in peak memory use while processing vmlinux.o."

      Comment


      • #4
        git would eat 4 GB RAM before building the kernel would eat 100 MB RAM ...

        Comment


        • #5
          Originally posted by NobodyXu View Post

          I was so confused by why this can reduce memory usage.
          After checking the commit https://lore.kernel.org/lkml/2022121...eissschuh.net/ , it turns out that it does two things:
          - Move allocation out of the loop
          - Replace `malloc` + `memset` with `calloc`

          Michael I believe it will be better phrased as "In particular, switching from malloc() in an objtool loop to calloc() out of the loop yielded a reduction of 3.7% in peak memory use while processing vmlinux.o."
          I'm not sure, I guess it comes from
          1. overhead of malloc structure.
          2. the internal functions called by malloc ( like 'brk' 'mmap' ) not like to cross memory pages
          3. fragments
          also the change should bring a little speed improvements.
          the malloc algorithm in this case is already 96.3% perfect, no so bad.

          Last edited by neoe; 27 December 2022, 11:19 PM.

          Comment


          • #6
            Originally posted by neoe View Post

            I'm not sure, I guess it comes from
            1. overhead of malloc structure.
            2. the internal functions called by malloc ( like 'brk' 'mmap' ) not like to cross memory pages
            3. fragments
            also the change should bring a little speed improvements.

            He merged multiple allocations into one big one, heap only has to keep track of one allocation, plus data is probably closer together (idk if the heap on linux stores its info intrusively).

            Comment


            • #7
              Originally posted by caligula View Post
              How much memory does it need? Shittiest rpi servers have 8 gigs of ram.
              Well, one thing about the mem usage is that if we lower it, we could compile more kernels at once on any given machine. So if I want to try to see what the kernel would be like with various patches and changes, I might just do concurrent compiles to save time and make it simpler.

              Comment


              • #8
                Well with the advent of Rust requiring SSE2 for the past several years and subsequently everything else depending on Rust, I have already tossed all of my old computers (that would have utilized this patch) in the trash.

                (Not... I merrily boxed the old computers up, for the day Rust is tossed in the trash for something better.)

                Comment


                • #9
                  Originally posted by rogerx View Post
                  Well with the advent of Rust requiring SSE2 for the past several years and subsequently everything else depending on Rust, I have already tossed all of my old computers (that would have utilized this patch) in the trash.

                  (Not... I merrily boxed the old computers up, for the day Rust is tossed in the trash for something better.)
                  Huh? What are you talking about? The official rust complier supports i586 just fine. i486 could probably work as well, it's just a matter of passing the correct flags to llvm.

                  Or are you talking about running the pre-compiled binary on an ancient machine? If so, you should consider cross-compiling. It's easy and saves time and energy.

                  Comment


                  • #10
                    Originally posted by caligula View Post
                    How much memory does it need? Shittiest rpi servers have 8 gigs of ram.
                    This is the highest end model which is almost never available.
                    ## VGA ##
                    AMD: X1950XTX, HD3870, HD5870
                    Intel: GMA45, HD3000 (Core i5 2500K)

                    Comment

                    Working...
                    X