Announcement

Collapse
No announcement yet.

Facebook's New Linux Slab Memory Controller Saving 30~40%+ Of Memory, Less Fragmentation

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Facebook's New Linux Slab Memory Controller Saving 30~40%+ Of Memory, Less Fragmentation

    Phoronix: Facebook's New Linux Slab Memory Controller Saving 30~40%+ Of Memory, Less Fragmentation

    Back in September we wrote about Facebook's Roman Gushchin working on a new slab memory controller/allocator implementation that in turn could provide better memroy utilization and less slab memory usage. This wasn't ready in time for the 5.5 kernel but a revised patch series was sent out last week...

    http://www.phoronix.com/scan.php?pag...Alloc-December

  • #2
    that in turn could provide better memroy

    Comment


    • #3
      Is it as cool as it sounds?

      Comment


      • #4
        Originally posted by timofonic View Post
        Is it as cool as it sounds?
        nope.

        Comment


        • #5
          Originally posted by timofonic View Post
          Is it as cool as it sounds?
          I hope it is, and this time I don't care if it's from FB. I remember when people spent time discussing and benchmarking SLAB, SLOB, SLUB, and SLQB (whatever happened to it) like they do now with i/o schedulers...

          Comment


          • #6
            Will it be available in kernel 5.6 ??

            Comment


            • #7
              +30% improvement? Ok, what's the trade-off?

              Comment


              • #8
                Originally posted by halo9en View Post

                I hope it is, and this time I don't care if it's from FB. I remember when people spent time discussing and benchmarking SLAB, SLOB, SLUB, and SLQB (whatever happened to it) like they do now with i/o schedulers...
                SQLB?

                I think that's called MariaDBB now.

                Comment


                • #9
                  Originally posted by pmorph View Post
                  +30% improvement? Ok, what's the trade-off?
                  Some use cases apparently show a performance regression. Remember that the hyperscallers have use cases that are somewhat different than what others see in their day-to-day use on a desktop or a small business server. The question may be if those performance regressions can be addressed, and if not, if they are considered acceptable for a potentially new default allocator.

                  Comment


                  • #10
                    It's good to have options. I have 'slub_max_order=2' on one of my system's boot lines because that prevents slabs from becoming too big (i.e. 32KB vs 16KB) which impacts fragmentation. You could reduce it to 1 to have them be 8KB - or 0 for 4KB, but then you perhaps might as well use the SLAB allocator. Which I do in general, it's just in this case it would have resulted in more memory being used due to the filesystem inode being just too big to fit into 1KB; and hence, a 4KB slab only fits 3.

                    The influence of the allocator can depend on other options you have compiled your kernel with - e.g. support for ext4's quota function increases structure size, but also weird things like reader-writer semaphores that can otherwise increase performance under heavy use, and that may push it over the point (impacting slab efficiency and fragmentation over time) - as well as what operations you actually do (rsync of millions of files?) which is why the workload matters so much.

                    It's also one reason 32-bit can be faster; I have one minimal 32-bit system which fits seven ext4 inodes in 4KB, and this can mean less I/O. But it might also mean memory gets fragmented because just one inode can hold up the 4KB slab from being reallocated.

                    Comment

                    Working...
                    X