Latest Slab Cgroup Memory Controller Patches Saving ~1GB RAM Per Host On Facebook Servers

[smitty3268]

A note on browsers. My use case tends to be a bit weird. Under Chrome, I usually open 5-6 different profiles for different gsuite|o365|other uses, each with their own plugins, settings, everything. Same with firefox using profile manager, never both at once though. I do this as I log into that many different companies, orgs, personal, etc profiles every day at least, and this tends to lead to the memory usage over uptime, which I tend to measure in months. When I can killall -9 firefox|chrome|chromium|brave and see some 40gb of memory drop instantly, I do consider it a memory leak, but seems mostly related to the multi-profile use. This is consistent between browsers, virtual memory and used memory is getting harder to tell apart.

Have you tried viewing about : performance on Firefox? It should show you the memory use per tab and addon.

[tuxd3v]

You might want to keep fewer browser tabs open

why?
That guy brought in 64 GB Ram exactly because he thought it would be great.. but the kernel is using cache like a b*ch ( suck*ng more and more.. ).
No knob to limit cache usage in Linux, there are some knobs spread around, but none works like a barrier( like in kernel 2.4 series )..

[tuxd3v]

(...)
Curious thing is how windoze machines are still typically around 8gb, and people find this acceptable, so what is linux doing wrong?

What is going wrong?
Everything!!

toolkits like gtk3/qt4-5, and so on..

Also Linux doesn't provide a knob for limiting the amount of Ram that can be used as cache..
So it will be always a Pain, because majority of Ram will be cache..
In Kernel 2.4 was there a knob for that, but was deleted in 2.6 branch..I think we still miss it today..

[cynical]

the least usage I tend to see is roughly 10gb usage with things freshly opened on a restart of the desktop.

Linux isn’t doing anything wrong. The memory usage scales with the amount available. On my 16GB desktop system, initial memory usage is around 1.2GB. On my 8GB laptop, It’s around 950MB.

[mikus]

If you're only considering mikus and yourself it could also be your usage the atypical one

Seriously though, how come you have such a high memory Firefox session going on mikus? Is there any extension able to suspend unused tabs after a certain amount of time? Ala The Great Suspender for Chromium based browsers. For me it's godsend, I could live without it of course, but it makes things go smoother.

Still, 64GB of RAM in that laptop, damn... plenty of memory to spread around several VMs.

Oddly I still run great suspender under chrome, new tab suspender (best I found under ff migrating) across all profiles, and mostly the same. It's sort of creeping me out at this point, wondering if some sort of bitcoin miner running in .js on a page, but no.

It's not just my tubes and pr0n using my memory.

I always use noscript/ublock, so nothing works at first, and after whitelisting 10 things, maybe a site works. Pretty sure it's not general malfeasance, but really just seems like a nasty vectored leak of some sort. This is not something new, and have been dealing with for years with consistent use patterns/builds across systems.

I moved from ubuntu to arch to try to escape mostly same issues. Moved from amd to nvidia, even intel on my bastard hybrid xps15 (9560, 64gb ram, 1gb toshiba ssd) I use now. My desktop was a precision 7910 with 20 cores and 40 threads, 128gb of ram, dual m.2 960 pro samsungs, gtx1070, and mostly ran into the same odd crippling sometimes even blowing that box out using arch after time when the system would freak out one way or another. My xps is a good medium point currently, but doesn't seem to matter how much I upgrade systems, start over, my usage is always the same, and thus... odd.

I have gotten so sick of it occasionally I try windoze again, as my xps has the factory win10 pro partition still. It falls apart instantly when I throw even another single 4k display on it, let alone the 2 that it can't seem to do with the built-in 4k.

Really, I'd just like to know what it takes to uncork things and not have lag and other weirdness in usage I tend to get with linux over time. Or having to pay for 64-128gb of ram per system.

[mikus]

I'm usually watching htop all day every day, under kde|cinnamon|mate as this is really nothing new. Usually I notice my desktop lagging, freaking out, and notice htop showing my memory pegged at some 60gb, all used. I'll start killing things using some scripts, firefox first - oh, there went 20-40gb of memory, kill libreoffice, there went another 5gb, kill the stupid electron apps (slack, signal, telegram, teams [yeah, like that]), there went another 5gb of ram.

I don't see my system typically at my most basic apps using less than 3.5gb of ram. I've not found a really good visual memory usage explorer that shows differences from virtual/actual memory, so always difficult to tell. When the system starts freaking out visually, I always know however.

Virtual vs. real seems a fine line. When I notice, virtual doesn't seem bad, but generally consumed, I notice the system freaking out. Some 15 years watching different systems, first decade+ or so ubuntu, last several arch. Things just sort of run away. Pulseaudio's pavucontrol for years used to grab a good 30-40gb of ram before I realized it had gone crazy, another dumb app doing dumb shit. Luckily I could afford the ram to watch it be dumb or not care for a bit, as it's a useful thing, but it took a while before someone made it not crap the bed.

A note on browsers. My use case tends to be a bit weird. Under Chrome, I usually open 5-6 different profiles for different gsuite|o365|other uses, each with their own plugins, settings, everything. Same with firefox using profile manager, never both at once though. I do this as I log into that many different companies, orgs, personal, etc profiles every day at least, and this tends to lead to the memory usage over uptime, which I tend to measure in months. When I can killall -9 firefox|chrome|chromium|brave and see some 40gb of memory drop instantly, I do consider it a memory leak, but seems mostly related to the multi-profile use. This is consistent between browsers, virtual memory and used memory is getting harder to tell apart.

Libreoffice gets a bit crazy too, virtual memory in htop I'll see grabbing some 32gb of ram, but really tends to use around 5-6gb for me. Killall again shows me just what it's using, but I have to wonder how much it *really* uses in that virtual memory allocation normally that things tread on each other, and the cpu/memory in the process.

I am heavy business 24/7 as a network/security consultant dude, and ask a lot of my desktop/laptop systems, usually running a few vm's, browser profiles as above, very complex spreadsheets in libreoffice, aforementioned crap. Same since starting with ubuntu full-time circa 2005. Throw in youtube, steam, and other random crap it gets busy, but I've never needed consistently so much ram constantly than I do today. I do things like run a 3-6 displays at a time off various gpu's, and have seen about every flaw in linux and every Dist/DE along the way for the past few decades, so no idea what weirdness I hit.

[KaoDome]

Sitting here writing this, Firefox is using under 1 GB of RAM. So either you've uncovered an immense memory leak, or your usage is highly atypical.

If you're only considering mikus and yourself it could also be your usage the atypical one

Seriously though, how come you have such a high memory Firefox session going on mikus? Is there any extension able to suspend unused tabs after a certain amount of time? Ala The Great Suspender for Chromium based browsers. For me it's godsend, I could live without it of course, but it makes things go smoother.

Still, 64GB of RAM in that laptop, damn... plenty of memory to spread around several VMs.

[NotMine999]

Firefox is probably the worst offender (I've caught it using as much as 48gb of memory lately), and linux seems to amplify the memory usage terribly. I don't even know how folks use a pc with less than 16gb of ram with linux these days. Curious thing is how windoze machines are still typically around 8gb, and people find this acceptable, so what is linux doing wrong?

You might want to keep fewer browser tabs open

[NotMine999]

i am guessing those are some huge servers.

They have to be huge to tracking innocent web users across every Internet-attached device they use and web page they visit.

[JustRob]

Some of the what and why is explained here: https://lkml.org/lkml/2019/9/5/1132