Yes, Linux Does Bad In Low RAM / Memory Pressure Situations On The Desktop

[Volta]

I had hope I won't see posts coming from birdie, but damn it. What's worth to note installing game on Steam makes Windows crawl.

[Neraxa]

It's exactly how I'd expect it to behave.

A system with the memory all used up and no swap space available runs like shit...you don't say...

No, it should not. It should handle the situation gracefully, not cause the entire system to lock up. If the system locks up, thats a major bug. There is simply no reason for it.

If memory is exhausted, the most critical processes have to be protected such a X.org, audio servers, etc, so there should be away to immunize certain processes from being killed. Another possibility is to implement an epoll interface allowing a monitor program to be notified if system is out of memory and throw up a dialog to the user asking which process should be killed. There is no reason at all for a system to just lock up except that it is designed in a totally incompetent manner by people who dont know what they are doing. Its been doing this for ever and Linux runs like an unstable crap OS that makes Windows shine by comparison. Its been this way for over 10 years. Its just appalling and infuriating that it takes so long to get the kernel people to do anything about it, showing again that nobody cares about desktop users.

[Neraxa]

OOM killing is hard to generalise, and the memory pressure indicators are hard to interpret. Not only is earlyoom likely to be the fix that desktop users are looking for, it will be an early adopter of PSI stats once someone works how to use them in the general case, so that's another reason to follow it. Meanwhile, the discussion around the project is more more informed. It is astounding that so few people know about it. I could understand if people reported that it didn't work, but actual knowledge of the tool seems close to zero. https://github.com/rfjakob/earlyoom
I tried killing a 4GB VM ... it is not too hard to get the kernel to deal badly with the situation. I could not kill the same machine when earlyoom was running. swap made no difference either way, of course; swap advocates are barking up the wrong tree if they think swap settings can save the kernel's OOM reaction.

early killing would help, I would also recommend having an epoll event API for an external monitor program to be notified early of an OOM event and then can throw up a dialog to ask the user which process to kill, certain processes that are critical have to be protected like X , window manager, etc. Can be user configurable to automatically kill processes or to ask. Monitor program could be configured by user to go after certain processes in particular firefox first and to leave critical processes alone. Another possibility is a reserved memory space for near OOM conditions which allow critical applications to keep on using that memory and allow the user to have a responsive system to be able to use the dialog to kill a process.

I wonder if killing is not the only issue, the problem has felt like some sort of bug in the scheduling system or interaction between allocation, i/o scheduling and process scheduling which causes the lockups. An OOM condition should never cause a lock up.

[tuxd3v]

Not at all - which would you rather debug?
"The production DB server became super slow and caused everything else to fail strangely for several hours until someone manually rebooted the VM", or
"The production DB server died, there is an easy to understand OOM message in the logs, and since it stopped completely it was immediately restarted by a monitoring service".

The memory reclaim, feature doesn't work very well..
probably will be there tons of GB of pagecache, that can be reclaimed, but usually they are not( Linux do favour speed over resources, prefer to continue caching, until the system dies, but it works better now than in the past, but still insufficient.. )..

first you should drop your speedup caches,
Which will lead to more ram available, in that way, you can work, it will increase IO to access data, but better that, than kill the entire stack..
Limit the number of sessions, when you are recovering, then probably you will need to kill some session that is doing something nasty..
Workout a better 'limits.conf', enforce Soft and Hard-Limits..

Rebooting complex stacks is not a so simple task, and is not the answer

[archkde]

The memory reclaim, feature doesn't work very well..
probably will be there tons of GB of pagecache, that can be reclaimed, but usually they are not( Linux do favour speed over resources, prefer to continue caching, until the system dies, but it works better now than in the past, but still insufficient.. )..

first you should drop your speedup caches,
Which will lead to more ram available, in that way, you can work, it will increase IO to access data, but better that, than kill the entire stack..
[…]

This is precisely what the kernel does right now and what causes the kind of problem the article describes when the entire working set is evicted from the cache.

[Neraxa]

There should be userspace daemon for this like in Android. However, "desktop" part of Linux is busy with removing features and braking userspace all the time. It amazes me how Linux kernel developers care about not breaking compatibility while "desktop" does it every time. I hope Valve comes with sane distribution and ends this madness.

With the recent 32 bit removal, which is a disaster, its time valve should just get into making their own distro, based on Ubuntu or Fedora of course, but with no more of these stupid anti-desktop moves, creating unuseable user interfaces like Gnome, breaking backward compatability, or other anti-desktop user movies. a desktop distro should fully support 32 bit only installs, should have a strong focus on backward compatability, desktop apps like gaming and desktop publishing. It should have a feature rich window system that allows the user to configure everything through GUI configuration tools, rather than treat the user like an idiot. I want settings and configurability. Its hard to figure out how to even change the color scheme on Gnome.

I get the feeling that Fedora/Red Hat and Ubuntu absolutely hate their desktop users and do everything they can to throw water in their faces and especially with this 32 bit removal move which is really a stab at desktop users.

[onicsis]

. There is no reason at all for a system to just lock up except that it is designed in a totally incompetent manner by people who dont know what they are doing. Its been doing this for ever and Linux runs like an unstable crap OS that makes Windows shine by comparison. Its been this way for over 10 years. Its just appalling and infuriating that it takes so long to get the kernel people to do anything about it, showing again that nobody cares about desktop users.

No you are wrong, it's not about [in]competence, they don't give a sh%t about OS stability. They do not care, the problem persist from many, many years.
No differences if booting from a optical media, LIVE CD or HDD. Or from floppy drive . OOM cand do nothing, because it's more a I/O problem, a constant bottleneck. Neither applications don't do too well, don't seem to shine in this regard [concurrent I/O management far from having a good implementation]. Some type of concurrent I/O, on spinning magnetic drives, freeze/lock UI doesn't matter which DE is used KDE, XFCE, LXQT etc, or distribution ...

[aksdb]

I consider it user error and not a deficiency of the OS. The user pushing something too hard isn't necessarily a computer problem for that matter. No matter how many safe guards are in place, it doesn't help it if a user pushes something beyond its limits. In this case they removed a safe guard and pushed it beyond its limit.

How is the kernel supposed to know if it needs to halt Plasma, Firefox, Mplayer, LibreOffice, KSP, Kate, Yakuake, makepkg or what to keep the system in a usable manner? All it can do is juggle stuff with what little resources it has available.

IMHO, this is really a problem that should be solved by a daemon that a user can configure it to kill/halt/suspend-to-disk programs in a specific order because the kernel can't read my mind to know what I consider to be the more important task. It would also need a blacklist of things to not kill ever like the actual desktop environment.

There may be situations where it is not clear which process to kill. But IMHO most times this thing happens, it is perfectly clear. If my gradle java build managed to eat up 24 GB of my 32 GB RAM within a few minutes, pushing my system to its limit ... then just kill that fucking process and leave the other processes alone.
I guess a good heuristic would at least be, to kill the process with the most memory usage whenever the system starts stalling for a few seconds. Although I'm aware that the main problem might be the detection of the stall.

[andreano]

Has nobody mentioned earlyoom yet?

Edit: My phone browser's page search is broken.

[pc777]

Both the kernel and applications (in this case Chrome/Firefox) can do better (unless you think 4GB is not enough to run a browser "reasonably"). Do I think app developers will actually _do_ anything in response to failing mallocs()... not likely. They've had decades of not worrying about memory and they're probably not gonna start now. Shit, my first computer ran in only 128k. By the time java entered the scene it was game over.