GNOME's GLib Adds GMemoryMonitor As Another Step In Helping Cope With Linux RAM Pressure

[starshipeleven]

I'd agree that this is is unreasonable. It is also unreasonable to kill one application in place of the other.

OOM isn't random, it does use some euristic and some rules to decide what must die. root processes for example are automatically prioritized, processes using a ton of RAM have their priority decreased as the usage increases.

In most cases the OOM will kill a web browser, or another similar application that is using multiple GBs of RAM.

Any way I'd argue that desktop critical functions should run on determenistic set of assumptions, including required minimum of memory which could be reserved for stable operation, so DE itself would not have out-of-memory situation, it is possible with pre-allocating necessary minimum and using some sort of internal allocator.

This can be done simply giving the DE higher prioroty for the OOM, but I quite frankly never had a situation where the OOM killed my desktop.

It's always Firefox or Thunderbird or a VMWare VM that get terminated when I run out of RAM.

[ferry]

...
Let's make an example: some application is wasting RAM, some important process (say the GUI) is allocating some more RAM and finds no free. Clearing caches is only so much useful and it still needs more RAM. What you do? Shut down the GUI? Implement some OOM logic in the GUI to terminate the less important processes within itself and/or degrade performance to save RAM?

You are just moving the burden of OOM to each application, while not solving the overall issue.

The terminology is clouding our perspective:

• There is no such thing as wasting RAM. The user chooses an application, the application allocates RAM, due to virtual memory the RMA gets swap in and out. I learned programming on a PDP11 with 1MB RAM. I allocated 10MB because I needed it. That was a multi-user system (HP-UX). Oops.
• It's not for the OS to decide which process to close. To enable the user to close a bad program (yes they do exist) at least the system "must remain responsive". But that doesn't mean anything either, except we want the terminal or the GUI to function. And probably a few other process to that we forget in the heat of the discussion.

The real problem IMHO is that disk accesses cause IO Wait (as they are called in the system monitor) and this doesn't only block the process causing it, but blocks the kernel itself and all other processes.

The real solution is not to kill a process, or the offending process. The solution is to stop the offending process. This stops swapping and all other process will (after swap in) start functioning normally again.

So if the user chooses to run an application with a bigger need for RAM than present he should be allowed to. But the process should be scheduled to run for short times and then paused again, based on the amount of IO Wait it causes.

Shifting the logic to the application is wrong as this allows a DoS attack (the current situation). The OOM is wrong too, as it can not know which application to kill and we don't want it killed.
The right place is in the scheduler of the kernel, which needs to consider resource usage of the application in the scheduling algorithm.

What we need is a completely unfair scheduler :-)

[blacknova]

Wouldn't it be possible for kernel's scheduler to throttle normal applications with intensive disk IO? Some of them run directly with mmap to file...

[starshipeleven]

There is no such thing as wasting RAM.

In that post I meant that there is an application that is actually wasting RAM, allocating RAM for no other reason than to fill it and cauuse an OOM situation.

It's not for the OS to decide which process to close.

You need to understand that when the OOM killer is triggered the situation is dire, and the whole system is at risk of not being able to accept input anymore.

But that doesn't mean anything either, except we want the terminal or the GUI to function. And probably a few other process to that we forget in the heat of the discussion.

This is why the OOM killer is more about finding the most likely RAM hog than the more important application to keep.

It's much easier to detect what applications the system can do without, especially on a desktop or a server.

The real problem IMHO is that disk accesses cause IO Wait (as they are called in the system monitor) and this doesn't only block the process causing it, but blocks the kernel itself and all other processes.

True, and one of the main reasons a lot of people aren't using swap anymore.
If I have to choose between the system locking up or losing Firefox or something, I'm ok with the latter.

The real solution is not to kill a process, or the offending process. The solution is to stop the offending process.

define "offending" in this context. How do you decide what process to stop. Do you want to randomly stop (freeze) the first process that goes OOM? That's only barely better than killing another process, as you can very well lock processes that are waited for by others in the GUI or network or whatever.

So if the user chooses to run an application with a bigger need for RAM than present he should be allowed to. But the process should be scheduled to run for short times and then paused again, based on the amount of IO Wait it causes.

Would be nice, but what about people without swap like me, or most embedded devices?

The OOM is wrong too, as it can not know which application to kill and we don't want it killed.

As said above, so far the OOM has always chosen wisely and killed secondary or non-critical applications, so I dispute this claim.

[arQon]

OOM isn't random, it does use some euristic and some rules to decide what must die. root processes for example are automatically prioritized, processes using a ton of RAM have their priority decreased as the usage increases.

sidenote, IDK if you just typo'd it, but it's "heuristic".

> In most cases the OOM will kill a web browser, or another similar application that is using multiple GBs of RAM.

jep

> This can be done simply giving the DE higher prioroty for the OOM, but I quite frankly never had a situation where the OOM killed my desktop.

Me neither, but then I don't use GNOME Shell. :P

> It's always Firefox or Thunderbird or a VMWare VM that get terminated when I run out of RAM.

And there's the rub. Even some POS app that's *leaking* 2GB (or is effectively doing so thanks to a crappy garbage collector), or an app that's flat-out bugging out and is caught in an infinite alloc loop, isn't going to get close to the working set of one of my VMs. And I expect similar is true for any heavy Chrome / Firefox user. So instead of the POS app getting killed, something important to the user gets killed instead.
(When I developed firmware for devices, the OOM killer would basically almost NEVER get the right process (which was invariably the test build I'd compiled 5 minutes ago). Instead, it would kill apache. But that's a bit of a special case. :P)

> You need to understand that when the OOM killer is triggered the situation is dire, and the whole system is at risk of not being able to accept input anymore.

I've never really seen this matter, frankly. At this point, swap thrash alone has probably made the system completely unusable already, even with an SSD.

> what about people without swap like me

I'd say you're doing it right. I run nearly all of my machines without swap now, because if the system NEEDS to touch swap in the first place then something has probably gone horribly wrong. I'd rather know about it sooner and still have a WORKING machine than have swap try to hide the problem for several minutes and have the machine totally locked anyway because the VMM handler is in a permanent uninterruptible thrash state.

Of course, in an ideal world, 99% of developers wouldn't be incompetent monkeys just copypasting code from SO, and we wouldn't need to worry about this sort of thing in the first place. :P

[ferry]

In that post I meant that there is an application that is actually wasting RAM, allocating RAM for no other reason than to fill it and cauuse an OOM situation.

That would be a bug. The user should be able to terminate it.

You need to understand that when the OOM killer is triggered the situation is dire, and the whole system is at risk of not being able to accept input anymore.

Critical applications should probable allocate what they need at startup. And with enough swap, you shouldn't be running of memory at all.

This is why the OOM killer is more about finding the most likely RAM hog than the more important application to keep.

So why does it kill my desktop instead of chromium (on my laptop without swap)

It's much easier to detect what applications the system can do without, especially on a desktop or a server.

Have to tried to build rust? I would prefer the build to complete slowly than be terminated after 2 hours.

True, and one of the main reasons a lot of people aren't using swap anymore.
If I have to choose between the system locking up or losing Firefox or something, I'm ok with the latter.

yeah, or something

define "offending" in this context. How do you decide what process to stop. Do you want to randomly stop (freeze) the first process that goes OOM? That's only barely better than killing another process, as you can very well lock processes that are waited for by others in the GUI or network or whatever.

the task causing massive IO Wait when running

Would be nice, but what about people without swap like me, or most embedded devices?

Embedded devices are normally fixed purpose, with known memory usage.The can configure swap, or not depending on the purpose. Which might be something real time = no swap and preallocated RAM.
On the desktop, you should use swap. And linux should handle it correctly.

Let me say again, currently it is easy to cause a DoS on a linux system for any ordinary user. No action is taking to fix this. Which is strange given the efforts to mitigate any other vulnerability.

As said above, so far the OOM has always chosen wisely and killed secondary or non-critical applications, so I dispute this claim.

Build rust. Open Qgit on the kernel. Open a few too many tabs in chromium. I never had OOM resolve the situation, with or withoutn swap.

[starshipeleven]

That would be a bug. The user should be able to terminate it.

The user can't do much when the system hits true OOM and nothing reacts automatically to free RAM.

Critical applications should probable allocate what they need at startup.

Good luck with that.

And with enough swap, you shouldn't be running of memory at all.

If you start heavy swapping the system becomes unresponsive because linux swapping is complete and utter fucking useless trash even on a SSD. Not that it is much better on Windows.

So why does it kill my desktop instead of chromium (on my laptop without swap)

Because you use GNOME I guess. GNOME is a known RAM hog.

Have to tried to build rust? I would prefer the build to complete slowly than be terminated after 2 hours.

If you have swap it will chug along slowly in any case, the only difference is that the system will be usable-ish or kinda unresponsive.

On the desktop, you should use swap. And linux should handle it correctly.

I can agree on that. Currently, swapping is basically a DoS and that's something that is not acceptable.

[ferry]

Because you use GNOME I guess. GNOME is a known RAM hog.

Nope, KDE. But you mean the effectiveness of OOM killer depends on your desktop?
If, so I guess we agree that it doesn't really work and we need something better.

If you have swap it will chug along slowly in any case, the only difference is that the system will be usable-ish or kinda unresponsive.

With one parallel thread on 8GB it will (needs 6GB). With 8 threads is dies.

If you start heavy swapping the system becomes unresponsive because linux swapping is complete and utter fucking useless trash even on a SSD. Not that it is much better on Windows.

Strong words. And yes when I trigger it for the nth time I want to smash my keyboard through my screen too. So agreed.

I can agree on that. Currently, swapping is basically a DoS and that's something that is not acceptable.

Not the swapping itself. The unrestricted swapping. But as is, there is no way to restrict it.

[onicsis]

Better now than never, millions of peoples use their software. OOM killer it's not a proper answer, it's the last measure that could be taken in the last moment.
Memory or CPU/GPU/Power consumption, resources exhaustion should be main concern and priority, not only for GNOME but for any complex software like desktop environments or modern browsers.