Meta's Transparent Memory Offloading Saves Them 20~32% Of Memory Per Linux Server

[Linuxxx]

It sounds like Meta set swappiness to zero (emergency overflow under severe memory pressure), then wrote a kernel patch to 'fix' it.

From the Meta article:

Swap algorithm

TMO aims to offload memory at pressure levels so low that they don’t hurt the workload. However, while Linux happily evicts the filesystem cache under pressure, we found it reluctant to move anonymous memory out to a swap device. Even when known cold heap pages exist and the file cache actively thrashes beyond TMO pressure thresholds, configured swap space would sit frustratingly idle.

The reason for this behavior? The kernel evolved over a period where storage was made up of hard drives with rotating spindles. The seek overhead of these devices results in rather poor performance when it comes to the semirandom IO patterns produced by swapping (and paging in general). Over the years, memory sizes only grew. At the same time, disk IOP/s rates remained stagnant. Attempts to page a significant share of the workload seemed increasingly futile. A system that’s actively swapping has become widely associated with intolerable latencies and jankiness. Over time, Linux for the most part resorted to engaging swap only when pressure levels approach out-of-memory (OOM) conditions.

For us normal plebs, I still recommend setting vm.swappiness=1, because any higher value will inevitably lead to those aforementioned "intolerable latencies and jankiness" when working with multiple apps and constantly switching between them.

Also note that there is a significant difference between swappiness=0 & swappiness=1, as noted in the other article above:

When talking about vm.swappiness, an extremely important change to consider from recent(ish) times is this change to vmscan by Satoru Moriya in 2012, which changes the way that vm.swappiness = 0 is handled quite significantly.

Essentially, the patch makes it so that we are extremely biased against scanning (and thus reclaiming) any anonymous pages at all with vm.swappiness = 0, unless we are already encountering severe memory contention. As mentioned previously in this post, that's generally not what you want, since this prevents equality of reclamation prior to extreme memory pressure occurring, which may actually lead to this extreme memory pressure in the first place. vm.swappiness = 1 is the lowest you can go without invoking the special casing for anonymous page scanning implemented in that patch.

And for anyone claiming otherwise:

You either don't use your PC properly or enjoy thrashing your SSD with unnecessary writes...

[dekernel]

No, it is because due to its late entry, gpl had a lot of catching up to do ...

This has done more for M$s monopoly more than what M$ has done for it directly. The result of the gplague ...

Big corporations however have the resources to protect their IP, plus they cared entirely about contributing things that they need, and most people are not big tech corporations, nor do they have their needs. It doesn't in any way put regular people more on parity with the industry - all it did was to facilitate the industry's growth, and boy did it grow invasive and parasitic.

But hey, at least one day, when the terminators come for us, they will probably run foss. So yey!

I am really confused by your statements. So are you for or against corporations contributing to the LInux world? Personally, I am all for it because they are the only ones that have the $$$ to put forth the development work for the large and complex features that we all benefit from and use on a daily bases.

[erniv2]

For us normal plebs, I still recommend setting vm.swappiness=1, because any higher value will inevitably lead to those aforementioned "intolerable latencies and jankiness" when working with multiple apps and constantly switching between them.

Also note that there is a significant difference between swappiness=0 & swappiness=1, as noted in the other article above:

Swappiness=1 is the min. recommend value, 0 basically deactivates page scanning unless you hit the min. freeram wich i dont know how high it is nowdays but 10 years back if you had 1GB ram it was 48mb or so, freeram is the min ram that is allways available to secure operation of the system if i remember correct it´s allways there and is only used by the kernel in emergencys and freed imidatly because it has to stay free, so if you set swappiness=0 you basically ask for the shitstorm coming because the system pages all the time once you hit freeram.

But thats not what we talk about here realy, the settings where there for platter based media with access times for random write around 20ms. Not for ram with ns access time or ssd´s with <=1ms write latency.

And for anyone claiming otherwise:

You either don't use your PC properly or enjoy thrashing your SSD with unnecessary writes...

Nowdays you dont have to use a swapfile, get fedora dude with inram swap, it is called zswap(instead of paging to a non volatile media, nvme, ssd, hdd, it compresses part of the ram the compression ratio in most cases should be 3:1 3GB memory can be stored in 1GB ram) wich is exactly whats discussed here, actually it´s better to start the compression of unused pages those at the end of the lru list early, so you never face a situation where the shitstorm happens and all the io/cpu cycles come at once wich is why linux stalls so hard when it hit´s the freeram threshold.

And when zswap runs out then you have to hit the next media wich would be your nvme. I once knew how to setup all this stuff but age and beer is taking it´s toll, i think there once was something like swap nice, where you could specify the priority of every swapfile, do systems eaven still use /etc/fstab back in the days you had to edit it all manually, haven´t looked at it for ages. I became a runs out of the box guy.

The standard value for swappiness is 50 btw.

And it makes sense to set swappiness=100 with zswap, wich makes it completly fair, for every fscache page destroyed one page gets compressed so in the end more fscache remains because every zswap page equals 2 fscache pages after compressing wich increases the chance of a cache hit and lowers io again

[andreano]

Yeah, just use zram with algorithm zstd and swappiness 100. If you don't use fedora, install the systemd service for it.

I skimmed the article, but I don't see an answer to the elephant in the room: If swap doesn't do the right thing, why not fix it?

[polarathene]

Nowdays you dont have to use a swapfile, get fedora dude with inram swap, it is called zswap(instead of paging to a non volatile media, nvme, ssd, hdd, it compresses part of the ram the compression ratio in most cases should be 3:1 3GB memory can be stored in 1GB ram)

The standard value for swappiness is 50 btw.

And it makes sense to set swappiness=100 with zswap, wich makes it completly fair, for every fscache page destroyed one page gets compressed so in the end more fscache remains because every zswap page equals 2 fscache pages after compressing wich increases the chance of a cache hit and lowers io again

Fedora is using zram not zswap, they are different (zswap tends to be stuck around the 3:1 ratio).

The default swappiness value is 60, as per the kernel docs, and it's been that value for quite some time. You'll also notice the docs mention swappiness range is 0-200.

I forget when but in the past decade sometime I believe the range was increased from 0-100 to 0-200, so that you could favour bias towards swapping pages (anon memory) vs file cache (executables / scripts needed from disk to run actual logic). 100 has been for a balanced/unbiased weight between those two types.

The docs advise going beyond 100 for zram and zswap or swap device on faster storage than the filesystem (which file backed pages would be re-read from), but rather than 200, state a ratio should be considered which may vary by workload.

IIRC, when pages are read back into active pages from swapped pages, they aren't immediately cleared from swap storage (as if they were soon swapped out again, that'd be wasted I/O on writing back to swap?), so with zram (and I think zswap) you'd get some pages duplicated in the ram swap area and the active used memory?

With ZRAM, if you have additional swap devices of lower priority such as swap files on disk, once those start getting used due to zram being filled, you get LRU inversion as more active swap doesn't get moved into zram. There is a tunable IIRC that requires some manual/timer command to mark zram pages, and then when performed again at a later time, any inactive pages that were marked would be moved to a backing storage such as a swap file, which could avoid the LRU inversion, but otherwise not desirable until zram capacity is near full.

ZSwap is a bit interesting with it's cache pool in RAM which can be sized to a specific % of memory and is a capacity of compressed pages. ZRAM is sized with an uncompressed capacity, but with a much more capable compression ratio (I've had 7:1 with zstd before using 1GB compressed, most of it was from leaky apps slowly incrementing memory usage over weeks), wheras zswap is stuck with roughly 3:1 IIRC (not due to compression algo, but allocator z3fold, zram has a zsmalloc allocator that can compress more at once I think). When Zswap cache pool is filled up, it'll evict based on LRU to the backing swap devices, but the pages will be decompressed (as opposed to the similar zram feature I mentioned which evicts compressed pages).

I don't think you can have zram prioritized in front of zswap though, the zswap cache pool would be in front of all swap devices? Thus some wasted CPU with compression and wasted memory with pages between them (zswap evicts pages from pool once they're decompressed to a swap device though). Maybe that's worthwhile if you need disk backed swap.

[ddriver]

I am really confused by your statements. So are you for or against corporations contributing to the LInux world? Personally, I am all for it because they are the only ones that have the $$$ to put forth the development work for the large and complex features that we all benefit from and use on a daily bases.

I just find it sad and silly for plebs to celebrate that as if it somehow benefits them. I guess for many people all they have left is to cheer at the advances of their masters. Evidently that makes them less losers in their own eyes. Yey for them new alloy chains, amazing, cutting edge tech progress, now we can have chains a lot stronger while retaining the same weight. Yey, stronger chains, now we can hope they will reduce the weight by a few percent in the next revision.

What I am saying is foss has made little to nothing for the world, and has overwhelmingly benefited its exploiters. Which does strongly clash with that notion of humanitarian nobility that the foss initiative has been parading to be.

I have a problem with dishonesty really. I'd have zero problem with everything they do, if they just didn't lie and pretend otherwise. It is merely another business model, which is precisely why despite all the claims of taking the side of "humanity" foss ended up immensely benefiting precisely the same entities it was once designed to oppose.

If you design something to fix a severe and growing issue, and instead all it does is end up contributing immensely more to it, to the extent of cementing the issue as chronic, I'd say that is a monumental failure and a huge embarrassment to everyone involved. That is not a force of good to oppose vile monopolies, just a parallel effort in precisely the same direction.

Who in the right mind would cheer at that?

[dekernel]

I just find it sad and silly for plebs to celebrate that as if it somehow benefits them...
Who in the right mind would cheer at that?

If by celebrating you meaning being appreciative of the effort put forward for the missing functionality without judgement for intent, then I guess you can put me in that group. I find it interesting that if an individual puts forward effort to "scratch their itch", you are apparently Ok with that, but when a business does the exact same thing, you present ill-will towards them. Who cares what they say their core intent is as long as they follow the intended license agreement. You seem hell-bent on reading between the lines from what business's say they are doing in hopes to find all the negative possibilities. The day that Linus starts following your apparent dislike for business's and rejecting patches (yes, I am also reading between the lines here for your intentions) is the day that Linux starts it's downward trajectory in my simple plebeian view.

[sinepgib]

If by celebrating you meaning being appreciative of the effort put forward for the missing functionality without judgement for intent, then I guess you can put me in that group. I find it interesting that if an individual puts forward effort to "scratch their itch", you are apparently Ok with that, but when a business does the exact same thing, you present ill-will towards them. Who cares what they say their core intent is as long as they follow the intended license agreement. You seem hell-bent on reading between the lines from what business's say they are doing in hopes to find all the negative possibilities. The day that Linus starts following your apparent dislike for business's and rejecting patches (yes, I am also reading between the lines here for your intentions) is the day that Linux starts it's downward trajectory in my simple plebeian view.

Well, if their itch is more efficiently destroying privacy, then I don't see presenting ill-will as intrinsically bad. But, at the same time, keeping it closed wouldn't stop them from doing that, but we would also miss the positive side-effect of having the ability to use the same code for ourselves.

[dekernel]

Well, if their itch is more efficiently destroying privacy, then I don't see presenting ill-will as intrinsically bad.

I would agree with you here in theory, but your statement is incongruous with reality. By the shear fact that the product/feature is being open-sourced shines the light on them attempting to destroy privacy because you can actually see what they are doing which allows you to change your behavior to not allow it.

[sinepgib]

I would agree with you here in theory, but your statement is incongruous with reality. By the shear fact that the product/feature is being open-sourced shines the light on them attempting to destroy privacy because you can actually see what they are doing which allows you to change your behavior to not allow it.

What? No. I mean running their servers more efficiently FWIW, not that the feature itself violates privacy. You see, Meta is a company whose business model is a direct attack on privacy, just like Google's. This feature is made so their servers, whose purpose is to attack privacy, run more cheaply. And we don't see what runs on userspace on those servers.