Meta's Transparent Memory Offloading Saves Them 20~32% Of Memory Per Linux Server

[akira128]

Let me fix that one real quick for you:

Open source is not about people, it's about big tech, whether it's stuff they need to oppress and exploit the serfs better or sharing something largely irrelevant to cultivate a positive image. It started as an alleged opposition to monopolies like m$, but quickly defined itself as the "cheapo collab alternative" business model to those who didn't felt like having the big tech boys hands rummaging through their pocket. Which is why toady foss, while super useful for hyperscale enterprise or mission critical solutions, is still useless to a regular consumer. The same m$ that foss was created to oppose now makes more money on foss than the entire end user collective combined.

And no, before you say it, android users are not linux users, it is google that is using linux for android, so it can in turn use people through those devices. So in a way, an android user is rather being used by linux by being used by googul.

That's one more reason to sleep better tonight, knowing a big tech company nobly contributed to foss something it need for its own specific use case.

WTF!? I'm struggling to sort out exactly how this TMO GPLv2 licensed code oppresses and exploits people. It seems like Meta funded the development for it. If someone wants to submit a PR to modify the codebase they can -- but it's their choice. I think it's more likely that Meta wanted to improve performance, and so they developed this TMO feature so they could benefit from it. Including it in the Linux kernel makes sense as they wouldn't need to constantly compile and install an external kernel module (or equivalent operation) for their production systems. I don't think anyone's making the argument that it's inherently noble or honorable to contribute to open source -- it's not. But if a company develops a feature for a problem that they're trying to solve and then open sources it -- what's wrong with that? Does the community as a whole not benefit from that? Isn't that why the GPL was created in the first place -- to encourage and protect open source innovation?
Ok, so if open source is such a dirty, nasty, oppressive system....then please tell us O wise master, what/where exactly is this perfect system that we should all be using?

[erniv2]

TMO performs memory offloading to swap at subliminal memory pressure levels, with turnover proportional to file cache. This contrasts with the historical behavior of swapping as an emergency overflow under severe memory pressure.

How do you get that to work for home users ? It would be nice if zswap kicks in early espacialy when linux gets used with memory balloon and hyper-v, i get it all the time that the memory usage of a vm is maxed out to the balloon size while the actual free command in linux shows stuff like 1gb used 1 gb cache swap 0 unused 4g when you allocate 6gb ram or more for ballooning it hogs all 1st and to realy fill the ram actually never happens.

[MorrisS.]

Which kernel will integrate it?

[ddriver]

TMO performs memory offloading to swap at subliminal memory pressure levels, with turnover proportional to file cache. This contrasts with the historical behavior of swapping as an emergency overflow under severe memory pressure.

How do you get that to work for home users ? It would be nice if zswap kicks in early espacialy when linux gets used with memory balloon and hyper-v, i get it all the time that the memory usage of a vm is maxed out to the balloon size while the actual free command in linux shows stuff like 1gb used 1 gb cache swap 0 unused 4g when you allocate 6gb ram or more for ballooning it hogs all 1st and to realy fill the ram actually never happens.

Well, not that much pressure, just a single threshold value. I've personally found default system settings a bit too swappy. It is OK for systems with little memory, but that legacy is gragging on a bit too long. There's a difference in starting to swap at like 60% memory load, depending on whether it is a couple of gigs vs tens of gigs vs hundreds of gigs. But I dare say simply tweaking the settings outta be enough. There's no reason to excessively swap when you still have gigabytes upon gigabytes of memory. Waiting out a bit longer, and perhaps using windows of low memory access to dump large pages, that alone can go a long way.

Isn't that why the GPL was created in the first place -- to encourage and protect open source innovation?
Ok, so if open source is such a dirty, nasty, oppressive system....then please tell us O wise master, what/where exactly is this perfect system that we should all be using?

No, it is because due to its late entry, gpl had a lot of catching up to do if there's any hope actually catching up to its intended competitors. Therefore the decision to reduce the license to a plague that infects and drags everything in it. It was about building up inventory, if you want a hand out, instead of money you surrender your IP... Which in many cases is btw far more precious....

This has done more for M$s monopoly more than what M$ has done for it directly. The result of the gplague was that the smaller, not quite as rich makers of end user software stayed away from a M$ alternative, after there was one. Because if you spill your trade secrets you can pretty much forget about monetizing that stuff, it will be stolen right away, and good luck protecting your IP without big tech's army of highly paid lawyers and donation eager legislators. So no end user software - linux dominates everything that makes money yet is still anemic in end the user market. By the time foss came up with more permissive licenses, the damage was already done, big tech runs the world. So thanks for nothing.

Big corporations however have the resources to protect their IP, plus they cared entirely about contributing things that they need, and most people are not big tech corporations, nor do they have their needs. It doesn't in any way put regular people more on parity with the industry - all it did was to facilitate the industry's growth, and boy did it grow invasive and parasitic.

But hey, at least one day, when the terminators come for us, they will probably run foss. So yey!

[intelfx]

TMO performs memory offloading to swap at subliminal memory pressure levels, with turnover proportional to file cache. This contrasts with the historical behavior of swapping as an emergency overflow under severe memory pressure.

This is a completely incorrect summary.

Take a read: https://chrisdown.name/2018/01/02/in...e-of-swap.html

[erniv2]

This is a completely incorrect summary.

Take a read: https://chrisdown.name/2018/01/02/in...e-of-swap.html

Transparent memory offloading: more memory at a fraction of the cost and power - Engineering at Meta (fb.com)

Under the Solution Point 3

It´s what the Blog Post says not me.

[intelfx]

It´s what the Blog Post says not me.

The blog post is wrong then. Perhaps they needed to justify the effort :-)
(I'm not saying that Facebook did useless work, but this specific point is just factually wrong.)

[macemoneta]

If that's the effect that Meta wants, why not set /proc/sys/vm/swappiness to 100?

[macemoneta]

The blog post is wrong then. Perhaps they needed to justify the effort :-)
(I'm not saying that Facebook did useless work, but this specific point is just factually wrong.)

It sounds like Meta set swappiness to zero (emergency overflow under severe memory pressure), then wrote a kernel patch to 'fix' it.

[sinepgib]

By the time foss came up with more permissive licenses, the damage was already done, big tech runs the world. So thanks for nothing.

BSD license predates the GPL.

https://en.wikipedia.org/wiki/BSD_licenses

https://en.wikipedia.org/wiki/GNU_General_Public_License