New MGLRU Linux Patches Look To Improve The Scalability Of Global Reclaim

[NobodyXu]

Mitch I believe you are looking for madvide https://man7.org/linux/man-pages/man2/madvise.2.html

It can be used to info kernel of the expected mem access pattern, or use MADV_DONTNEED to lazily free up the backing memory for a virtual mapping.

There's also MADV_FREE that performs similar task but can be only applied to pruvate annoymous pages.

And there're MADV_COLD and MADV_PAGEOUT to encourage kernel to swap out pages.

[coder]

Mitch I believe you are looking for madvide https://man7.org/linux/man-pages/man2/madvise.2.html

It can be used to info kernel of the expected mem access pattern, or use MADV_DONTNEED to lazily free up the backing memory for a virtual mapping.

Does this apply only to mmap()'d memory, or are there other use cases, besides that?

[NobodyXu]

Does this apply only to mmap()'d memory, or are there other use cases, besides that?

AFAIK it can be only applied to memory mappings, but isn't everything in the virtual memory part of the memory mapping?

Loading the elf is essentially mmap, the vdso also loaded similar to a dynlib which is mmap.
The sbrk syscall managed heap is also kind of memory mapping.

[binarybanana]

I know a lot of people say they don't need any swap,

Those people are crazy and deserve all the pain they get.

[coder]

AFAIK it can be only applied to memory mappings, but isn't everything in the virtual memory part of the memory mapping?

Well, getting back to Mitch 's original point, there are some notable instances of application-level caching that's not page-based. Examples would include web browsers caching uncompressed images, games caching assets, and databases caching tables and indexes.

So, if there were a feedback mechanism for the OS to tell the app that it's under memory pressure, then the app could evict some of its cached objects to free up RAM. That would be a lot more efficient than if the OS swaps out some of these assets, and the app then has to page them back in (as compared with possibly re-reading a more compact representation from persistent storage).

[NobodyXu]

Well, getting back to Mitch 's original point, there are some notable instances of application-level caching that's not page-based. Examples would include web browsers caching uncompressed images, games caching assets, and databases caching tables and indexes.

How is that not page-based?

The memory backing the caching is either allocated using sbrk or mmap, both of them returns pages.

You might have confused a few concept here.
The paging I talk about is from the virtual memory standpoint.

In the virtual memory, the mapping between virtual memory and physical memory is described using page.

It doesn't matter whether the cache in your web browser/database is based on pages or not, as long as they use sbrk/mmap or other ways to modify the virtual memory it's always backed by pages and can just use madvice.

In fact, I think database already uses madvice, to explicitly allocate HUGE PAGE (since THP usually hurts database performance), and some memory allocator (I think it's mimalloc from microsoft) uses mimalloc to free up the memory without unmap the virtual memory since changing the page table is very expensive and might even block all threads in the process.