Major Rewrite Of Linux's FS-Cache / CacheFiles So It's Smaller & Simpler

[tuxd3v]

Finally someone that knows how it works. Not used memory is wasted memory and the more of it that is used for cache the better...

If you want to be living in Swap, then its good idea, I prefer not having pages swapping out and in.
But hey, I understand your comment, it seems coming out of a Sales manager of a Memory producing company, like intel,samsung,micron,etc..

For me a knob to telling "what is the amount of pagecache that you want"...its the big feature that we already had and we are now missing..badly.

[down1]

The knob you want exists it is swappiness.

[tuxd3v]

No. If you use the command line tool "free" you will see that it has free memory and available memory. the important one is the available number and not the free one.

The "available" number, its just for show,
Because in some cases it doesn't work.. the retrieving of memory from cache is too slow, and if you are reading files at a high pace, your machine will start swapping before pages get retrieved and free from cache..

Free memory is just memory that is currently unused for anything. available is the memory that can be used/allocated by a program.

Exactly, I need a lot of this memory, because I don't want to be always in the swap scenario..

[tuxd3v]

The knob you want exists it is swappiness.

No I am not talking about swapiness, swapiness has a different purpose..

I am talking about the ability that, you yourself, have to tell the system... look..., you can "dance and flip around" with 20% of Total Ram( just an example.. ), its the maximum that ever will be assigned for pagecache, now..live with that..!!
Leaving all the rest of memory in the free pool.
In this way, newer pagecache will get written above the oldest pagecache in that 20%( example ) pool and nothing more..in other words, a buffer ring with x% of total Ram size..

The rest, I want to my own workloads( free pool ), and I want if free, so that I can launch at anytime what I want, without going to swap..

Each server will have its workloads, and so, those values can be defined by the administrator.
It should be a percentage number, like ex: 20% Ram for pagecache..

Then, if system Ram( free pool ) starts to be low, start to retrieve freeing up pagecache( from those 20% it had initially ), so that it will be available to applications..
In other words..
Do what you already do, but give the user a possibility to set the max amount of cache..

[cl333r]

posix is c api, so you are showing c code. you can't compare strings like that, you have to compare their contents.

I thought it's obvious that I provided meta code (or whatever it's called), not actual real C code. And the second part you also got it wrong.

[down1]

No I am not talking about swapiness, swapiness has a different purpose..

I am talking about the ability that, you yourself, have to tell the system... look..., you can "dance and flip around" with 20% of Total Ram( just an example.. ), its the maximum that ever will be assigned for pagecache, now..live with that..!!
Leaving all the rest of memory in the free pool.
In this way, newer pagecache will get written above the oldest pagecache in that 20%( example ) pool and nothing more..in other words, a buffer ring with x% of total Ram size..

The rest, I want to my own workloads( free pool ), and I want if free, so that I can launch at anytime what I want, without going to swap..

Each server will have its workloads, and so, those values can be defined by the administrator.
It should be a percentage number, like ex: 20% Ram for pagecache..

Then, if system Ram( free pool ) starts to be low, start to retrieve freeing up pagecache( from those 20% it had initially ), so that it will be available to applications..
In other words..
Do what you already do, but give the user a possibility to set the max amount of cache..

You can launch whatever you want and it will instantly write over the cache. If you want less swap turn swapiness down. The only case where I've ran into these problems is with zfs and that is because they don't integrate their cache with the kernel cache.

[waxhead]

If you want to be living in Swap, then its good idea, I prefer not having pages swapping out and in.
But hey, I understand your comment, it seems coming out of a Sales manager of a Memory producing company, like intel,samsung,micron,etc..

For me a knob to telling "what is the amount of pagecache that you want"...its the big feature that we already had and we are now missing..badly.

As other have already pointed out. Swapiness... for per. process swapiness you must look into systemd.resource-control and set MemorySwapMax (unless I am mistaking) and there are some other options to fiddle with as well. That ensures the best of two worlds.

PS! I am not a sales person at all. I have 4GB DDR2 memory in my main machine and are happy with it.

[tuxd3v]

You can launch whatever you want and it will instantly write over the cache. If you want less swap turn swapiness down. The only case where I've ran into these problems is with zfs and that is because they don't integrate their cache with the kernel cache.

I understand your concept, but I don't want less Swap, I mean, I want, but not the way things work now..
Yes you can retrieve pages from cache to use as memory for applications, but in a concurrent manner with pagecache itself, its slow,and will never bring great results.

That's why you have tons of cache, and still, you launch a small application that reads all files in your filesystem, and bam...you end up in Swap, like in the examples I showed to you in previous posts( because cache fills up the entire memory you have.. )..

• A less good solution( and simpler.. ) would be, stop caching immediately when your system really needs to retrieve pages from cache, in that way, you can free more pages since the page cache doesn't continue to grow..
• A better solution( maybe its harder to implement, I suspect...), would be to have a maximum amount of cache set, in that way, cache would never fill your entire memory( think of it as a buffer ring.. ). Also in that situation, even when you would need more memory( for applications.. ) you could try to retrieve it from cache as a last resort( like in the current situation.. )..providing that cache stops to grow under memory pressure..because if cache continues to grow under memory pressure puff, you will end up in swap no mater what( in case you exhausted your free pool.. )..But in this situation, the probability to swap would be dictated more by the administrator, than from the system itself( since you are the one who should set limits for cache and that will define the free pool too.. )..

In the examples I showed before,'vm.swappiness=1', and still systems go to swap..swappiness has a different meaning, its related but not the same..

[pal666]

I thought it's obvious that I provided meta code (or whatever it's called), not actual real C code. And the second part you also got it wrong.

there was nothing obvious about it and of course i was right on both parts. and it's called pseudo code

[cl333r]

there was nothing obvious about it and of course i was right on both parts. and it's called pseudo code

I'm not surprised it wasn't obvious for you.