Linux 4.19 I/O Scheduler SSD Benchmarks With Kyber, BFQ, Deadline, CFQ

[geearf]

It's a problem in the virtual-memory subsystem. If you are curious, write to me and I'll send you a short document describing the causes of the problem. I have also made a solution, but never turned it into a production-quality kernel patch, for lack of time (and of funding from possible interested companies).

I think that would be a very amazing patch to have, so many of us are really annoyed with this.
It'd be great if you could get this out.

Thank you!

[paolo]

I'm still trying to figure out why my Linux servers hang when writing large amounts of data. I have been battling this for like 10+ years. Does not matter what scheduler I use. I've tried native board chipsets, external cards, different motherboards, everything, doesn't matter. If I write a large amount of data the whole system freezes up periodically. This is independent of the drive(s). A heavy write task to any drive periodically freezes the whole system.

This is especially problematic in virtual machines. There MUST be something I'm doing wrong because cloud hosts don't seem to exhibit this behavior (or maybe I don't notice).

So frustrating and this is not the right place to ask for help but I just throw this out everywhere hoping that someone, somewhere, knows what the problem is.

It's a problem in the virtual-memory subsystem. If you are curious, write to me and I'll send you a short document describing the causes of the problem. I have also made a solution, but never turned it into a production-quality kernel patch, for lack of time (and of funding from possible interested companies).

[paolo]

This. Asking the right questions. Does anyone have idea about a test for this to add to the benchmarking?

Already available, in several forms. For example, directly for the phoronix suite:

https://openbenchmarking.org/test/pts/startup-time

If you try it, and find issues, please ask for help on the bfq-iosched google group, I'll be more than happy to fix any problem for you. Or just ask even if you just want more information.

[FireBurn]

Well that was a waste of a click

[Vistaus]

Well, I've been using 4.19.0 for 18 days 24/7 and 4.19.2 for 14 days now without any corruption. The system has 3 x ext4 partitions and 5 x btrfs. Seems to work just fine..

Doesn't matter, if he says it's unstable, it's unstable ;-)

[Vistaus]

I call them "unstable" because of the EXT4 corruption issues, and because some people were reporting AMD troubles with 4.19 (yes, I know they are officially called "stable" but it doesn't feel like so).

I experience (a few) bugs in KDE Plasma 5.14, Deepin 15.18 and a few more pieces of software. Guess I should all call them "unstable" from now on.

[duby229]

So there is something important for readers to try to understand here and that is that IO performance is -VERY- much a perceptive experience. Winning benchmarks can give great insight into performance potentials and such, but they don't tell you how a user experience "feels"....

My advice is to benchmark -your own- typical workload and judge the user experience for yourself.

Anyway this was a great benchmark run.

[linner]

I'm still trying to figure out why my Linux servers hang when writing large amounts of data. I have been battling this for like 10+ years. Does not matter what scheduler I use. I've tried native board chipsets, external cards, different motherboards, everything, doesn't matter. If I write a large amount of data the whole system freezes up periodically. This is independent of the drive(s). A heavy write task to any drive periodically freezes the whole system.

This is especially problematic in virtual machines. There MUST be something I'm doing wrong because cloud hosts don't seem to exhibit this behavior (or maybe I don't notice).

So frustrating and this is not the right place to ask for help but I just throw this out everywhere hoping that someone, somewhere, knows what the problem is.

[starshipeleven]

Unless I'm mistaken this test is it does not tell us much about latency, which for many of us is more important than throughput.
I think the most important test is this: under heavy load, is the system still responsive enough? And lately that has not been the case on my system using BFQ which was supposed to be the whole point :/

This. Asking the right questions. Does anyone have idea about a test for this to add to the benchmarking?

[caligula]

I call them "unstable" because of the EXT4 corruption issues, and because some people were reporting AMD troubles with 4.19 (yes, I know they are officially called "stable" but it doesn't feel like so).

Well, I've been using 4.19.0 for 18 days 24/7 and 4.19.2 for 14 days now without any corruption. The system has 3 x ext4 partitions and 5 x btrfs. Seems to work just fine..