Linux 5.10.8 Kernel Released - Finally Fixes That Btrfs Performance Regression

[Hi-Angel]

Are there any plans for any sort of CI for the kernel? It seems these kinds of things happen frequently enough that running some automated benchmarks (like you do, Michael ;-) ) before tagging releases would be reasonable. Surely any one of the big cloud players could easily donate the compute power for this through the Linux foundation, and scaffolding the whole thing should not demand that much considering how many tools exist for this these days. Could have a lot of neat features like bisecting to find the culpable commit. Or even the reverse, highlight commits with noticeable performance improvements.

There is some CI. If you look e.g. at 5.10.7 changelog, you'll see a number of patches with a line `Reported-by: syzbot*`. Although, I don't know if there is a CI for performance regressions specifically (because I guess this is what you were asking about), but on that matter I concur with piorunz in that the regression discussed was manifesting itself in quite specific conditions.

[geearf]

Guys you realize these regressions were invisible on testing environment & VM but visible only once hit bare metal machines only?

You're talking like bare metal is some strange and rare thing.
Sure it may be more problematic for CIs, but aren't devs able to test on bare metal on their own machines anymore?

Testing is being done, but it's not critical software

The fact that it is not a critical bug is no excuse, we're shocked by the pattern not the exact incident or the time it took to fix it.
Even then, I do disagree that a filesystem is not critical software, it very much is.

Anyway, there's no benefit in this discussion so I'll stop here.

[cynic]

You're talking like bare metal is some strange and rare thing.
Sure it may be more problematic for CIs, but aren't devs able to test on bare metal on their own machines anymore?

there's already a big suite of tests in place for filesystems developers and it is constatly used to avoid functional regressions.
That specific bug could pass unobserved even on physical machine: I used 5.10 since rc releases on btrfs and didn't noticed the bug until the report came out.

if you believe that kernel devs are stupid and release code without testing it, then go and show them how to do things properly, they'll surely thank you.

Also, if you can't bear with such regressions, use and enterprise distro and stop whining.

[piorunz]

there's already a big suite of tests in place for filesystems developers and it is constatly used to avoid functional regressions.
That specific bug could pass unobserved even on physical machine: I used 5.10 since rc releases on btrfs and didn't noticed the bug until the report came out.

if you believe that kernel devs are stupid and release code without testing it, then go and show them how to do things properly, they'll surely thank you.

Also, if you can't bear with such regressions, use and enterprise distro and stop whining.

That's right! I am thankful for devs work, people sitting in front of their machines with Linux on it didn't ever lifted a finger or write a line of code but are whining "no excuses for bugs".

[archkde]

And based on what Michael wrote, it's not like this regression needed a very specific system to find:

Well, it did. Not every system has a very fast NVMe SSD. You won't notice this regression when the bottleneck is the disk taking five minutes to write the uncompressed files.