Btrfs For Linux 5.1 Brings Configurable Zstd Compression Level, A Number Of Fixes

[phoronix]

Phoronix: Btrfs For Linux 5.1 Brings Configurable Zstd Compression Level, A Number Of Fixes

The initial feature updates were sent in a short time ago for the Btrfs file-system changes targeting the Linux 5.1 kernel cycle...

http://www.phoronix.com/scan.php?page=news_item&px=Linux-5.1-Btrfs-Changes

[S.Pam]

Cool. It's quite useful with zstd variable compression. For example backups could be done with highest setting while an active HDD or SSD are better suited for lower settings.

Really nice to see all the improvements that the Btrfs team is bringing.

[Chugworth]

Is anyone working on the RAID5/6 problem? Or is that just the elephant in the room that no one wants to address?

[jacob]

Is anyone working on the RAID5/6 problem? Or is that just the elephant in the room that no one wants to address?

More like the the corner case that noone really cares about. Users with real valuable data use RAID10, the majority of the rest doesn't bother with RAID at all. RAID5/6 is for hobbyists and perhaps the odd small business that doesn't have everything hosted in the cloud. At any rate, such users don't seem prepared to actually do anything to fix it, or to pay for someone to fix it.

[ultimA]

Users with real valuable data use RAID10... RAID5/6 is for hobbyists and perhaps the odd small business...

Since when does RAID10 provide greater data redundancy/guarantees than RAID6? Not to mention how inefficient it scales with the number of discs. I'm fine with a claim stating there is no current interest in paying for RAID6 fixes for whatever reason (I don't have data on that myself), but claiming that RAID10 is more professional for some reason? How? Why?

[caligula]

claiming that RAID10 is more professional for some reason? How? Why?

I heard Facebook uses RAID10. They don't need large capacity in this use case.

[S.Pam]

Raid 5 is slow with a missing disk. Mirroring is much essier and better that way .

[useless]

Is anyone working on the RAID5/6 problem? Or is that just the elephant in the room that no one wants to address?

There was a discussion in the mailing list a few months ago. Essentially, what they said was that there are more important things to do before, so some features were pushed forward. Parity code was going to be reworked, probably including a intent journal of some kind. Patches were proposed, but still...

However, parity RAID is almost never used in production. RAID5/6 is poor's man setup and write speed is worst (compared to RAID10). Though, I would really like some progress on RAID6 because, lets be honest, I'm targeting a poor's man setup.

[jacob]

Since when does RAID10 provide greater data redundancy/guarantees than RAID6? Not to mention how inefficient it scales with the number of discs. I'm fine with a claim stating there is no current interest in paying for RAID6 fixes for whatever reason (I don't have data on that myself), but claiming that RAID10 is more professional for some reason? How? Why?

Firstly it's important to remember that Btrfs RAID is not the same as standard RAID. Standard RAID applies to entire disks, Btrfs RAID applies to individual files. If you have N disks, Btrfs RAID10 can recover your files in the event of a failure of anywhere between 1 and N-1 disks, depending on the circumstances. RAID6 can survive the failure of 2 disks, no more and no less, in all circumstances. With RAID10 disk failures are basically transparent, Btrfs just keeps going and will repair itself the moment you plug in a new disk. With RAID6 you need to rebuild the array, which takes the system offline for a while and also it increases the risk of a cascade failure.

I'm certainly not saying that RAID6 is not worthy of attention. It's actually often a very good compromise between safety, performance and cost. RAID10 is clearly expensive, even more so if you want to achieve a higher level of redundancy, so obviously it's not for everyone. But those who develop Btrfs and/or who pay for its development don't seem to fall into the RAID6 use cases, and those who do have never decided to do anything beyond complaining on forums.

[useless]

Standard RAID applies to entire disks, Btrfs RAID applies to individual files.

Isn't duplication applied at the chunk level?