Btrfs RAID 0/1/5/6/10 Benchmarks On Linux 4.12

[pcxmac]

Because without checksums then how can you be sure that parity isn't corrupted?
In its RAID1 everything is checksummed for example, not just one of the two copies.

yeah, it's better to use RAID 1/0 or 10 w/ btrfs and backup religiously. You can even create your own virtual disk on top of a ZFS pool and send incremental snapshots from btrfs to the virtual disk. BTRFS is a lot, A LOT more easier to work with, and if you have a solid back up strategy, it's very effective.

[starshipeleven]

Are you sure the disk-stripes are checksumed? Doesn't sound reasonable, cause this will be already a layer-break.

No, what I was saying was that the code checking for checksums when reading parity wasn't working or is still very new (a patch supposed to fix that landed in 4.12, at least for btrfs scrub), not that they need to add checksums on block layer.

[PuckPoltergeist]

No, what I was saying was that the code checking for checksums when reading parity wasn't working or is still very new (a patch supposed to fix that landed in 4.12, at least for btrfs scrub), not that they need to add checksums on block layer.

This doesn't make sense. The FS layer, that does the checksumming, doesn't care about RAID levels. It only cares about redundancy in case of checksum mismatch. If a mismatch happens, we try to get the data from some other place. For RAID1 this means some copy, for RAID5 it reconstructs the stripe from data + parity.

As far as I remember from the discussions on btrfs mailinglist, the idea was checksumming the parity too, additional to the checksums of data/metadata. And this doesn't make much sense.

[starshipeleven]

This doesn't make sense. The FS layer, that does the checksumming, doesn't care about RAID levels. It only cares about redundancy in case of checksum mismatch. If a mismatch happens, we try to get the data from some other place. For RAID1 this means some copy, for RAID5 it reconstructs the stripe from data + parity.

As far as I remember from the discussions on btrfs mailinglist, the idea was checksumming the parity too, additional to the checksums of data/metadata. And this doesn't make much sense.

since I can't seem to explain myself decently, see here for the fixes I was talking about: https://www.mail-archive.com/linux-b.../msg63365.html

Without these RAID5/6 is basically retarded bullshit, and for now it's still too early to say if it's all clear.

[PuckPoltergeist]

since I can't seem to explain myself decently, see here for the fixes I was talking about: https://www.mail-archive.com/linux-b.../msg63365.html

Without these RAID5/6 is basically retarded bullshit, and for now it's still too early to say if it's all clear.

Thanx for the link, will read it (and the patches, Dave is mentioning) and repost tomorrow. Too late for me for an analysis and technical response now.

[starshipeleven]

And raid5/6 scrub patches floating around the mailing list get updated again https://www.mail-archive.com/linux-b.../msg64474.html and it explains in more understandable english what scrub can or cannot do atm.

[kreijack]

That if parity isn't checksummed then the guarantee that my data isn't corrupted goes out of the window.

This is not correct: you still have the data checksum to check if the rebuild data is correct. So even in the case that the parity is corrupted, btrfs is able to detect it.

[kreijack]

And raid5/6 scrub patches floating around the mailing list get updated again https://www.mail-archive.com/linux-b.../msg64474.html and it explains in more understandable english what scrub can or cannot do atm.

These patches are related to the user space implementation. As described in the first email, the scrub user-space implementation is not related to the kernel space one:

> To get a comparable tool for kernel scrub, we need a user-space tool to
> act as benchmark to compare their different behaviors.

The normal user should relay to the kernel implementation, which, unfortunately, is far to be perfect...

[starshipeleven]

This is not correct: you still have the data checksum to check if the rebuild data is correct. So even in the case that the parity is corrupted, btrfs is able to detect it.

Same answer as to the other guy above, the code to check checksums in case of RAID5/6 was failing to check parity properly, and it's still too soon to say it's all clear, see the other link I gave him.

[starshipeleven]

These patches are related to the user space implementation. As described in the first email, the scrub user-space implementation is not related to the kernel space one:

Yeah, I know how to read mail subjects, if I see "btrfs-progs" I know it's talking about the userspace tools.

It makes a comparison between btrfs-progs and the kernel's capability, and does so in english with easy ascii graphs, that's why I posted it.

The normal user should relay to the kernel implementation, which, unfortunately, is far to be perfect...

One of the reasons ext4 is still very liked is the quality of is userspace tools, I don't mind having decent fixing ability in btrfs-progs too.