Btrfs Gets More RAID 5/6 Fixes In Linux 4.16

[starshipeleven]

bcachefs is where my moneys at - it's just extremely well-thought out from the get go.

Could you explain better or link to some sources on this?

I'm not keeping tabs on bcachefs but I'm still interested in it.

[Niarbeht]

I use ZFS currently for the stuff I Really Care About (TM), but it'd be nice to have a working, in-tree, feature-parity (or close enough) replacement. Publishing block devices from ZFS is freakin' sweet.

[DrYak]

Making such a thing into a daemon that calls `btrfs snapshot` shouldn't be too difficult, right?

`cp --reflink` and `btrfs snapshot` use the same technology under the hood : copy-on-write.

It's the granularity that is different :
- reflinks work at the scale of single files
- snapshots work at the scale of whole subvolumes.

If you want to track the changes and versions of a giant tree (like GIT does with tree diffs/patches in the SCM-world), you need snapshots,
if you want to track the changes and edits on a single files (like CVS does with list of linear changes in the SCM-world), you need reflinks.

I believe the answer may be bcachefs.

And as of today, the current features available for testing in bcachefs are tiny subset of what is currently considered stable in BTRFS.

Don't get me wrong, I don't want to belittle the work of Kent. he's doing a great job.

But writing a feature-ful filesystem is an extremely complex and long time-consuming task. There's still a lot of work to go.
(Replication (RAID1), Snapshots, Send Received aren't there yet ; Erasure Coding (RAID5/6) has even started
And those are also the difficult parts).

[duby229]

Yeah, let's interrupt something when it's like 70% done and start anew just because we are impatient.

Of course, because that's where it's already proven to be perpetually stuck.

[caligula]

you can continue, raid1 is faster anyway

Faster at rebuilding perhaps? OTOH raid5/6 has significantly faster RW speed. You can easily achieve throughput such as 500 MB/s with spinning disks and RAID5. It's because you can do independent reads and writes of different blocks using multiple disks. On RAID-1 read speed has a theoretical max of 2x disk speed (assuming the file system can do checksums for blocks independently). On RAID-5 it's N-1, where N is the number of disks.

[waxhead]

Of course, because that's where it's already proven to be perpetually stuck.

Actually no... I have been following BTRFS for years now and while it does move slowly it is far from stuck. If you know how it works and stay away from experimental stuff it is very stable and have saved me from a corruption or two which other filesystems simply can't protect against (and the same goes for MD raid in some configurations).

[waxhead]

Faster at rebuilding perhaps? OTOH raid5/6 has significantly faster RW speed. You can easily achieve throughput such as 500 MB/s with spinning disks and RAID5. It's because you can do independent reads and writes of different blocks using multiple disks. On RAID-1 read speed has a theoretical max of 2x disk speed (assuming the file system can do checksums for blocks independently). On RAID-5 it's N-1, where N is the number of disks.

Just keep one thing in mind: do not mistake BTRFS RAID-1 for "RAID-1". It is not and the naming is a huge mistake. RAID-1 in BTRFS means two identical copies of data on two different devices regardless of how many disks you have. BTRFS reads these copies simply by using the last number of your process identified (PID) which wraps around (modulo) depending on how disks (mirrors) you got.
In fact a 10 disk BTRFS "RAID"-1 can teoretically read from all 10 disks at once providing that enough processes read at the same time. BTRFS "RAID"-10 can also read from all 10 disks, but with stripe over 5 and 5 disks.

Usually even on MD-RAID two or more disks in RAID 1 will not increase read speed. For example one process reading 10k of data will have disk A read the first 5k and disk B needs to skip the first 5k and then read the remaining 5k. MD would still need to wait for disk B to skip the 5k and the total read speed will therefore NOT be increased. If multiple processes reads from the array at the same time things may speed significantly up as MD will roughly behave like BTRFS and try to read all over the array.
(look up: man md - go to the RAID1 section)

"RAID5/6" on BTRFS should fly , I just hope it will be possible to set stripe width at some point. For now BTRFS does a stripe width equal to all disks in the array.

[caligula]

In fact a 10 disk BTRFS "RAID"-1 can teoretically read from all 10 disks at once providing that enough processes read at the same time. BTRFS "RAID"-10 can also read from all 10 disks, but with stripe over 5 and 5 disks.

Having 10 disks in RAID-1 is just dumb.

on MD-RAID two or more disks in RAID 1 will not increase read speed. For example one process reading 10k of data will have disk A read the first 5k and disk B needs to skip the first 5k and then read the remaining 5k. MD would still need to wait for disk B to skip the 5k and the total read speed will therefore NOT be increased. If multiple processes reads from the array at the same time things may speed significantly up as MD will roughly behave like BTRFS and try to read all over the array.
(look up: man md - go to the RAID1 section)

Well, MD-RAID is a dumb RAID. AFAIK it has no access to filesystem level metadata and checksums. With Btrfs you can do "smart" RAID. You can even double sequential reads assuming the application can communicate the need for "readahead" without wasting perf on small intermediate buffers. The big difference is, with Btrfs you can check the integrity of data after a single read from a single disk. If the data differs from stuff on the other disk, this can be done in the background as a separate delayed job or even as late as in scrub / only when a read fails. So it performs basically like RAID-0, perf-wise.

[pal666]

Care to put some context around that statement?

In what sense is it not stable (apart from being out-of-tree)

well, that is the main part. stable is solaris zfs. linux zfs is garbage, tainting your kernel. stability is reached by active usage and usage of outoftree garbage is not registering on radar.

- it's pretty much at feature parity with OpenZFS on BSD and Illumos,
has far exceeded the feature set of Oracle ZFS.

since when more features mean stability?

My experience with ZoL has been far from unstable.

there are whole enterprise linux distros with btrfs by default, so obviously your zfs experience has zero weight

I honestly wouldn't trust my data on any other filesystem at this moment in time, but in the future I don't believe btrfs will win the battle of Linux filesystems, bcachefs is where my moneys at - it's just extremely well-thought out from the get go.

lol. i can't take seriously people who believe in bullshit advertisements and unsupported outoftree crap. your money would be safer on santa

[pal666]

¯\_(ツ)_/¯ The BTRFS ship has sailed. This sentence is telling: Nice - good to know that it is going to "try" harder. Kent Overstreet's assessment is spot on:

so in what millenium will your sect start actually developing filesystem instead of patreon advertisements?