Bcachefs Prepares Last Minute Fixes For Linux 6.7

[ferry]

It would also be great if btrfs some day could do what ever it is that zfs is doing when detecting that something like a sql server is running on it so COW+checksumming can be still enabled without loosing performance.

Also I've been thinking about a delayed Raid c2, aka since one most likely are always using two similar drives from the same batch and with the same characteristics there is a huge chance that both will die quite close in time so I think there could be a benefit from a c2 type of raid 1 where the duplication is delayed (and for SSD:s also batched/deduped) so the drives would experience a large difference in the number of writes.

I like btrfs a lot, but it would be great if they would finally just implement what was more or less promised many years ago: hot data relocation https://lwn.net/Articles/400029/
This would automatically store hot data on ssd and move the rest to hdd. I understand this is not a necessity for corporate use, but for home / soho use this would be a big plus.

[guzz46]

I've been using it for about 10 days now, and so far so good, file copy speed seems to be roughly on par with xfs, it does take up more hard drive space, and it does use more ram, but its handy having snapshots, I've got it on root. /home, and two 2tb sata SSD's which are effectively in raid 0, which I use for pictures videos, and steam games, I had to create a systemd servcice to mount the sata SSD's on boot due to some sort of systemd issue I think? where adding multiple devices in fstab wouldn't work.

[Mark Rose]

Given that it's still buggy and has a lot to be done, I doubt that is the case and that will happen.
Not to mention the benchmark posted on this site shows that Bcachefs still lags behind Btrfs in a lot area.

Home users who prefer Btrfs over Ext4 are people who care about data integrity and likely also use snapshot, so them all deserting Btrfs so quickly after Bcachefs is unlikely.

The lack or send/recv is a killer for any home user who use Btrfs for NAS or have to habit of backing up their system, so it's unlikely any serious user of Btrfs will desert Btrfs and switch to Bcachefs until send/recv is ready.

I do local and offsite backups using btrfs send/receive. Btrbk makes it easy. It's faster than rsync.

[flower]

The lack or send/recv is a killer for any home user who use Btrfs for NAS or have to habit of backing up their system, so it's unlikely any serious user of Btrfs will desert Btrfs and switch to Bcachefs until send/recv is ready.

IMHO not really. I use zfs but I dont use send/recv. I just prefer borgbackup and explictly want a different fs for my backups.

bugs can and will happen. also fs bugs. to me it just feels safer to let my backup tool handle checksumming.

what i want (and btrfs lacks) is encryption and raidz.

also vm and db performance was horrible the last time i tested btrfs (yes i know, i could disable cow. but well... i don't want to, because i want to use cow features. zfs just lets me do that without getting slow) i am really curious how bcachefs will handle vm's and db's after a year of heavy usage.

[Quackdoc]

the new kernel cant come soon enough, cant wait to have a FS that will actually reliably work for me, data dedupe is the major feature i am looking forwards to, but ofc the other cow stuff will be nice, most importantly ive heard great things about its reliability

[pWe00Iri3e7Z9lHOX2Qx]

I am biased towards btrfs and i would like to challenge the btrfs haters to come up with whats actually wrong with btrfs.

Yes it is slow for some things, "raid 5/6" should never have been added so early and it sucks that per subvolume data/metadata profiles are not yet possible. But that's is actually all luxury problems.

Challenge accepted!

"Raid" 0/1/c2/c3 and 10 work great and do compare what your get (for free) with any other filesystem and see if you can beat features/reliability.

No, RAID10 certainly does not "work great". RAID0 or RAID1 are probably fine, but I typically only use btrfs on single disk roots, where it's excellent with snapper / grub integration on OpenSUSE. But RAID10 is busted by design. The write strategy for the chunks can / does reorder your disks. Each chunk written is like a "mini" RAID10. Your data is guaranteed to be written to two devices. But because btrfs is all about flexibility and maximizing available storage, some basic things like "RAID10" don't mean what 99% of users would expect. When any 2nd disk fails in a traditional 4 disk RAID10, you should have a ~66% chance of not losing any data. When any 2nd disk fails in a btrfs "RAID10", you are basically guaranteed to lose data because of the chunk write strategy.

It's not hard to see why so many people are excited about bcachefs when the other in-tree COW filesystem has had such bad RAID5/6/10 support. All 3 of those modes are excellent in ZFS, but then you get the pain in the ass of an out-of-tree filesystem.

I'd love for all 3 of these things to happen, we'll see which ones pan out.

1. Larry Ellison promises that Oracle will never sue anyone for ZFS and it ends up being in-tree (yeah right!).
2. BTRFS either fixes, or better yet renames some of their existing "RAID" modes and introduces solutions that work as expected and are highly performant for RAID 5/6/10 (could happen).
3. Bcachefs matures and offers all of the things I find really important from ZFS (could happen).

If anyone thinks I'm making this up, see https://lore.kernel.org/linux-btrfs/...hesusis.net/T/ (waxhead appears to be there so they already know this) and the disk reordering bit was even mentioned in the old btrfs documentation.

[F.Ultra]

you shoudn't do that in first place, whatever the fs you're running on it.

yeah good luck with that idea when you prep a few thousand servers per batch, it's basically unavoidable. But even if you would, you would still have two SSD:s with similar DWPD, and if not why run one with a lower DWPD just to have it fail sooner?

[NobodyXu]

also vm and db performance was horrible the last time i tested btrfs (yes i know, i could disable cow. but well... i don't want to, because i want to use cow features. zfs just lets me do that without getting slow) i am really curious how bcachefs will handle vm's and db's after a year of heavy usage.

IIRC bcachefs uses log structure for small writes to avoid the perf problem with frequent small writes into CoW fs.

I read Kent's blog long ago and I recalled it also uses a dict for mapping blocks, can't remember how it contribute to perf though.

[varikonniemi]

people are going to be amazed about the reliability of bcachefs, i see many skeptics on this forum. What they forget is that this has been in production use for more than 5 years. And development is only accelerating.

It would be interesting to put up a betting pool about how long it takes before someone experiences&finds a data loss bug after stable release.

[waxhead]

Challenge accepted!

No, RAID10 certainly does not "work great". RAID0 or RAID1 are probably fine, but I typically only use btrfs on single disk roots, where it's excellent with snapper / grub integration on OpenSUSE. But RAID10 is busted by design. The write strategy for the chunks can / does reorder your disks. Each chunk written is like a "mini" RAID10. Your data is guaranteed to be written to two devices. But because btrfs is all about flexibility and maximizing available storage, some basic things like "RAID10" don't mean what 99% of users would expect. When any 2nd disk fails in a traditional 4 disk RAID10, you should have a ~66% chance of not losing any data. When any 2nd disk fails in a btrfs "RAID10", you are basically guaranteed to lose data because of the chunk write strategy.

Well , first of all there is nothing wrong with BTRFS' "RAID"10 implementation. It works and is as reliable as RAID1 which is precisely what it is designed to be. And besides traditional RAID10 is also vulnerable to loss of a second disk if you happen to loose the "right" one.
Also BTRFS does document it's "RAID" redundancy here : https://btrfs.readthedocs.io/en/late....html#profiles
That being said - the problem is that BTRFS does use the RAID terminology when it in reality should have been called something else such a copies or as bcachefs does - replicas.

It's not hard to see why so many people are excited about bcachefs when the other in-tree COW filesystem has had such bad RAID5/6/10 support. All 3 of those modes are excellent in ZFS, but then you get the pain in the ass of an out-of-tree filesystem.

I'd love for all 3 of these things to happen, we'll see which ones pan out.

1. Larry Ellison promises that Oracle will never sue anyone for ZFS and it ends up being in-tree (yeah right!).
2. BTRFS either fixes, or better yet renames some of their existing "RAID" modes and introduces solutions that work as expected and are highly performant for RAID 5/6/10 (could happen).
3. Bcachefs matures and offers all of the things I find really important from ZFS (could happen).

If anyone thinks I'm making this up, see https://lore.kernel.org/linux-btrfs/...hesusis.net/T/ (waxhead appears to be there so they already know this) and the disk reordering bit was even mentioned in the old btrfs documentation.

If I understand you correctly , and I think i do - I really think is it wrong of you to claim that BTRFS has "bad" RAID5/6/10 support.
The only think you are "making up" is that BTRFS is broken which it in my humble opinion is not.
"RAID"10 is fine, - it is NOT BROKEN - it is just very badly named.
"RAID"5/6 is admittedly far from great, but it is neither supported or recommended either. There is even several ugly warnings if you attempt to use it.

I agree with your other points, but to summarize :
* "RAID" in BTRFS terms are not RAID!!!
* "RAID" 0/1/1C3/1C4/10 on BTRFS are NOT broken and works as designed and actually can have benefits (in addition to drawbacks).
* "RAID" 5/6 is not supported nor recommended (write hole)
* The RAID_STRIPE_TREE recently introduced in BTRFS should eventually fix the write hole.
* The extent tree v2 should fix some of the drawbacks of BTRFS current design, but will introduce new quirks.

I also have big hopes for BCacheFS, but quite honestly I think it has a way to go before it is on par with BTRFS' as far as reliability goes , and there probably will be need for some redesign on BCacheFS as well sooner or later. For now I am all about BTRFS, but in 5-10 years? who knows!