Bcachefs File-System Pull Request Submitted For Linux 6.5

[Old Grouch]

It bothers me that the bcachefs snapshotting is less fully featured than that of NILFS2, and that that is unlikely to change. I use it as a substitute for file versioning (well known to users of VAX/VMS ODS). The ability to roll back to an arbitrary previous version of a generic file is remarkably useful.

[piorunz]

Filesystem is a last thing I want to experiment on in Linux. My storage data is not for gambling 😅 I may look at it few years from now, but currently I am perfectly comfortable with Btrfs on Raid, with checksums, and ECC RAM.

[christianb]

I doubt it will, I assume this will be punted back and forth more then a couple times. if it makes it in before 6.5 I would consider that extremely lucky

It's probably going to be longer than that, the way he's responding to comments:

> > (Worth noting the bug causing the most test failures by a wide margin is
> > actually an io_uring bug that causes random umount failures in shutdown
> > tests. Would be great to get that looked at, it doesn't just affect
> > bcachefs).
>
> Maybe if you had told someone about that it could get looked at?

I'm more likely to report bugs to people who have a history of being
responsive...

> Nacked-by: Christoph Hellwig <[email protected]>
>
> Kent,
>
> you really need to feed your prep patches through the maintainers
> instead of just starting fights everywhere. And you really need
> someone else than your vouch for the code in the form of a co-maintainer
> or reviewer.

If there's a patch that you think is at issue, then I invite you to
point it out. Just please deliver your explanation with more technical
precision and less vitriol - thanks.

[ihatemichael]

I am currently using btrfs but I've been contemplating going back to ext4, while the features of btrfs are nice (subvolumes, etc), I/O is pretty bad in some situations (running VMs, etc).

I hope bcachefs can provide the features of btrfs while providing better performance.

[Quackdoc]

It's probably going to be longer than that, the way he's responding to comments:

not sure I understand Hellwig's response, the response to Axboe not good though for sure.

[DrYak]

How does it compare to btrfs feature-wise and performance-wise?

BTRFS:
- has snapshotting that is production ready for quite some time. (simply by leveraging its copy-on-write nature and subvolumes).
(Meanwhile BCacheFS isn't there yet, but Kent keeps promising he'll work on it next, so don't despair).

BCacheFS:
- has a tiered storage system (I mean it started out of BCache, so obviously...) meaning that you can have a bunch of ultra fast NVMe SSD and a very large back storage of spinning rust HDDs.
(Doing the same would require running Btrfs on top of lvm-cache or BCache and that would lose some of its own storage management options).

Both:
- have checksumming and scrubbing. So, e.g., knowing which copy of a file is broken or which copy of the metadata is the correct safe one is trivial.
- are copy-on-right
- but do both of above, neither needs an fsck.
- support some form of redundancy/copies (RAID1) and stripping (RAID0)

Neither:
- support parity (RAID5/6, erasure coding). e.g. in BTRFS it's still considered unsafe, it's still possible to end up with an incoherent storage in some sudden power failure scenarios.

full disclosure: I have no idea how BTRFS handles metadata repair on a detailed level, so I can't comment there.

It's also fully checksummed and fully copy-on-write like ZFS (except for files flagged with special attributes).
So basically the same: you don't try to make sense of half borked data, you use the checksum to determine which copy of the copy-on-write is the correct one.

Though in the specific case of BTRFS, there exist some fsck tools (that where written for some compliance) but they are not the first choice to use in case of accidents.

perfectly comfortable with Btrfs on Raid, with checksums, and ECC RAM.

You means btrfs on top of mdadm RAID (I use that for RAID6 on my home server) or btrfs' own raid0 and raid1 (I use that on my laptops and my workstation)?

[Brisse]

I am currently using btrfs but I've been contemplating going back to ext4, while the features of btrfs are nice (subvolumes, etc), I/O is pretty bad in some situations (running VMs, etc).

I hope bcachefs can provide the features of btrfs while providing better performance.

Are you not using the nodatacow attribute on your VM-images, or is performance still bad even if you do?

[Barley9432]

It's probably going to be longer than that, the way he's responding to comments:

Not to mention he isn't able to come up with a co-maintainer as far as I'm aware. This filesystem is a one man operation, not sure how I feel about that tbh.

[piorunz]

You means btrfs on top of mdadm RAID (I use that for RAID6 on my home server) or btrfs' own raid0 and raid1 (I use that on my laptops and my workstation)?

Btrfs' own Raid1 and Raid10. I am fearing Raid5/6 is not stable so I am not using it.
Instead, one main PC with 2x SSD is on Btrfs Raid1, and another, home server, with 4xHDD is on Btrfs Raid10.

Main PC is regularly (like once a month) dropping one drive, probably I need to replace some SATA cable finally, instead of just re-seating them. Btrfs works like a charm, not losing one bit of data, and I can continue to work while one drive is dropped. After shutdown, fix and restart, I just scrub to find unsynchronized data and everything is 100% consistent again.
On home server, I replaced all 4 hard drives three times already, each time different drive sizes, upgrading from 1 TB to 2 TB and then to 4 TB drives this year. All replaced using Btrfs functions, while still online. Replacing one drive a day, because of massive amount of data. I had one failing disk as well, and Btrfs handled everything, including online migration away from that drive while avoiding access to the failing drive, because it would have caused massive slowdown and kernel timeout errors. All completed well in the end. All of that done online while filesystem was live and serving.

Not to mention ECC RAM has saved my bacon as well, more than once.
Modern computing and linuxing is amazing, I love it.

[ihatemichael]

Are you not using the nodatacow attribute on your VM-images, or is performance still bad even if you do?

I am not using that.