On-Disk Format Changes Ahead To Improve "Painful" Parts Of Btrfs Design

[JustK]

@Thread: Why not combine the planned on-disk format changes with patches that fix raid5/6 and be done with it? Otherwise fixing raid will need another format change…

If we're breaking backwards-compatibility anyway, let's drop the inferior compression algorithms zlib and lzo, while adding lz4 and bringing zstd up to date. Calling it btrfs2 (see ext2/3/4) is also an option, it even preserves backwards-compatibility.

[NobodyXu]

If we're breaking backwards-compatibility anyway, let's drop the inferior compression algorithms zlib and lzo, while adding lz4 and bringing zstd up to date. Calling it btrfs2 (see ext2/3/4) is also an option, it even preserves backwards-compatibility.

I think Btrfs uses zstd built in the linux kernel, so once the linux kernel updates its zstd, Btrfs will also able to enjoy newer zstd implementation.

Edit:

It has already been updated! https://www.phoronix.com/scan.php?pa...For-Linux-5.16

[Jannik2099]

@Thread: Why not combine the planned on-disk format changes with patches that fix raid5/6 and be done with it? Otherwise fixing raid will need another format change…

extent tree v2 is meant to fix the parity raid issues and bring encryption support, amongst other things. I recommend you take a look at https://github.com/btrfs/btrfs-todo/issues/25

[King InuYasha]

Honestly I, as a user, don't like where this is going.

This work (well, not specifically this work, but the whole "extent tree v2" effort) is basically destroying the single killer feature of btrfs, that is, the ability to relocate extents at will (i. e. the btrfs balance operation, which leads into RAID reshape/restripe).

When this is done, there will be no practical reason to prefer btrfs to zfs.

This won't actually change that. This just pulls out the block groups structure into its own extent tree, and btrfs balance will still do the right thing here.

[Chugworth]

It's interesting that in the blog post, one of the problems they're trying to resolve is long mount times on "very large file systems". Why not fix the RAID bug first so that we could actually have "very large file systems"?

[waxhead]

Honestly I, as a user, don't like where this is going.

This work (well, not specifically this work, but the whole "extent tree v2" effort) is basically destroying the single killer feature of btrfs, that is, the ability to relocate extents at will (i. e. the btrfs balance operation, which leads into RAID reshape/restripe).

When this is done, there will be no practical reason to prefer btrfs to zfs.

This is not correct. I have read about Extent tree v2, and watches Josef's videos about it on youtube. The only drawback from this change is that scrub will be less linear e.g. more random. Personally I do not like that Scrub will be causing more trashing of the disk heads on spinning media which eventually will cause scrub to take a little bit longer. On the other hand , scrub is not something that you run that often either so it is not a major drawback.

What i really do NOT like as much is that this change also introduce a garbage collectors (I hate garbage collectors). Luckily Josef explained that in his videos and I am not sure I fully understand how it was meant to work , but the garbage collector seems to be designed in such a way that it does not risk stalling the system. It will essentially take a bite here and a bite there gradually nibbling its's way through the garbage, so perhaps not as bad as you may think.

What is less clear is if this extent tree v2 thing will somehow positively or negatively affect BTRFS abillity to repair corruption. Not sure if the system will be more or less robust as far as (major) data corruption is concerned with this change. All in all the changes seem to be good.

And for those advocating for btrfs2 : That is a bad idea. This is not a remake of the filesystem, this is evolving the system gradually. Even with the patches that are fairly fresh - btrfs passes (almost) all tests in the test suite. I am sure that the extent tree v2 will be marked as experimental for a while though.

[timofonic]

Are they going to fix RAID issues and surpass RAID-Z approach too? Any hopes?

[zexelon]

I was seriously considering moving my companies production systems to btrfs as it was finally looking stable and some of its features could improve our processes... changing the on-disk format just sunk that option for us. This essentially moves btrfs back to square one wrt to stability and bugs, sure its likely to re-use a very large body of code already there, but you never know where some line made an assumption about the on-disk format and will now thrash your data.. and its likely to be an assumption the dev didn't even realize they were making.

We will be going to LVM2+xfs, basically gives you everything btrfs is trying to do, with about a similar level of complexity to maintain, but many more years of testing and abuse.

I wonder how Facebook's Tupperware team feels about this? Do they even still use Tupperware+btrfs? I heard they were moving everything over to Tectonic (thats a FS i really want to see!).

[avem]

I still don't understand why people consciously use btrfs and zfs instead of tried and proven LVM + ext4. Yes, snapshots, in my 30 years of computing I've needed them ... 0 times.

I cannot recommended xfs until they implement partition shrinking.

[mangeek]

This doesn't seem like too big of a deal, I'm sure it will end up being phased in as a capability before it becomes a default, and it will still be possible to create gen1 filesystems. There have been plenty of filesystem features added that break backwards compatibility since btrfs debuted, even if they didn't change the fundamental on-disk format.