On-Disk Format Changes Ahead To Improve "Painful" Parts Of Btrfs Design

[mdedetrich]

I wonder if they can bundle in the on disk changes required to remove the write hole RAID 5/6?

[mdedetrich]

I still don't understand why people consciously use btrfs and zfs instead of tried and proven LVM + ext4. Yes, snapshots, in my 30 years of computing I've needed them ... 0 times.

I cannot recommended xfs until they implement partition shrinking.

tl;dr (and this isn't completely technically correct but illustrates the point), btrfs/zfs treat your filesystem like an atomic database such as postgres that rather than mutating a "row" on data changes (where every row is represents a block of a file) they instead add another row with the changes. Ontop of this they are store checksums of every file in multiple places to make sure that data doesn't silently get corrupted. This gives you all of the advantages that a database like postgres does, i.e. security of your data.

[Old Grouch]

LVM gives you snapshots by the way, I use them to make consistent backups with rsync.

Snapshots DO NOT guarantee consistent backups. You might have a consistent filesystem; but filesystem snapshots say precisely nothing about the consistency or otherwise of the contents of files where an application uses multiple files simultaneously, as a filesystem has no idea if a write to file B needs to be treated as part of an atomic transaction with a write to file A - that is up to the programmer.

Let's say you add a new storage device to a system. You update fstab. You take a snapshot. You update crypttab to use the new device. Something happens, and you need to restore from the snapshot. Suddenly, your restored crypttab is referring to an old device. Oops.

Absolutely obvious when it is made clear with the above example. How could anyone be so stupid. But applications open multiple files all the time, and unless you can guarantee all your transactions are atomic, filesystem snapshots do not guarantee file content consistency.

One of the reasons (SQL) databases are popular is that you can (in principle) ensure all transactions are atomic. You just have to hope the programmers have identified all the transactions correctly (hint: it is hard), so the only way you can be reasonably sure you have captured a nice backup is by shutting down all the applications cleanly*, then the system, booting into an OS purely for taking a backup, take your backup and restart. Anything else is relying on other people doing their jobs properly all the time. Years of experience show that is a fool's game.

Don't get me wrong. Taking a snapshot of a well defined and implemented application can work - you just have to be aware of possible gotchas. There are one or two.

*Cacheing strategies on 'non-volatile' storage hardware can also ruin your day. Storage devices can and do lie about whether data has been committed to non-volatile storage or not. Data that you think is on spinning rust can in fact be in an all-too-volatile on-device RAM cache.

[sb56637]

How would this on-disk format change actually happen in practice? Would it require manual intervention by the user? (I understand that it will need to be explicitly enabled via mount flags at first, but I'm referring to once it passes the experimental phase.) And what happens if the system is running Btrfs on / and it gets booted with a newer kernel that includes this new Btrfs version? Would it 1) automatically and irreversibly convert to the new format and mount or 2) refuse to mount and thus not allow the OS to boot, requiring booting a rescue system to convert the format manually?

Another question: Are these patches just proposed, or have they already been merged?

[timofonic]

Unlikely. The people who hold the purse only care about RAID 10.

Those are very very bad news...

Seriously, I distrust Btrfs. It seems only focused on Facebook/Meta interests, rather than a more general solution.

[billyswong]

*Cacheing strategies on 'non-volatile' storage hardware can also ruin your day. Storage devices can and do lie about whether data has been committed to non-volatile storage or not. Data that you think is on spinning rust can in fact be in an all-too-volatile on-device RAM cache.

This was my experience! Two years ago I upgraded my PC and started using SSD as my main drive. Once in a while, the filesystem would be found corrupted and require me a fsck and then reboot. This year I shared my issue here in this forum and someone suggested me to disble write-back cache. Problem solved.

Hearing from the bad experience of people losing almost everything after power failure of btrfs drives / arrays, I am quite sure if I had picked btrfs, what they experienced would be my experience. Btrfs put too much trust into hardware conforming the specification and lack resilience in this kind of corruption. Meanwhile ext4 is more veteran and reliable in this regard. Meanwhile, in btrfs, "repairing" can become "erasing". And fanboys will blame the victim if the victim don't have spare drives and spare computers to do offline rescue.

[kreijack]

This was my experience! Two years ago I upgraded my PC and started using SSD as my main drive. Once in a while, the filesystem would be found corrupted and require me a fsck and then reboot. This year I shared my issue here in this forum and someone suggested me to disble write-back cache. Problem solved.

Hearing from the bad experience of people losing almost everything after power failure of btrfs drives / arrays, I am quite sure if I had picked btrfs, what they experienced would be my experience. Btrfs put too much trust into hardware conforming the specification and lack resilience in this kind of corruption. Meanwhile ext4 is more veteran and reliable in this regard. Meanwhile, in btrfs, "repairing" can become "erasing". And fanboys will blame the victim if the victim don't have spare drives and spare computers to do offline rescue.

I think that you are seeing the problem from the wrong point of view. In your case if you had used btrfs likely:
- BTRFS would told exactly which file were compromised
- BTRFS would warned very early that you SSD had problem (not after 1 year)

[useless]

Meanwhile ext4 is more veteran and reliable in this regard.

But it gives you no guarantee that your data is safe. fsck.ext4 checks filesystem consistency, it doesn't look for data corruption. Disabling the disk's write cache won't undo whatever changed in your data because of missed writes. btrfs, OTOH will refuse to mount or it will throw you an EIO.

lack resilience in this kind of corruption.

It would be great indeed if it survived at least a couple of missed transactions (it actually kind of does, see backuproots, but no guarantee). However, how much history should the filesystem keep to prevent missed writes? Performance/space constrains wouldn't make it feasibly (there was a discussion in the btrfs mailing list about this a while ago).

Btrfs put too much trust into hardware conforming the specification

You have to, to some extent. If the hardware doesn't conform to spec then whine about that (refuse to mount) or work around it (disable write cache). Either way, you won't get silent corruption; can't say the same in ext4 case.

[pal666]

If the ondisk format changes, why isn't btrfs still experimental?

because it will support both formats

[pal666]

This work (well, not specifically this work, but the whole "extent tree v2" effort) is basically destroying the single killer feature of btrfs, that is, the ability to relocate extents at will (i. e. the btrfs balance operation, which leads into RAID reshape/restripe).

what makes you think so?