Btrfs To Ship Multiple Performance Improvements In The Next Linux Kernel

[pcxmac]

Is there automatic way of disabling CoW on “bad to CoW” files like VMs or bittorrent?

you need to look at how you partition your system and what kind of performance/qualities you want at various points on your system's file structure. For instance, you can use EXT4 for your VM images which might use QCOW or QED or other snapshot capable images, where as you might want to use BTRFS for '/' or ZFS for your /srv/zfs | nfs ... BTRFS makes a lot of sense in certain circumstances, zfs, ext4, etc... the real crux of the matter is how you want to use your system. If you want one size fits all, you will compromise heavily almost always. It's worth it to look at things like autofs and how you want your file system(s) to perform.


... with BTRFS it is possible to mount several subvolumes at once, to have readonly snapshots or ones you can modify, you can snapshot different parts of your tree, set different mount options for, including nodatacow. ZFS offers a lot, but IMO, it's much less intuitive and more cumbersome with it's additional overhead when it comes to system images.

[Nille]

Are there any News about the Raid5/6 State? It is stable and bullet proof now? And any news on checksums for parity against bitrot? ZFS is not on in the Mainline and it inhale RAM like if it is nothing.

[pcxmac]

you should ALWAYS backup your data, RAID5/6 is a strategy for recovery and keeping the volume/pool going while it's degraded, it's not a way to ensure you keep your data. Plenty of things can go wrong for any file system where you end up losing your data. I have lost quite a bit of data w/. ZFS. I assure you, YOU MUST BACK UP at least to one other spot, 2-3 other locations (physical and logical) is much safer. If you are serious about your data, you have at least 2 machines, electrically isolated. It also doesn't hurt to have critical data on a hot swappable hard drive you can plug in and record on.

[Nille]

I don't use it as backup but a full restore is a pain in the ass and needs plenty of hours. if a device is failing and the parity data are invalid you are down for hours. for this reason we use raid and not a simple LVM (because i could just restore the data from a backup in case of a failing drive ).

Btw. there is another reason why i don't use ZFS. The Number of Drives is constant. i cant simply just add another device to the raid.

Thats brings us back to the question about the reliability from btrfs. pcxmac since you direct jump to common knowlege about backup, i assume that it is in not a good shape.

[macemoneta]

Is btrfs stable for daily use? I see people regularly mention that ZFS should be used for critical data and btrfs is still a toy...

Anecdotally, I've been running BTRFS for about 7 years on multiple systems, through several device failures, without any loss of data. It actually caught a silent failure on one of my drives that was writing corrupted data (saved by BTRFS RAID1). Google is now using BTRFS on Chromebooks with Crostini (Linux apps). And, of course, Facebook uses BTRFS.

[jacob]

Is there automatic way of disabling CoW on “bad to CoW” files like VMs or bittorrent?

Yes, there is. You can disable CoW per-file or per-directory. See man chattr. Caveat: it only works on newly created files so the best approach is to create a new dir for your VM images, disable data CoW on it, and THEN create your VMs.

[nranger]

Are there any News about the Raid5/6 State? It is stable and bullet proof now? And any news on checksums for parity against bitrot? ZFS is not on in the Mainline and it inhale RAM like if it is nothing.

Not sure what you mean by "checksums for parity against bitrot"? Btrfs has had checksumming for reliable data and metadata for years now.

Perhaps you're referring to the RAID5/6 "write hole" where existing parity data could be insufficient to determine if a write was completed across multiple devices? If that's what you were thinking of, it's still an open issue. However I think it's a little overblown. Btrfs is still checksummed and CoW, which can at least help. I've been running btrfs RAID5 on my NAS for a few years (I think kernel 3.14-3.16 time frame) and it survived more than a few power outages and I've never had an issue. Admittedly I'm not an enterprise user and the box has at most 2-3 users at once. But it has been reliable and things like scrub performance have improved considerably over the years.

Status is documented on the Btrfs Wiki.

https://btrfs.wiki.kernel.org/index.php/Status#Overview

The "unstable" definition for RAID56 sounds a little ominous on that page, but I think that's the developers being overly cautious. I don't think they're wrong, since eating someones data could immediately sour them on the project. But for many people who want to give btrfs RAID a whirl I think they'd be positively surprised.

[dreich]

Nice to see improvements coming along!

[pcxmac]

I don't use it as backup but a full restore is a pain in the ass and needs plenty of hours. if a device is failing and the parity data are invalid you are down for hours. for this reason we use raid and not a simple LVM (because i could just restore the data from a backup in case of a failing drive ).

if you are working in a professional setting you might want to consider spending more and mirroring / load balancing. Probably use ZFS too. There are really any number of factors which affect which file system you use. Having to rebuild a RAID array while it is operational can be a little taxing. As for LVM, lulz. Hate it with a passion, no need for it when you have file systems like ZFS or BTRFS, and MDADM is good enough anyways. LVM on the commandline = puke.

[profoundWHALE]

BTRFS ate my RAID10 data so that was fun.