Btrfs Gets Fixes & Low-Level Improvements With Linux 4.19

[polarathene]

With cow disabled, disk images work just fine on Btrfs. It won't provide the world's best VM performance but it's largely adequate for desktop VM needs. In practice, non-cow files on Btrfs behave and perform basically similarly to zvols on ZFS. In fact they are actually the same thing, just presented differently to the user.

What benefit is there to still using btrfs for vm images with Cow disabled? That prevents most of the useful features that makes Btrfs stand out as a FS doesn't? Does the multi-disk of varied sizes support still work? No checksums or deduplication right?

I'm not sure of Michael has done nocow benchmarks against other systems, I guess it is faster than cow BTRFS but would I be better off with ZFS or XFS instead for databases and VM disk images?

[jacob]

What benefit is there to still using btrfs for vm images with Cow disabled? That prevents most of the useful features that makes Btrfs stand out as a FS doesn't? Does the multi-disk of varied sizes support still work? No checksums or deduplication right?

I'm not sure of Michael has done nocow benchmarks against other systems, I guess it is faster than cow BTRFS but would I be better off with ZFS or XFS instead for databases and VM disk images?

The benefit is that CoW is disabled for those vm images only. That also means the images are not checksummed (not by Btrfs, anyway) but all the rest still works, including built-in RAID, resizes, varying sizes etc. You can have your entire disks under btrfs, without having to worry about partitioning etc., with CoW and all the other goodness, only the vm images stores on that same volume will have CoW disabled. That's fine, by the way, because although those images don't then benefit from all the consistency and integrity protection that Btrfs has normally to offer, the data stored within are in any case no better protected that what the virtual machine's filesystem does.

[kgardas]

[...]
If you already have an existing disk image, you need to enable +C on the directory and then physically copy it so that a new non-cow file is created.

With cow disabled, disk images work just fine on Btrfs. It won't provide the world's best VM performance but it's largely adequate for desktop VM needs. In practice, non-cow files on Btrfs behave and perform basically similarly to zvols on ZFS. In fact they are actually the same thing, just presented differently to the user.

Except that ZVols still provide snapshoting, checksum capability and hence self-healing when possible and still are CoW. Quite a difference, isn't it?

[jacob]

Except that ZVols still provide snapshoting, checksum capability and hence self-healing when possible and still are CoW. Quite a difference, isn't it?

Zvols are not CoW, except manually on demand (e.g. snapshotting). By the same token they can't and don't checksum the contents, only metadata about the zvol itself. Non-CoW files in Btrfs are the same: the file itself is not full CoW but its metadata are, and you can take snapshots (in which case CoW will be used to create the snapshot itself), and yes they also self-heal etc.

[waxhead]

Thank you

I just hope it will get better someday. most of my vm's are on lvm-lvs right now. another problem for me as btrfs doesnt like lvm that much (ok, it just dont likes snapshots or lvm-raids)

i knew that chattr "trick" but it seems to risky to me - i tend to forget such things.

There is one pitfall that is worth mentioning. If you turn *on* compression , even for no-cow images you "break" the no-cow on that file (despite the attributes set on it).

In other words - some people believe their files are no-cow but since they ran a "btrfs filesystem derfrag -vrc /some_mnt_point" they accidentally defragged and compressed their images file.

Keep in mind that nothing actually "breaks" e.g. the file is good, but thanks to suddenly being COW again things slow down and users gets (understandable) confused. Yet another prime example about why BTRFS gets an undeserved bad reputation , people don't know about these things so even if I think you are better off with another filesystem , it all depends on your workload and you should test it out. Just maybe it is (plenty) fast enough for your use.

[flower]

There is one pitfall that is worth mentioning. If you turn *on* compression , even for no-cow images you "break" the no-cow on that file (despite the attributes set on it).

i learn something new everyday

it all depends on your workload and you should test it out. Just maybe it is (plenty) fast enough for your use.

i tried a while back and it really hurt my performance.
i had two solutions:
1) dont change the raw image directly. instead i wrote a script that created a new overlay qcow and merged it back after my vm stopped (which they do frequently so it was ok for me.)
2) backup whole vm and restore it after the extent count exceeded a certain value (i forgot which one it was though)

atm i am very happy that my vm's live directly on lvm and dont use any image file anymore. to be hones: as much as i love btrfs-concepts... i dont want my vm's to run off an image file anymore.

but btrfs on top of lvm has its own drawbacks (eg. NEVER forget that you can NEVER do a lvm-snapshot)

another problem i had: thunderbird/chromium/firefox sqllite profile-databases where very fragmented on btrfs too. but i didnt notice a performance hit there. this one was easy too solve though: i just have them in tmpfs since then

[polarathene]

all the rest still works, including built-in RAID, resizes, varying sizes etc. You can have your entire disks under btrfs, without having to worry about partitioning etc.

OK, the RAID support with varying disk sizes is nice along with subvolumes. But besides those conviniences not much else offered by the FS for nocow files? Then I guess it's important to compare I/O perf under nocow vs something like XFS?

[jacob]

OK, the RAID support with varying disk sizes is nice along with subvolumes. But besides those conviniences not much else offered by the FS for nocow files? Then I guess it's important to compare I/O perf under nocow vs something like XFS?

If you want to use that machine only, or primarily, as a VM host then yes, obviously there is little point in using Btrfs, you're much better off with Ext4 or XFS. But if you have a general machine, like a workstation, laptop or server, you like what Btrfs offers but you want to be able to occasionally run a VM (testing various distros, using MS office, whatnot) then this lets you have the full benefits of Btrfs AND keep decent VM performance while sparing you the hassle to use dedicated partitions for the VM images.

[polarathene]

If you want to use that machine only, or primarily, as a VM host then yes, obviously there is little point in using Btrfs, you're much better off with Ext4 or XFS. But if you have a general machine, like a workstation, laptop or server, you like what Btrfs offers but you want to be able to occasionally run a VM (testing various distros, using MS office, whatnot) then this lets you have the full benefits of Btrfs AND keep decent VM performance while sparing you the hassle to use dedicated partitions for the VM images.

I'm not against BTRFS but if there is better benefits to using another partition with a different file system for anything nocow related, that's fine by me. I'll be sure to do a perf comparison but in general BTRFS and nocow only offering the advantage of avoiding partitions isn't that big of a benefit based on how it tends to perform for anything important I/O speed wise, I'm fine having a separate data partition, or just using BTRFS to manage my system files, but keeping my projects (at least active ones) in a different file system and archiving /backing up to BTRFS volume when done. I really like what openSUSE is doing with BTRFS and their integrations like Snapper and recent RO boot able snapshots that update to a new snapshot requiring reboot to avoid any breakage(eg kernel update).