XFS Copy-On-Write Support Being Improved, Always CoW Option

[Chugworth]

The COW features they're adding to XFS look pretty nice. In fact, reflink support is one feature OpenZFS doesn't have that I wish it did. XFS may end up being a clearly superior choice over EXT4, but it's still a very long way behind from the features you get in ZFS or even Btrfs.

I would trust Btrfs for a single disk system. Not so much for a mirrored disk setup due to some balancing issues I've seen myself if you replace a drive when the filesystem is nearly full. Even still, I wouldn't expect it to lose data on its own. But then as polarathene mentioned, Btrfs has some performance issues for VMs. It helps to turn off COW, but then you eliminate one of the main reasons to use Btrfs in the first place. ZFS gets around that issue with the way that it caches lots of the data to memory. For example, on a FreeNAS server with 16GB of memory, the ZFS ARC cache is commonly around 11 or 12 GB. That's a pretty nice feature. It takes an array of spinning disks and gives them some of the speed benefits of hybrid drives.

[bigletter]

The main reason for performance issues with VMs is due to COW. You can always disable COW on a file or on a directory. So just apply this to your VM disk images or directories that the vm uses and you will be fine. That way you can both eat the cookie and keep it.

[PuckPoltergeist]

Btrfs has some performance issues for VMs. It helps to turn off COW, but then you eliminate one of the main reasons to use Btrfs in the first place.

Why is CoW a main reason to use Btrfs for a VM image?

[pal666]

Last time I tried to install Fedora it wouldn't let me use brtfs for / or something like that.

i am using fedora with / on btrfs for many years. and i installed via anakonda, though last few releases i'm upgrading with dnf instead of reinstalling

[pal666]

Would the CoW support for XFS mean it'd get slower perf using that feature? Similar to BTRFS?

yes. probably even slower, since btrfs was designed for cow

ZFS also does CoW right?

yes

But has some differences from how it works compared to BTRFS

zfs doesn't use btrees because it was created before invention of cow btrees

where it doesn't share the same issues as BTRFS does with running Databases or VMs (without disabling CoW)?

no, you can't get fast dbs or vms without disabling cow

If so which type of CoW is the XFS support more likely to be?

my bet is the slowest one

[pal666]

BTRFS missed the boat... Kent Overstreet summed it up pretty well:

if btrfs missed the boat, what missed kent overstreet?

[pal666]

XFS may end up being a clearly superior choice over EXT4

if it will develop support for resize, but i'm not holding my breath

, but it's still a very long way behind from the features you get in ZFS or even Btrfs.

btrfs has more features than zfs

I would trust Btrfs for a single disk system. Not so much for a mirrored disk setup due to some balancing issues I've seen myself if you replace a drive when the filesystem is nearly full.

i wonder how your one drive setup fares in that situation

Btrfs has some performance issues for VMs. It helps to turn off COW, but then you eliminate one of the main reasons to use Btrfs in the first place.

no, you don't elimitate any reasons. vms have their own snapshotting facilities.

ZFS gets around that issue with the way that it caches lots of the data to memory.

linux also caches data in memory

[quikee]

I don't see why they would need to change the on-disk format.
The basic concept of CoW is just "write all new data in free space, THEN update the filesystem metadata to mark the new data you just wrote as current, and mark the old data as free space". THere is no need to store additional information in the on-disk format.

AFAIK this is already what journal based FS does, but a little bit more complicated (journal writes) so that a FS can't be left in a corrupt state when a power loss happens (fsck just clears the partial writes).

A CoW is a bit different concept and actually does what the name says - when a write happens, it copies the whole file metadata, writes, then marks the copy as current, modifies the reference count and deletes what is not referenced anymore (that's how you can have snapshots as the old copy just needs to be referenced by the snapshot).

[oiaohm]

BTRFS missed the boat... Kent Overstreet summed it up pretty well:

Kent Overstreet developer of bachefs talks against btrfs and zfs. Apparently you have not read this.

"btrfs, which was supposed to be Linux's next generation COW filesystem - Linux's answer to zfs.

Lets disregard the zfs documented design mistake here that btrfs copies.

Unfortunately, too much code was written too quickly without focusing on getting the core design correct first, and now it has too many design mistakes baked into the on disk format and an enormous, messy codebase - bigger that xfs. It's taken far too long to stabilize as well - poisoning the well for future filesystems

That part of what you said you can equally say about zfs and btrfs I will explain more latter.

because too many people were burned on btrfs, repeatedly (e.g. Fedora's tried to switch to btrfs multiple times and had to switch at the last minute, and server vendors who years ago hoped to one day roll out btrfs are now quietly migrating to xfs instead)."

This bit is ignoring that zfs is a incompatible license in many countries so Fedora has not been willing to try zfs full stop. There is another reason to go xfs.

Redhat is deprecating BTRFS - and that means the vast majority of Corporate Linux installations won't be using it. As you mention, the only distribution which is using it as default is SUSE - and that means the vast majority of people won't go out of their way to install it - and it's very doubtful other distributions will be choosing it as their default.

SUSE is major EU distribution with paid resources so btrfs is going to keep on development.

You have not really asked the question why has Redhat turned away from btrfs and does this apply to zfs. When you ask these questions you will find a list like the one Kent Overstreet listed against both btrfs and zfs.

Lets start with a developer working with Redhat issue against btrfs.

https://lwn.net/Articles/747633/

Chinner said. Btrfs subvolumes share the same superblock, which can cause some subtle issues about how they are treated by various tools like find or backup programs.

Does this problem effect zfs yes it does.

I will give you the problems.

1) You only have 1 backup program this is cp. You now need to extract 1 snapshot from btrfs or zfs to be placed on a new drive. Due to the source drive failing you do not have time and cannot install any other software.

The way LVM currently and the way XFS is lining to perform snapshots you are able to abstract snapshots with only the cp command. Yes XFS due to design.

2) Checksum offloading to raid controller??? How with btrfs and zfs.

This is one of the big differences about the xfs path instead embedded a stack of stuff into the file system driver like raid they are working on improving the communication with block devices like raid controllers. Yes raid controllers in hardware can be check-summing every block of data read from hard-drive.

I could keep on going.

[oiaohm]

A CoW is a bit different concept and actually does what the name says - when a write happens, it copies the whole file metadata, writes, then marks the copy as current, modifies the reference count and deletes what is not referenced anymore (that's how you can have snapshots as the old copy just needs to be referenced by the snapshot).

https://www.clevernetsystems.com/lvm...ots-explained/

COW can be implemented at the file-system and block subsystem in Linux. LVM is copy on write at block level.

This is the question where should copy on write and deduplication and checksums be implemented. XFS is working on making copy on write and deduplication and checksum able to be a block driver feature. Being a block driver features these features could be implemented in raid controllers and other accelerators. The idea of zfs and btrfs to integrate this stuff into the file system driver could have been completely wrong.

XFS developer in some ways has the advantage. He sat back and let btrfs and zfs go for it can look at how they stuffed it up and go back and implement it correctly.