EXT4 Lands A Nice Performance Improvement For Appending To Delalloc Files

[cynic]

Also Steam games will sometimes break on XFS. And after 3 decades XFS cannot shrink.

not being able to shrink is my only complain against XFS.

definitely not a must-have feature, but would be nice tho.

[stormcrow]

it's a performance improvement for EXT4. Where did it mention XFS?

At first glance, you might say that. But XFS, ZFS, and other filesystems also use delayed allocation. It's a legit question. This specific code change only touches ext4 code though (which is obvious if one clicks the link to the pull request).

[david-nk]

So basically a log file optimization?

[stormcrow]

Also Steam games will sometimes break on XFS. And after 3 decades XFS cannot shrink.

Shrinking filesystems aren't really a killer feature. It might be desirable on some desktops, but the people with the money driving development efforts usually only care about growing volumes. It is true about Steam games, but again, desktop tiny minority of Linux use. Steam is a minority even among Linux desktop users as most are developers, not proprietary game players. There's also some boot shims that will balk at XFS hosting the kernel or root filesystem, again, probably not a big issue for most.

The main reason outside of the above situational gotchas, is that XFS has some write amplification issues on SSDs with some work loads. If the issue doesn't bother you because the underlying reason for the amplification is more desirable, then there's no real reason not to use it if it's a fit for your use case. One reason you might pass on both XFS and EXT4 is if you need more advanced volume management capabilities in solutions like ZFS.

If you're in a mixed platform environment, you may not want to use it on external hard drives that are being passed among team members as neither Windows nor Mac can read XFS. FreeBSD only can through a FUSE driver. It's going to be slow. The other two BSDs are probably unable to read it.

[sbivol]

Can someone explain me the reason to use EXT4, when XFS is faster and has more features?

Unlike XFS, EXT4 never lost data on power loss. This was a design mistake of XFS which was fixed around RHEL 8, but every RHEL version up to and including RHEL 7.0 would simply lose recent changes on power loss or unexpected reboot.
And no, XFS is not faster than EXT4 in any real life benchmark that takes into account data persistence. If you need the fastest way to discard data then you might as well use /dev/null instead of bothering with XFS.

[digitaltrails]

Can someone explain me the reason to use EXT4, when XFS is faster and has more features?

Another aspect to consider is recoverability. I can't track down the reference, but I recall reading XFS was best used with a UPS, where as ext4 was generally OK without one (or at least remained recoverable via fsck). That was years ago, so it may well be XFS has improved since then.

That ext4 is mature and quite recoverable was further backed up by papers such as Shehbaz Jaffer, Stathis Maneas, Andy Hwang, and Bianca Schroeder, Evaluating File System Reliability on Solid State Drives (https://www.usenix.org/conference/at...ntation/jaffer). Their paper only considered btrfs, ext2 and F2FS, but the recoverability of ext4 turned out to be quite robust.

This evening, while searching for the above reference, I also found this paper: Anthony Rebello, Yuvraj Patel, Ramnatthan Alagappan, Andrea C. Arpaci-Dusseau, and Remzi H. Arpaci-Dusseau, Can Applications Recover from fsync Failures? (https://www.usenix.org/conference/at...tation/rebello). I've only briefly scanned it, it seems quite interesting.

My interest is purely from the perspective of a desktop user with about 2 TB of for /home on a single SSD, no UPS, and root on a separate smaller SSD, rotating media used for on and offline backups only. A lot may have changed since I last examined the issue, but I've stuck with ext4 because of reliability, not speed. By reliability, I mean a fsck has always managed to return me to a mountable file system, at which point it's easy to make comparisons with the various backups.

[szymon_g]

Can someone explain me the reason to use EXT4, when XFS is faster and has more features?

Does xfs support FS level encryption? What about enlarging the partition without losing the data?

[sbivol]

it may well be XFS has improved since then.

There were a few attempts at fixing XFS, starting with this pull. I don't know if the fundamental problem of writing metadata before data was ever solved in XFS, this is what allowed it to claim being faster than EXT4. Linus himself described writeback as idiotic, adding "It literally does everything the wrong way around - writing data later than the metadata that points to it. Whoever came up with that solution was a moron. No ifs, buts, or maybes about it".

I can say from experience that XFS in RHEL 8 does not have this particular flaw where any system reset causes data loss. I haven't bothered with investigating the writeback data loss scenarios since I gave up on XFS completely and would never use it no matter how much FUD Red Hat continues to peddle about EXT4.

I've stuck with ext4 because of reliability, not speed.

Then you're lucky since you also got the best speed possible in this scenario.

[coder]

Another aspect to consider is recoverability. I can't track down the reference, but I recall reading XFS was best used with a UPS, where as ext4 was generally OK without one (or at least remained recoverable via fsck). That was years ago, so it may well be XFS has improved since then.

That ext4 is mature and quite recoverable was further backed up by papers such as Shehbaz Jaffer, Stathis Maneas, Andy Hwang, and Bianca Schroeder, Evaluating File System Reliability on Solid State Drives (https://www.usenix.org/conference/at...ntation/jaffer). Their paper only considered btrfs, ext2 and F2FS, but the recoverability of ext4 turned out to be quite robust.

Used how? In the early days of EXT4, I remember performance with barriers enabled sucked, so the normal thing was to mount it with the nobarrier option. That surely had to impact recoverability, however.

I believe the main reason we use XFS, at my job, is for combating filesystem fragmentation. If you leave about 5% to 10% of free space, that's plenty for XFS to ensure fragmentation doesn't become too bad. On mechanical hard disks, fragmentation poses a serious performance issue.

This evening, while searching for the above reference, I also found this paper: Anthony Rebello, Yuvraj Patel, Ramnatthan Alagappan, Andrea C. Arpaci-Dusseau, and Remzi H. Arpaci-Dusseau, Can Applications Recover from fsync Failures? (https://www.usenix.org/conference/at...tation/rebello). I've only briefly scanned it, it seems quite interesting.

If fsync() or fdatasync() don't work as advertised, I consider it a filesystem bug. If the caller fails to handle an error, such as EINTR, in a robust way, then the application code is at fault.

My interest is purely from the perspective of a desktop user with about 2 TB of for /home on a single SSD, no UPS, and root on a separate smaller SSD, rotating media used for on and offline backups only. A lot may have changed since I last examined the issue, but I've stuck with ext4 because of reliability, not speed. By reliability, I mean a fsck has always managed to return me to a mountable file system, at which point it's easy to make comparisons with the various backups.

I like BTRFS for its checksums.

At my job, we make use of BTRFS' ability to do atomic snapshots - when making backups, we first make a snapshot and then backup that. This has two advantage:

1. minimizing the chance of inconsistencies, due to the backup process reading different files at different times.
2. letting you know exactly what was backed up, as a consequence, enabling you to verify the backup!

For relatively low-turnover data, snapshots also serve as a type of backup that protects against mostly user error (e.g. "whoops, deleted wrong file/directory!"). We use hourly snapshots, on a departmental fileserver, for this purpose.

[mrg666]

There are multiple other options but I see no reason to try another file system as ext4, and 2, 3 before that, have all been very reliable and fast for my workstations. Compiling the kernel, for example, is never limited by the file system as CPU load is always close to 100%. I don't think Abaqus or GIMP will be any faster with another file system either. It is good to see improvements are still being done on such a mature software.