Linux 5.5 SSD RAID 0/1/5/6/10 Benchmarks Of Btrfs / EXT4 / F2FS / XFS

[moltonel]

This is an apples to oranges comparison: Btrfs is running with a different scheduler and raid system (mq-deadline vs none, native vs md) than all the other filesystems. The different raid system is understandable (although they're not as equivalent as the names imply, and it would be interesting to compare native vs md on btrfs), but the different scheduler is perplexing.

[GreenReaper]

Recommended by the way it's been used by Facebook and Synology - as checksumming and snapshot layer over the top of the block storage (including mdadm RAID in Synology's case). The btrfs project itself does not see RAID56 mode as stable.

In our case we expect to use to store original media files, so the checksumming is important to us (in theory, that should be dealt with at the disk layer, but in practice...). As the data will essentially be written once and then read many times, some of the performance issues in the benchmarks here do not apply.

We could go RAID10, but the loss of capacity and read performance (due to being further along the HDD performance curve) would be significant.

[xinorom]

It's ironic that it's in RAID56, where btrfs has received so much criticism, that it is competitive. (Fortunately this is also the area that I want to use it in next.)

The solution to the RAID5 write hole is just to not use RAID5. The other solutions just make it even less efficient and performant than it already is and RAID5 is fairly shit to begin with. This issue is not unique to Btrfs.

The performance in single mode is disappointing ...

The performance of these particular tests may be disappointing to some, but testing a CoW filesystem against 3 much simpler, less featured filesystems and then not doing a single test of features afforded by CoW semantics (e.g. reflink copies) seems kind of biased. Of course, if all you care about is braindead throughput benchmarks, you don't really even need a CoW filesystem.

... given that this is the usage of it that is most-recommended.

Recommended by whom? Most people who want a CoW filesystem most likely also want some kind of RAID.

[GreenReaper]

It's ironic that it's in RAID56, where btrfs has received so much criticism, that it is competitive. (Fortunately this is also the area that I want to use it in next.)

The performance in single mode is disappointing, given that this is the usage of it that is most-recommended.

[S.Pam]

Zfs and Btrfs compared to plain filesystems https://markmcb.com/2020/01/07/five-years-of-btrfs/

[pkese]

You either want a CoW filesystem or you don't. If you don't, use XFS and enjoy the performance. If you do, XFS just isn't an option. Comparing them is pretty useless for any purpose other than masturbatory benchmarking...

Indeed. With COW you pay a bit of performance, but you get snapshotting, some extra reliability and other features for free. None of that is reflected in the benchmarks.

Ever since SSDs have become the norm, filesystem features (like snapshotting, data compression, online resizing, send-receive, subvolumes, etc) outweigh benchmark performance in most cases.

[xinorom]

Shame BTRFS is lagging behind in the performance category. XFS has made some nice leaps and bounds but, however no Windows compatibility for that one. (if that matters to you)

XFS isn't a CoW filesystem. A filesystem that lacks certain features (that imply bookkeeping overheads) is necessarily going to be faster, or at the very least easier to optimize. I don't know why so many people fail to understand that comparing the performance of a CoW filesystem to a non-CoW filesystem is not an apples-to-apples comparison.

You either want a CoW filesystem or you don't. If you don't, use XFS and enjoy the performance. If you do, XFS just isn't an option. Comparing them is pretty useless for any purpose other than masturbatory benchmarking...

[bash2bash]

I've been using XFS for many years on 100+ servers and desktops, its a great file system, reliable and fast.

Unfortunately, there is one major drawback, it does not support shrinking. Ultimately, this is a major disadvantage for cloud providers that need to shrink/grow filesystems on virtual machines and system images.

One sad example, is CentOS under various cloud providers, it has been modified to use EXT4 instead of the default XFS, due to the inability of XFS to shrink.


Maybe who ever is sponsoring XFS does not care to invest in shrink functionality, but ultimately they are shooting themselves in the foot.

[theriddick]

Shame BTRFS is lagging behind in the performance category. XFS has made some nice leaps and bounds but, however no Windows compatibility for that one. (if that matters to you)

[pkese]

Would be nice to include ZFS to the mix. I wonder how that compares to btrfs.