Btrfs Zstd Support Coming To Linux 4.14

[scineram]

What you can disable is ZIL (not really, but setting logbias=throughput kinda tells zfs to avoid it)

You can actually set sync=disabled completely.

For torrents in my understanding it is fairly standard among clients that they write out downloaded data in 16KB chunks or a multiple of that. So setting the ZFS blocksize to 16KB on that dataset should eliminate the RMW cycle and some fragmentation.
Now this is not as efficient as much bigger blocksizes both in terms of metadata and compression. Luckily a lot of clients support separate download directories for temporary files, so the best method would be to make a separate dataset for the tempdir with 16KB blocksize. Then when the client moves the tempfile to final place it will also defragment the data.

[scineram]

What you can disable is ZIL (not really, but setting logbias=throughput kinda tells zfs to avoid it)

You can actually set sync=disabled completely.

For torrents in my understanding it is fairly standard among clients that they write out downloaded data in 16KB chunks or a multiple of that. So setting the ZFS blocksize to 16KB on that dataset should eliminate the RMW cycle and some fragmentation.
Now this is not as efficient as much bigger blocksizes both in terms of metadata and compression. Luckily a lot of clients support separate download directories for temporary files, so the best method would be to make a separate dataset for the tempdir with 16KB blocksize. Then when the client moves the tempfile to final place it will also defragment the data.

[profoundWHALE]

For the record, Zstd is developped by Yann Collet, the guy behind LZ4.

...in the specific cases where Zfs basically automatically activates its "nocow" options - which would be the recommendations given by the BTRFS wiki too.

ZFS is able to detect these special case and shut down "COW", forcing all the files to be written "inplace" - thus exchanging performance for stability(*).

BTRFS doesn't do this (but still has an options to do auto-defrag and at least reduce the amount of fragmentation), in order not to take stability-reducing decision without the user's command(*).

What the heck are you guys talking about. We've got people saying that BTRFS is faster, which all depends on how the thing is configured. For instance, if Michael does a benchmark, ZFS will almost always be slowest because it is not properly configured. If you're screwing up ZFS with torrents then you're doing something very wrong. ZFS is COW as well so what the heck are you talking about?! It doesn't 'turn off COW' because that's like removing my spine for every time I want to get into a car. Wikipedia page has a nice little piece about it.

According to Zstd's own author, it plays in a different field.
LZOP (and LZ4) (and Snappy) are all designed to be as fast as possible algorithm while still providing some compression.

Zstd aims to the same category as GZIP (average compression / average speed), while beating it a bit (much faster and/or a tiny bit better compression).

TL;DR zstd pretty well ties or sometimes beats gzip for the same compression, but crushes it in speed. LZ4 is still speed king.

[omgold]

LZ4 is still speed king.

Not by much, though, by my recent speed test of various compressors on my Ryzen:

https://imgur.com/a/LT17f

As input was used a sosreport, which contains only ascii test, compressing very well. Of course results may differ for different input.

Short version: at fastest settings lz4 is roughly 40% faster than single-threaded zstd, parallel zstd is roughly 40% faster than lz4.

[AndyChow]

I agree. This is nice to have but hardly a killer feature. For me the one advantage ZFS has is encryption support. Apart from that BTRFS is just fantastic.

ZFS has had encryption for what, a month?

I like the ability of BTRFS to grow and SHRINK an array, it's speed. But ZFS wins in terms of stability.

[jacob]

ZFS has had encryption for what, a month?

I like the ability of BTRFS to grow and SHRINK an array, it's speed. But ZFS wins in terms of stability.

ZFS is stable... on Solaris. On Linux (and even on FreeBSD) it's a massive chunk of foreign code forced upon the kernel through a hackish API emulation layer. AFAIK it has never underwent any serious large scale testing. Plus, there are other issues with ZFS besides that (and its licence).