NixOS Takes Action After 1.2GB/s ZFS Encryption Speed Drops To 200MB/s With Linux 5.0+

The change in Linux 5.0 that initially broke ZFS On Linux compatibility ends up being pretty nasty for the ZFS encryption performance... A NixOS developer reports that the functions no longer exported by Linux 5.0+ and previously used by ZoL for AVX/AES-NI support end up dropping the ZFS data-set encryption performance to 200MB/s where as pre-5.0 kernels ran around 1.2GB/s.

This was the problem early on during the Linux 5.0 kernel cycle with the __kernel_fpu_ begin/restore functions no longer being exported. Even with the Linux 5.2 kernel merge window going on right now, a solution to satisfy the ZFS On Linux usage hasn't been achieved. Their used workaround by ZFS On Linux disables the vector-based (SSE/AVX) algorithms when building against Linux 5.0+. That in turn is what causes the big performance drop when running on the recent kernels.

Unfortunately, the key upstream kernel developers involved admittedly don't care about ZFS On Linux since it's out-of-tree / incompatible license for mainline and the reason they dropped the code in the first place was over the lack of any mainline users. So while it would be simple to restore the code, they have not.

NixOS is now the first distribution I am aware of that has decided to carry the tiny patch on their own kernel build so the symbols get exported and in turn gets picked up by the ZoL build system. They merged the patch earlier this week.

Hopefully more Linux distributions will do so or the support restored in the mainline kernel as to not decimate the ZFS encryption performance.