ZOL 0.8 Nears With RC3 Release - Big Update For ZFS On Linux

[phoronix]

Phoronix: ZFS 0.8 Nears With RC3 Release - Big Update For ZFS On Linux

ZFS On Linux (ZOL) 0.8 is going to be a big release... No, a huge release. But for ensuring it's going to be a successful release, a third release candidate was just issued for further vetting of all the new code...

http://www.phoronix.com/scan.php?page=news_item&px=ZFS-On-Linux-0.8-RC3

[darkbasic]

What's the new "pool initialization" feature?

[ypnos]

I don't really understand why they can't write their own checksumming code (with SIMD instructions). Anybody knows what's the issue here?

[danboid]

What's the new "pool initialization" feature?

I think this could be the new pool versioning / features feature. It will allow users to create pools with features specific to a certain year ie create a pool with all the ZFS features circa January 1st 2018.

[mskarbek]

What's the new "pool initialization" feature?

Basically zeroing whole drive after pool creation. Useful when you are creating a pool on a thin provisioned virtual drives - writing zeros to the entire drive will allow you to prevent paying initialization (storage acquisition on a physical drive) cost when you are actually needing it. This is done in the background as a task that runs when the pool is idle.

[mskarbek]

I don't really understand why they can't write their own checksumming code (with SIMD instructions). Anybody knows what's the issue here?

Portability and performance. ZoL has it's own code (without SIMD) for other architectures and 0.8.0 will probably fall back to it on x86_64 (https://github.com/zfsonlinux/zfs/is...ment-454169579). To use SIMD they would have to forward part of the computation to the user-space daemon which is unavailable in early boot stage and, let's just say, "hard to justify for filesystem" at all.

[Weasel]

I don't really understand why they can't write their own checksumming code (with SIMD instructions). Anybody knows what's the issue here?

They already did? The problem is that the SIMD state and registers need to be saved properly and then restored.

[oiaohm]

To use SIMD they would have to forward part of the computation to the user-space daemon which is unavailable in early boot stage and, let's just say, "hard to justify for filesystem" at all.

This is wrong. A Usermode helper can be placed in the initramfs that is single cpio archive that is loaded into memory by bootloader. Accessible as soon as the kernel fires up before it need to use file system drivers.

Usermode helper are accessible and usable as soon as you can use kernel modules not built into the primary kernel image as long as they were copied into the initramfs.

Next problem if all ZoL is using SIMD to speed up checksum create why are they not using the Linux kernel Crypto API.

https://www.kernel.org/doc/html/latest/crypto/index.html

Thing to remember is why the crypto API exists. Before it existed different kernel drivers use to implement their own versions of checksumming causing the following issues:
1) Not well optimised for platforms.
2) Security and coding issues.

ZoL is not quite NIH its NRL. Not Right License. Result of not being the right license is that you end up duplicating items you should not be duplicating. Yes if ZoL is fully GPLv2 compatible they should have been just able to swap the removed functions with the newer versions and have their code work.

[mskarbek]

This is wrong. A Usermode helper can be placed in the initramfs that is single cpio archive that is loaded into memory by bootloader. Accessible as soon as the kernel fires up before it need to use file system drivers.

Usermode helper are accessible and usable as soon as you can use kernel modules not built into the primary kernel image as long as they were copied into the initramfs.

Ofc you can put such thing into initramfs but I will definitely not rely on that to mount my root partition or dataset in this case. Dracut is broken as it is and now you tell me that I should add another moving part to not only configure my booting system but to actually do stuff that should be handled by kernel in any sane scenario. Nope.

Next problem if all ZoL is using SIMD to speed up checksum create why are they not using the Linux kernel Crypto API.

https://www.kernel.org/doc/html/latest/crypto/index.html

Kernel Crypto API is GPL-only - you can't interact with it if your module have incompatible license.

[oiaohm]

Ofc you can put such thing into initramfs but I will definitely not rely on that to mount my root partition or dataset in this case. Dracut is broken as it is and now you tell me that I should add another moving part to not only configure my booting system but to actually do stuff that should be handled by kernel in any sane scenario. Nope.

bpfilter puts the usermode helper inside the .ko file.

https://lwn.net/Articles/755919/

So this has really not much to-do with Dracut any more. 4.18 and newer kernels it possible for usermode helper and you kernel module to be 1 .ko file. You have to deal with Dracut or some other initramfs anyhow when using a kernel module that is not gplv2 as it cannot be legally merged into the main kernel image that you wish to use as your root file system..

Please note you said here "actually do stuff that should be handled by kernel" when kernel has a Crypto API checksums should be the core kernel problem not being implemented in drivers..

Kernel Crypto API is GPL-only - you can't interact with it if your module have incompatible license.

Look a little closer

https://www.kernel.org/doc/html/latest/crypto/userspace-if.html

There are two interfaces to the Crypto API direct and netlink. Netlink does not have the GPL-only limitation and is use-able from userspace. Of course Netlink interface will provide some performance overhead.

So when there is a route that ZoL could be using the kernel based Crypto API why are they implementing their own so duplicating what the kernel can do. I know the performance will be bad using user mode module and the netlink api but it would provide list of what parts of the Crypto API would help ZoL being license changed.

Also the Crypto API is currently GPL-only in kernel space but there never been a request that I can find to make required parts of Crypto API in kernel space not GPL license locked for ZoL. Its just like the sample user-space Crypto API library being GPLv2 this is because no one has put in the time to make a non GPLv2 or even put in request for non GPLv2 one.

ZoL should have been submitting requests to make different items not GPL license years ago.

See the problem the feature that was removed that broke ZoL may not be the kernel feature it in fact requires. ZoL has not done the leg work on Linux kernel integration to make sure they are taking full advantage of what the Linux kernel can offer or even attempt to take advantage of what the Linux can offer.

Reality is hard that maybe ZoL kernel module should have never been doing SIMD functions in the first place when on Linux. Only using the unstable function and hack because they have not done the integration work.