ZFS On Linux Runs Into A Snag With Linux 5.0

[Almindor]

I agree.

I would also submit to Greg K-H that if FP code in the Linux kernel code is "unstable" and "should not be used and that's why we deprecated it", then Greg K-H should do the noble thing by falling on his sword and organizing a project within the Linux kernel team to fix the d@mn Linux kernel code for these FP calls that is "unstable".

I think Linux kernel code fixing is squarely within the wheelhouse of Greg K-H and the Linux kernel devs, but it's easier for them to say, "Nobody using it anymore, and besides it's unsafe code, so it needs to be deprecated." while never saying, "We don't know crap about how to fix that unstable code."

And a few notes for Greg K-H personally:
Your NIH attitude belongs at Mozilla, not within Linux kernel development where the spirit has been to not lock up control of a user's own computer away from it's owner/user.

Blocking access previously given to the FP function (or specifically the SIMD instructions, as someone mentioned in this forum) that are embedded within the CPU that is in the computer that the user likely owns is JUST PLAIN WRONG. If there is no Linux API to access them, then Linux is BROKEN BY DESIGN and should be tossed in the bit bucket. Otherwise, STUFF YOUR ATTITUDE and suggest the proper API(s) to access these CPU features via Linux; try being helpful instead of hurtful.

Frankly Greg K-H, your elitist NIH attitude and the entire stupid GPL concept in the Linux kernel code shows how far some in the Linux kernel development community have strayed from the founding concepts of Linux itself. Linux is becoming a "walled garden" or at least "a very exclusive garden" due to people like you and Linux companies like Redhat and Oracle.

Why don't you follow your own advice and find a workaround for them or submit patches to the kernel to enable a better way instead of telling someone who is either developing on free time or being paid to do proper in-tree work to change their attitude.

The arrogance is on your side here, Greg has all the rights to say what he did, you have 0.

[oiaohm]

And this highlights the difference between Windows and Linux. On Windows, APIs get depreciated, but they still functionally work. And Linux, they just remove the API then blame everyone else when their previously working software stops working.

This is not true for windows kernel ABI. This is why Drivers from windows XP without service pack don't work on XP with service pack. Or modern day drivers that works on first version of Windows 10 don't work on current versions of Windows 10.

Inside kernel space does not matter where you are ABI/API breakage will come.

[Marc Driftmeyer]

1. Existent
2. Sun is no longer an entity and last I checked Oracle contributes to Linux.

[oiaohm]

It's not using floating point, it's using SIMD to accelerate the checksum computation. And it does have a fallback. Should ZFS run slower on Linux because it's not GPL?

SIMD has to have the same protection switches as using FPU. Thank you intel for deciding to recycle some the less stable register copy operations in side their cpus..

The Linux kernel has a crypto_shash that wraps over all you hardware acceleration for checksums. Really bad news to use a lot of the hardware accelerations you need patent licenses Guess what OIN has got those patents license for the Linux kernel. Wait for the other shoe to drop. The patent license that OIN aquired only cover you if your code is GPL.

On hardware with proper checksum hardware accelerators ZFS on Linux not being GPL is going to be slow vs GPL file-systems in the Linux kernel doing checksums.

This is one of the problems you hit when you go to freebsd from linux is that hardware accelerators no longer have drivers and it not possible to write a non GPL driver for them.

There big issues with ZFS on Linux or anyone else thinking they will do checksum at high performance while not using GPL. So yes ZFS is running slower because its the wrong license + avoid code of right license.

Video card drivers are normally MIT or equal license so they code when built against Linux can use GPL only features. Remember there are many GPL only features that are GPL only because of patents and they are patented because they provide high performance.

CDDL being a conflicting license is badly restricting ZFS on Linux hands.

[GrayShade]

On hardware with proper checksum hardware accelerators ZFS on Linux not being GPL is going to be slow vs GPL file-systems in the Linux kernel doing checksums.

This is one of the problems you hit when you go to freebsd from linux is that hardware accelerators no longer have drivers and it not possible to write a non GPL driver for them.

There big issues with ZFS on Linux or anyone else thinking they will do checksum at high performance while not using GPL. So yes ZFS is running slower because its the wrong license + avoid code of right license.

Using SIMD to compute the checksums is fast enough and is not covered by patents with exceptions for GPL code. This isn't about hardware accelerators. It's about arbitrarily denying non-GPL modules access to SIMD. Do you believe it makes Linux better for anyone?

[oiaohm]

My problem isn't the API change. I'm sure they will work that out. The issue is the emotional comments by Linux kernel dev's and the users.. whatever Sun's motivation. (and someone show me some proof they actually believed CDDL was not compatible, because it's was Debian that made that determination.) ..Whatever their motivation.. they are long since dead. Time to move on.

https://sfconservancy.org/blog/2016/feb/25/zfs-and-linux/

There have been many different legal reviews with CDDL compatibility with GPL. Most find it impossible to meet CDDL requirements and GPL requirements at the same time.

I have not found one case where they say CDDL and GPL is 100 percent fine. The closest say due to the type of breach its not going to be enforceable in court. Problem is that is in a copyright court not a patent infringement count. Linux kernel is not just copyright. With OIN and other agreements to allow patented ideas into the Linux kernel under condition that is done under GPL means that a minor technical infringement of copyright comes a patent case of using a patented idea without a license so the harm and the problem.

The funny part is ZFS is in fact under CDDL due to NetApp patent requirements. I think those patents might be now expired.

Linux is a hair ball of copyright and patent license requirements.

[oiaohm]

Using SIMD to compute the checksums is fast enough and is not covered by patents with exceptions for GPL code. This isn't about hardware accelerators. It's about arbitrarily denying non-GPL modules access to SIMD. Do you believe it makes Linux better for anyone?

The rules are the rules. The functions ZoL was using starting with __ is internal usage only really. The anyone use function by name is GPL only because is ABI/API has not been argued out.

https://linux.slashdot.org/story/12/10/11/1918251/alan-cox-to-nvidia-you-cant-use-dma-buf

People forget this stuff. Nvidia had to go trough quite along process to make interfaces so their closed source driver can use dmabuf.

2003 form when __kernel_fpu_begin and __kernel_fpu_end entered kernel to 5.0 kernel there has been a open window to submit safe versions of these functions for non-GPL modules.

It is possible EXPORT_SYMBOL_GPL to be convert to EXPORT_SYMBOL but this is not something that happens quickly. This means the function need to have its API debated and has to have a legal review. Like someone from 2008 to now from ZoL could have asked for the legal review and change on the Linux kernel mailing list. You can check the mailing list no one asked. If no one asks no must want the feature right GrayShade.

Linux kernel developers are not mind readers. Linux kernel developers don't have unlimited amount of time for legal and code reviews either. Nvidia got their change by putting most of the slog work.

Both __kernel_fpu_begin and __kernel_fpu_end are totally valid to be gone the request should have been done years ago to change kernel_fpu_begin and kernel_fpu_end GPL export to normal export. You have to ask if you are just doing SIMD why are you having to backup every single register of FPU and SIMD with these functions as well.

Reality here the was work in mainline Linux kernel that those making kernel modules that are not GPL or mainline should have done.

Yes EXPORT_SYMBOL_GPL or __ tag symbols should not be used in linux kernel modules that are not aiming to be part of the mainline linux kernel tree. If want you need to-do requires you to use those functions and you module is never mainline for licence or other reasons you really do need to be on the mailing list asking for solution.

GrayShade like it or not the rules are the rules. Break them some point in future you will suffer. The __ rule on symbol was 1992 of the Linux kernel. The EXPORT_SYMBOL_GPL is newer but they both basically same thing. ZoL is in trouble now because it was using a pair of functions that it should not have been.

Really I am feed up with my out of tree module broke because the Linux kernel changed and you look the problem is straight way using EXPORT_SYMBOL_GPL or __ . Yes your out of tree module broke but it you fault because you never told upstream that you need that function stable.

The Linux kernel upstream is savage if you have told them that you need something well in advance of them removing it there could be a year or two of debates before they make what will remain long term. You leave it to where they have removed the function and it will be basically stuff you why are you tell us that you use that now. We don't have the time to make a proper interface so suffer. This is so hopefully either you leave the Linux world or the next you hit something like this you get on the mailing list raise problem and get fixed before major issues.

[GrayShade]

2003 form when __kernel_fpu_begin and __kernel_fpu_end entered kernel to 5.0 kernel there has been a open window to submit safe versions of these functions for non-GPL modules.

kernel_fpu_begin and kernel_fpu_end were switched to GPL exports in this commit from 2015.

This is so hopefully either you leave the Linux world or the next you hit something like this you get on the mailing list raise problem and get fixed before major issues.

Yeah, careful with that...

[oiaohm]

Using SIMD to compute the checksums is fast enough and is not covered by patents with exceptions for GPL code. This isn't about hardware accelerators. It's about arbitrarily denying non-GPL modules access to SIMD. Do you believe it makes Linux better for anyone?

I also missed another horrible fact. Using usermode-helper API to perform your SIMD or floating point stuff can in fact be faster than doing it in kernel space.

It all todo with the nasty overheads of kernel_fpu_begin and kernel_fpu_end like you are locking the scheduler so that you cannot perform perform interrupts on that cpu core while you are in between those two and you need todo that because you are in 1 set of page tables the kernel space page tables. Switching to userspace you switch page tables this changes fpu and simd issue problems so you can run interrupts while in the usermode helper.

1) usermode helper program does not have to be GPLv2 because its a normal program covered by Linus exception to gpl license.
2) There is no demonstration of need. There is no driver demoed on the Linux kernel mailing list showing that usermode-helper will not meet their SIMD or FPU requirement.
3) there has been no formal request until after the functions had been removed what is too late.

Big thing here Linux kernel developers cannot be expected to be mind readers about what third party driver makers need or are using. Third party driver makers for Linux have a responsibility to be in the Linux kernel mailing list raising what they require. If they have not been in the Linux kernel mailing list attempting to get what they need I show them no mercy when function removal happens.

[pal666]

zfs is stable kokoko