ZFS On Linux 0.8.1 Brings Many Fixes, Linux 5.2 Compatibility Bits

[starshipeleven]

I wanted to add, my early post in this thread asking if Linux has anything else comparable was meant kind of as a poke, but equally so a legitimate question. I'm getting deeper and deeper into file storage stuff, and I want to learn. Obviously there are a lot of opinions, and one thing I have learned in the Phoronix forums is that I need to use my critical thinking skills to come to my own conclusions. Anyway, I have learned a bit here and that is helpful.

Basically the only other one-stop-shop filesystem that competes with ZFS is btrfs, but it's not really finished yet.

[starshipeleven]

At this point in time, anybody still fussing over ZoL, CDDL, and the GPL are just showing that they haven't researched this topic in-depth, that they haven't even read the licenses, or that they're just repeating propaganda because they lack critical thinking skills to do tasks like researching and comprehensive reading.

The usual IANAL disclaimer should apply.

[oiaohm]

ZFS gives you a lot of ability to implement the full stack on your own (tho you may want a cluster FS on top of an array of block devices) In the open source world it's lacking good tooling and UI's with the exception of FreeNAS that is.. getting better.. but still not enterprise. Again I mentioned the commercial products before that are based on it that are enterprise ready ZFS solutions..

So https://stratis-storage.github.io/faq/ Stratis from Redhat and EMC is not good tooling.

People can cobble together things into a product and that's been done for ages.. Such as NetApp uses and old forked version of FreeBSD with UFS. With ZFS much more of the stack is integrated and works with other systems since it's all open.

Notice you are not talking about EMC any more.

EMC and Redhat are both using the Linux kernel as their full stack. ZFS on Linux you now have two stacks.

https://lwn.net/Articles/747633/

Everything started changing in the Linux kernel at the start of last year. The big advantage of having your complete stack was reducing the odds of hitting ENOSPC errors. But this did not make it impossible particularly once you get into virtual machine instances or iscsi and the like that support thin provisioning.

XFS developer instead of integrating raid, volume management and so on like they he had in the past this time decide hey I will fix the block layer to provide the information file system need and accept data how file system needs as well and give a safe ENOSPC. Safe ENOSPC pass to block device a list of operating that all need to happen or all need to fail this fixed up the major reason for ZFS using CoW.

Now XFS choice means the option of using hardware accelerated raid is still there and better than before. Yes BTRFS and ZFS both have trouble using hardware accelerators coming from implementing their own unique stacks.

Implementing the full stack on your own means you have NIH(Not Invented Here) Illness. As in if it not invented by you the end users cannot use it. Both BTRFS and ZFS suffer from this. Its also simple to forgot XFS developers attempting the path before XFS was ported to Linux. XFS development team has a lot of experience under their belt.

I will be truthful Linux block layer in reality had been broken even since thin provisioning had been added and it took the xfs lead developer to fix it.

Yes lots of people put ZFS up as this magic solution. ZFS and BTRFS as a solution lacks a lot of flexibility due to the single stack problem.

It does not pay to reinvent the wheel in most cases and that is exactly what ZFS and BTRFS have done in implementing raid and volume management.

Basically the cobble together arguement is excuse to ignore the defects of reinventing the wheel has caused ZFS and BTRFS.

[oiaohm]

Basically the only other one-stop-shop filesystem that competes with ZFS is btrfs, but it's not really finished yet.

Important question is should one-stop-shop file systems exist or is the XFS route with https://lwn.net/Articles/747633/ and https://stratis-storage the correct way forwards.

I would say ZFS is a prototype and prototypes all have major design defects. I have a horrible feeling that both ZFS and Btrfs are going down completely the wrong path.

[aht0]

Now XFS choice means the option of using hardware accelerated raid is still there and better than before. Yes BTRFS and ZFS both have trouble using hardware accelerators coming from implementing their own unique stacks.

Implementing the full stack on your own means you have NIH(Not Invented Here) Illness. As in if it not invented by you the end users cannot use it. Both BTRFS and ZFS suffer from this. Its also simple to forgot XFS developers attempting the path before XFS was ported to Linux. XFS development team has a lot of experience under their belt.

Actually you can use both for a time over RAID controller, it just won't end up well.

ZFS, might not have been NIH so much as attempt of offering it's users cheaper option hardware-wise. Back then hw RAID was built using rather expensive SCSI. Not that much time between designing ZFS and spread of ATA-RAID.

[oiaohm]

Actually you can use both for a time over RAID controller, it just won't end up well.

Solution Raid on top of Solution Raid is mostly never a good thing the increased latency between data being sent to drive and write is bad.

ZFS, might not have been NIH so much as attempt of offering it's users cheaper option hardware-wise. Back then hw RAID was built using rather expensive SCSI. Not that much time between designing ZFS and spread of ATA-RAID.

Like it or not ZFS on Linux is NIH they are basically keeping Solaris like lower level structures instead of using the Linux kernel or freebsd kernel provided ones. This is caused by the CDDL license a lot.

Reason for using a hardware raid is to get increased performance.

Do note I said hardware accelerated raid not hardware raid there is a difference. Hardware accelerated raids like for checksums instead of processing in CPU/FP you might be offloading to encryption processor or FPGA board.... There are a lot of ways to hardware accelerate your raid processing without having a hardware raid. Of course ZFS and BTRFS design makes it really hard to take full advantage of this stuff. Mind you BTRFS uses more standard Linux kernel encryption parts so it does pick up accelerated checksum processing when Linux kernel has that as a option.

[starshipeleven]

Important question is should one-stop-shop file systems exist or is the XFS route with https://lwn.net/Articles/747633/ and https://stratis-storage the correct way forwards.

I still don't think that a layered system is going to beat the performance of a one-stop-shop, but I'm not partial to anything. Whoever wins I'll take.

It's very cool that, with dm-integrity, Stratis is back in the race, as with that it can checksum the data and correct it (if there is a RAID providing redundancy).

[starshipeleven]

Actually you can use both for a time over RAID controller, it just won't end up well.

If your RAID controller corrupts your arrays it's shit and should be thrown off the window.

running btrfs or ZFS over a hardware raid card isn't different than running any other filesystem.

[starshipeleven]

Mind you BTRFS uses more standard Linux kernel encryption parts so it does pick up accelerated checksum processing when Linux kernel has that as a option.

Yes btrfs was reworked around a year ago to actually offload to the kernel, so whatever accelerator you can see with the kernel will be used.
then again it's just crc32, it's not a particularly strong algorithm. It's just there for data integrity, not encryption.

[skeevy420]

The usual IANAL disclaimer should apply.

Yeah, but still...

While the CDDL allows a project more freedoms like using a closed source compiler, there is no reason a project can't be compatible with the GPL if they're following the GPL requirements.

With how the CDDL allows the license to be changed upon code compile, there is no reason why a special CDDL to GPL Compat License can't be used upon compile if the CDDL project is following what the GPL requires; especially by adopting a guideline requiring the code in a GPL compatible state that all code contributors are expected to follow (basically, leave the entire build system open source). That, theoretically, solves the ZoL dilemma.