Open ZFS File-System Running On Windows

[Gphoronix]

Hi trying my first post the last was unapproved for some reason. I'm very frustrated because I upgraded an HP Z220 for ECC with mirror refs storage and MS pulled the plug for full refs support for Windows 10. My requirements of photo processing and storage are mainstream app support, data checksums, and reasonably affordable hardware selection. Hardware and mainstream app support nearly rule out Mac and apfs does not have data checksum anyways so it's actually strike 3 for Mac. Linux does not have mainstream app support. I did use Windows for processing and BSD/ZFS for network storage but really it's impractical to boot 2 computers networked every time I wish to process and store photos. I would hope Windows would gain ZFS support or develop supported Linux compatibility layer. Linux compatibility layer would seem to be double suicide, 3rd party and OS for MS but they seem to be abandoning phones and desktops anyways and they have always sold software side by side with 3rd party apps so people may as well be able to run Windows apps and cloud services, if they cannot use Windows desktops and phones. So please MS support ZFS and/or a genuine supported Linux compatibility layer.

[Gphoronix]

Hi first Phoronix post. I want a data checksum file-system with mainstream app support for photo processing and storage. I recently converted my HP Z220 to ECC with a processor and memory upgrade added 2 WD red hard drives and installed Windows 10 pro, right afterword turns out MS is dropping full REFS support, to say the least I'm extremely disappointed. I tried OSX on a MAc mini recently and found the hardware options unusable for my needs and the software situation extremely undesirable but workable if necessary. I had been using a Z220 workstation with HP XW6600 with Freenas ZFS with good performance but little practicality to boot up 2 computers each time I wanted to process or even just store photos, so I also purchased a Dell T20 server and am running Kubuntu with ZFS for storage and browsing, although Freenas and ZFS was better performance and reliability constantly booting two computers and syncing them is just not practical. Hoping for sudden Mac mainstream app support is unrealistic and Linux seems to be moving even further away from being a practical desktop for mainstream use outside of a web browser and specialty applications. Windows needs to expand refs support not limit it to business and education, or support Linux compatibility layer and sell MS apps and cloud, or adapt ZFS or at least show some official support. BTRFS could have been a good multi-platform file-system but seem to be a tangled mess not ready for production use any time soon if ever. APFS does not even have data checksums even if I was retarded enough to spend about 3-5x the cost of an equivalent Windows system and be locked into their system with little support for hardware that is not nearly new and little mainstream app support. Why can't we have an OS with a modern filesystem, mainstream app support and option to install on wide variety of hardware, coming into 2018, when PCS have been common for over a decade. Linux seems to be self destructing and moving further away from mainstream desktop use instead of closer with no serious graphics support in discussion let lone on the horizon, even though I use Linux 95% of the time it's as a web browser and document/pdf storage end even then it's has a lot of compatibility issues with rtf, word, utf-8 etc. So please Windows ZFS and/or real MS app support for Linux, as MS seems to be abandoning desktop and phone. PS I have Windows phone also and hate Android and Apple so I have a phone dilemma also.

[AndyChow]

It's not quite clear from your phrasing ("I still can't believe that Windows and MacOS don't have checksum metadata as basic requirements for their FS") what you mean, but to clarify, APFS DOES have checksumming over the FS metadata, but NOT over user data.

This is obviously not ideal. MY GUESS (and this is only a guess) is that it's a transitional issue.
One problem with user data checksumming is where to store the checksums. (This is not an issue for metadata checksums because you can set the size of each metadata structure as you like). If you control hardware, then you can add extra padding to each disk sector or flash page to hold a checksum. (And of course such padding exists and is used, at the HW level, but Apple can't ask for that padding to be made larger.)
Otherwise you have to store the checksum somewhere else on the storage device and that presents performance problems. In particular it means that writes will be generating writes to two places (data and checksum) and you have to wait for both of those to reach persistence before the write can return. If you're ZFS and targeting huge arrays, this is likely not an issue. If you're Apple and targeting single disks, it is an issue --- obviously for HDs, but even for SSDs because of the suckiness of writes on SSDs. You can punt and not force the checksum writes to disk at the same time as the user data writes --- but then what's the point? If you're not ordering writes carefully and enforcing ACID, you've already decided you don't care THAT MUCH about safety...

So Apple is (right now) screwed because of HW limitations. Likewise for MS. How does ZFS do it?
There is something about this here

https://blogs.oracle.com/bonwick/zfs-end-to-end-data-integrity

with some pictures here

http://wiki.lustre.org/images/0/00/Tuesday_shpc-2009-zfs.pdf

and I have to admit the idea is VERY nice! So nice that I fear it is patented, and Apple and MS can't use anything similar unless they want to pay Oracle a whole lot of money :-(
Also what are the performance implications on a single commodity drive?
The claim here

https://forums.freenas.org/index.php?threads/single-drive-zfs.35515/

(I have no idea as to its truth) is that ZFS on a single drive is ridiculously dangerous. Basically it's optimized for a very different use case, so while one can consider trying to use some of its most clever features when targeting a single drive FS, the way to do so is not completely obvious.

Like I say, my guess is that the long term Apple plan is to provide HW without those limitations. This might take the form of using flash that's been engineered to provide some extra storage space. (Being part of the Toshiba acquisition group could help with that...) Or it might take the form of APFS being engineered mainly for nvRAM, with the flash and HD forms as transitional and the full set of ideas optimized for nvRAM. This may seem a stretch, but Apple says, for example, that APFS uses a "novel copy-on-write metadata scheme", and these sorts of novel schemes are part of the thinking around nvRAM and the way its byte addressability allows for different types of data structures.

(If you read about experiences with ReFS on the desktop, you see the same sorts of issues. For example ReFS is SUBSTANTIALLY slower than NTFS [like less than half the write speed] on a single spindle, presumably because of how it stores integrity data.)

You are correct. But checksums on metadata and not data means .. almost nothing. "The structure is good, but the data is maybe okay."

On a 5 TB array, the checksums on data is probably less than 500MB. Probably even much less. Any writes on data checksums would simply be ridiculously unnoticeable unless you are experiencing high queue depths, which is rare. A dedicated partition could easily be silently implemented, and writes to it buffered out in free time. Not writing checksums on data at the same time makes sense, because often your drives will be free. You could even have a parameter to enforce "stickiness". And if the checksums isn't written, it doesn't mean your data is bad, simply that it's not certified "integral".

I would agree with your point that this is a transitional issue, but how much longer must we wait? Computers have increased their capacities several thousand times by every metric. Checksums, take CRC, was invented in 1961. We are now 56 years later, encrypted drives, redundant raids, redundant ethernets even, but no redundant checks on data integrity on common systems? Nothing to let you know "Hey, this file here, it might still be usable, but it appears corrupted".

Modern flash implements huge amounts of error-correction algorithm. But we have no similar options on the top-level data as is routine on hardware level. ZFS and BTRFS are fine for expert users, but why can't the masses have such assurances? Apple's APFS is a great step forward, but was released last year only. And on Windows, I am unaware of similar technology for common users. ReFS I believe was removed from direct use and reserved to premium server licences.

I see it as a virtually fundamental need, but it is stubbornly denied to basic users, who are even more likely to be harmed from data loss than more "advanced" users, which probably already use scripts to generate SHA and MD5 lists on their data, of which they have multiple backups, and versioning.

[SystemCrasher]

Reactos seems a possible target

Sounds like a perfect setup for someone who has decided to go nuts for the real.

[k1e0x]

ZFS is fine on a single drive pretty much. I believe it does copies=3 for it's uber block so it can correct itself in some situations but data lost will be data lost like any other FS. In some ways it can fail gracefully as opposed to other FS because it will know it's bad instead of just sending you corrupt files silently. So at least it can tell you to restore your backup.

[name99]

License wise, I think it might be easier to have ZFS on Windows than Linux. I think this is interesting. I still can't believe that Windows and MacOS don't have checksum metadata as basic requirements for their FS. And I'm not talking CRC. How is it that we still treat digital data as a basic sheet of paper? ZFS and BTRFS aren't advanced FS, they are basic FS. My first job out of university, I was an actuary for a pension fund. We had all (paper) files in quintuplicate (5 copies, not 15), with the 4 extras send at different archive places. Because 40-50 years from now, we might need the actual original agreements and proof of signatures.

Data space is cheap, I don't know why we haven't adopted such measures for our everyday data.

It's not quite clear from your phrasing ("I still can't believe that Windows and MacOS don't have checksum metadata as basic requirements for their FS") what you mean, but to clarify, APFS DOES have checksumming over the FS metadata, but NOT over user data.

This is obviously not ideal. MY GUESS (and this is only a guess) is that it's a transitional issue.
One problem with user data checksumming is where to store the checksums. (This is not an issue for metadata checksums because you can set the size of each metadata structure as you like). If you control hardware, then you can add extra padding to each disk sector or flash page to hold a checksum. (And of course such padding exists and is used, at the HW level, but Apple can't ask for that padding to be made larger.)
Otherwise you have to store the checksum somewhere else on the storage device and that presents performance problems. In particular it means that writes will be generating writes to two places (data and checksum) and you have to wait for both of those to reach persistence before the write can return. If you're ZFS and targeting huge arrays, this is likely not an issue. If you're Apple and targeting single disks, it is an issue --- obviously for HDs, but even for SSDs because of the suckiness of writes on SSDs. You can punt and not force the checksum writes to disk at the same time as the user data writes --- but then what's the point? If you're not ordering writes carefully and enforcing ACID, you've already decided you don't care THAT MUCH about safety...

So Apple is (right now) screwed because of HW limitations. Likewise for MS. How does ZFS do it?
There is something about this here

https://blogs.oracle.com/bonwick/zfs-end-to-end-data-integrity

with some pictures here

http://wiki.lustre.org/images/0/00/Tuesday_shpc-2009-zfs.pdf

and I have to admit the idea is VERY nice! So nice that I fear it is patented, and Apple and MS can't use anything similar unless they want to pay Oracle a whole lot of money :-(
Also what are the performance implications on a single commodity drive?
The claim here

https://forums.freenas.org/index.php?threads/single-drive-zfs.35515/

(I have no idea as to its truth) is that ZFS on a single drive is ridiculously dangerous. Basically it's optimized for a very different use case, so while one can consider trying to use some of its most clever features when targeting a single drive FS, the way to do so is not completely obvious.

Like I say, my guess is that the long term Apple plan is to provide HW without those limitations. This might take the form of using flash that's been engineered to provide some extra storage space. (Being part of the Toshiba acquisition group could help with that...) Or it might take the form of APFS being engineered mainly for nvRAM, with the flash and HD forms as transitional and the full set of ideas optimized for nvRAM. This may seem a stretch, but Apple says, for example, that APFS uses a "novel copy-on-write metadata scheme", and these sorts of novel schemes are part of the thinking around nvRAM and the way its byte addressability allows for different types of data structures.

(If you read about experiences with ReFS on the desktop, you see the same sorts of issues. For example ReFS is SUBSTANTIALLY slower than NTFS [like less than half the write speed] on a single spindle, presumably because of how it stores integrity data.)

[aht0]

MS could simply block OpenZFS with the expedient of making use of unsigned drivers impossible. Even at the moment, using unsigned drivers is pita.

[k1e0x]

OpenZFS contains a lot of code contributed by Oracle under the terms of their license which is incompatible with the GPL. The same license therefore also applies to OpenZFS, with the added problem that copyright is spread out between a lot of contributors who would have to agree to a change in license.

Yeah pretty concise lay of the licence land for it. As far as Oracle as a controlling body is concerned they don't have an open source version of ZFS and don't control OpenZFS (that would lie with the contributors of Illumos, FreeBSD and ZoL). I think Oracle is starting to wake up and realize what they have.. and that can't be good.

I wonder how the Linux people feel about OpenZFS in Windows. It's been a feather in the cap if Linux/Unix for some time now but they haven't embraced it and they are loosing that advantage over Microsoft now.

[nils_]

I don't understand what Oracle has to do with OpenZFS.. Oracle could open up their version of ZFS.. but who would even want it? Oracle's version sucks and it's way behind OpenZFS. I really don't see these projects getting back together.

OpenZFS contains a lot of code contributed by Oracle under the terms of their license which is incompatible with the GPL. The same license therefore also applies to OpenZFS, with the added problem that copyright is spread out between a lot of contributors who would have to agree to a change in license.

[k1e0x]

I don't understand what Oracle has to do with OpenZFS.. Oracle could open up their version of ZFS.. but who would even want it? Oracle's version sucks and it's way behind OpenZFS. I really don't see these projects getting back together.