OpenZFS Merges The New FreeBSD Support

[jacob]

Lawrence Livermore National Laboratory is using CentOS and ZoL for their HPC storage.

Interesting, thanks for the info.

[mskarbek]

I don't know if anyone is really using ZFS with Linux in production though (I mean actual industrial or commercial production, not home NAS and similar projects).

You can ask Datto (native encryption sponsor), Intel (dRAID work) and Delphix (sponsor and OpenZFS Developer Summit organizer).

[CommunityMember]

Lawrence Livermore National Laboratory is using CentOS and ZoL for their HPC storage.

Last I knew (have not worked with them for a bit), they actually use (real) RHEL licenses., as they (and RH(*)) understand the advantages of direct support when you are running huge clusters (at the time they also had an on-site RH TAM, too, (who, at least at that time I was involved, had kernel expertise) to be able to get the right people engaged immediately within the larger RH company). While there are parts of LLNL that are interested in computer research, the point of the large computational power available at the national labs is to get science done, not tinker with the system(s), so a RH support contract makes sense.

(*) LLNL (and a sister lab or two) offers RH a somewhat unique opportunity to learn about certain types of very very large scale problems, and because of the huge scale, if some unusual edge case can happen, there is some chance it has been seen at places like the national labs.

[ehansin]

I work at a larger public university. There is a research computing division that provides a large scale storage solution for research data. They are running OpenZFS (ZoL) latest on top of RHEL 7.

On a FreeBSD note, I am excited about this. I've dabbled a little in FreeBSD, it isn't that difficult to me. Would I run FreeBSD over Linux as a ZFS storage solution? Not necessarily, but I suppose it would depend on the features I needed. I do like that FreeBSD is able to include ZFS in core without issues, and even more so like that the FreeBSD and Linux people are 'harmonizing' their ZFS code bases. The idea that you could drop a pool from one the other, and have it just work sounds cool to me, among other benefits.

Also, I traded a few emails with someone at iXsystems (very responsive, just a kudos on that!) recently, and he pointed me in the direction of getting the OpenZFS for FreeBSD (ZoF) code base running on FreeBSD. That was before all of this, which is a significant move in the right direction. He did mention, that he is all for seeing this move into a standard release sooner than later if possible. Not sure if it will make it into 13-RELEASE once it is released as 'stable' (their use of -STABLE on the next upcoming major version release confuses me sometimes!) Would be cool if it could though!

Couple more thoughts, here is the FreshPorts page for OpenZFS:

https://www.freshports.org/sysutils/openzfs/

You can compile the port or binary install via pkg. Also, there are some other things you need to do get get the OpenZFS kernel module to load, etc., which the linked on the front page mailing list post mentions:

https://lists.freebsd.org/pipermail/freebsd-stable/2020-April/092235.html

Lastly, I attended a talk by the guy who started the rsync.net cloud storage company, and they run on a pretty vanilla FreeBSD and ZFS setup. He really is into simplicity, where it can be achieved. Was very impressed with how he explained their architecture. He had nothing but good things to say about FreeBSD and ZFS in his experience, and this is the 'old' ZFS, hopefully once the 'new' ZFS goes mainline, it will be even better!

https://www.rsync.net/

[starshipeleven]

Would I run FreeBSD over Linux as a ZFS storage solution? Not necessarily, but I suppose it would depend on the features I needed.

FreeNAS/TrueNAS (iXsystems) is still the only good turnkey storage appliance OS to use if you want ZFS on a relatively small scale (i.e. not SAN scale) and don't like to DIY all the things and commandline your way into system configuration. Also with businness support contracts for those that employ it in businness environments.

[ehansin]

FreeNAS/TrueNAS (iXsystems) is still the only good turnkey storage appliance OS to use if you want ZFS on a relatively small scale (i.e. not SAN scale) and don't like to DIY all the things and commandline your way into system configuration. Also with businness support contracts for those that employ it in businness environments.

Just to emphasize, my interactions were super pleasant, and the person was very helpful. That alone makes me feel good about the company. It also looks like they are big contributors to the FreeBSD ecosystem, and at the time I installed the 'openzfs' package, I think they were the ones putting it forward to make it available via ports and pkg. Anyway, they seem like a good company. I'm not too caught up in the politics of operating systems, glad to try different things out. That does not say I do not have values in what I find important, but I'm not hard core about certain things. Glad to see all this happening, that is all I have to say about it.

[k1e0x]

Lawrence Livermore National Laboratory is using CentOS and ZoL for their HPC storage.

For most business outside of research there isn't a lot of benefit to them, and also security implications, for talking about their storage platforms. But, Yes I can tell you that even RedHat support will "support it", or get you back on your feet. (if you are a paying customer with a corporate contract that is).

[scineram]

FreeNAS/TrueNAS (iXsystems) is still the only good turnkey storage appliance OS to use if you want ZFS on a relatively small scale (i.e. not SAN scale) and don't like to DIY all the things and commandline your way into system configuration. Also with businness support contracts for those that employ it in businness environments.

Proxmox, Unraid are not a thing? What is wrong with Ubuntu?

[starshipeleven]

Proxmox,

Uses ZFS but it is not a storage appliance. It's a virtualization host OS. It's great and all, it's just not its job.

Unraid

does not support ZFS natively and it isn't really its main goal (it's about using their own RAID4-like implementation that isn't particularly better than mdadm), there is a plugin for ZFS support from some guy but it's command-line only, which makes it NOT turnkey.
Why the fuck should I be paying Unraid license to use ZFS with command line when I can have the same experience in any other distro for free?

What is wrong with Ubuntu?

It's not anywhere near turnkey? I mean did you look at FreeNAS web interface, plugins (also businness paid plugins) and whatnot? Does Ubuntu have that? No it does not. If I have to set it up with commandline I'd go with OpenSUSE, because the distro itself is plain better on all metrics and zfs is exactly the same on any Linux system anyway.

The only storage-oriented linux distro that has ZFS in plugins and is also turnkey enough is OpenMediaVault, but it's kind of meh in my experience. I consistently manage to break something within a few days after installing, by just installing some plugin or the other for stuff I needed, and I have to go in with commandline to fix it.

Another thing I'm readying my body for is the migration of the FreeNAS's main application (the web interface and core functionality that makes it a turnkey storage appliance) to Debian, which is what iXsystems said they are doing at the moment. Because that's the main selling point of FreeNAS. Being turnkey. ZFS is the same everywhere, especially now that they merged their codebase with ZoL.