FreeBSD ZFS vs. ZoL Performance, Ubuntu ZFS On Linux Reference

[scineram]

They use UFS because they don't care for data integrity at the server/VM level. They are using Amazon datacenters to store their media and virtual servers anyway.

Where they do use ZFS. On Linux.

[Dedobot]

Where is the freaking illumos on the charts??? WTH?? SmartOS, OmniOS, Dilos, Openindiana.. OpenZFS..
Comparison with Solaris 11 proprietrary ZFS...
Very bad guys to intentionally FORGET form where Zfs came from..

Oracle Solaris is a free for testing and education- i.e. home use, but strictly prohibited from public benchmark. My own observation is its fastest zfs platform, no surprise. The big thing is not zfs itself but Solaris SMB and NFS fail sharing protocols.

[gclarkii]

They use UFS because they don't care for data integrity at the server/VM level. They are using Amazon datacenters to store their media and virtual servers anyway.

First off, they have servers/systems all over the place. Some back end workloads are at Google, others are with Amazon, those run Linux of one flavor or another. They run FreeBSD on their Open Connect Appliance which are at their foundation are a type of file server cache, think Squid on steroids.
Second, if I'm running a network/file cache, I'm not really worried about integrity so much as I am speed. Plus I can't remember the last time I had a disk die with ZERO warning, S.M.A.R.T. starts bitching long before it actually does die. I will admit however that I've only had 80 - 90 disks working for me over the last 5 years, someone from a server farm company might have seen instant death somewhere. That said, the OCA's are under constant monitoring; If a disk shows problems, they cut it out of the cache, and replace it with a spare. It's a cache, it will rebuild itself over the next couple of rebuilds. That is why they are running UFS, pure speed. On the other hand, I'm almost willing to bet they are running some type of mirroring/redundancy for the system drive.

As far as I know, their back end runs Linux of whatever flavor and so talking about AWS in regard to FreeBSD does not make sense. The way that last part sounds to me is that your saying the UFS usage does not matter as it all runs on AWS anyway, but it does not. Yes, the master content servers is at AWS, but it is the OCA's(which are FreeBSD based), which carry the heavy load for content distribution. And yes, every so often some knothead claims the OCA's no longer run FreeBSD, but as of February of 2019, they still do.

Take a look at the 2019 FOSDEM talk slides which shows one of their 40Gb/s Open Connect Appliances(with 249TB of storage). Talk about a file server from hell.
And if you really want to go fast, how about hitting 90Gb/s+ with TLS traffic. That thing blows the above OCA away. And they all run FreeBSD.

[Markore]

Oracle Solaris is a free for testing and education- i.e. home use, but strictly prohibited from public benchmark. My own observation is its fastest zfs platform, no surprise. The big thing is not zfs itself but Solaris SMB and NFS fail sharing protocols.

And this is what I call intentionally avoiding the issue. My comment is "illumos distros are out there to test and compare"
and you are talking about some other os..

[aht0]

What is that huge Linux - FreeBSD storage performance difference due to ?
In past I was used to see quite the opposite (Linux far ahead regardless filesystem), now maybe due to the new massive cpu security patches ?

FreeBSD has the same cpu security fixes applied.

[aht0]

And this is what I call intentionally avoiding the issue. My comment is "illumos distros are out there to test and compare"
and you are talking about some other os..

Test suite is open source. Feel free to test Linux_vs_bsd_vs_solaris out yourself. I am sure that extra 3-party content would be okay in Phoronix.

[Markore]

Test suite is open source. Feel free to test Linux_vs_bsd_vs_solaris out yourself. I am sure that extra 3-party content would be okay in Phoronix.

Again illumos is not Solaris. Those are 2 different OSes and platforms. illumos is opensource and free software , Oracle Solaris is proprietary
You are obviously refusing to acknowledge illumos distros existence. Please write it in the sentence, it's not hard. It is typed with small "i": illumos .