Benchmarking ZFS On FreeBSD vs. EXT4 & Btrfs On Linux

[kraftman]

Oh, Kebbabert, it's funny SUN used such jerks to spread FUD about its competitors. SUN was a FUD company, but they're no more. Now, you're an alone FUDder, sorry gunman.

[kraftman]

No, you dont. You dont consider Linux to be credible, or Andrew Morton, or researchers or whatever.

Yes, I do. Stop lying about me. Stop spreading FUD also.

[kraftman]

Solaris completes all threads slow (not very slow)". But when we ramp up difficulty and many cores, then Solaris continues to work slowly but steadily. Linux chokes and can not scale as well. Some threads take very long time, others finish quickly.

Can you give me example? It reminds me some stupid bsd dev benchmark where they used broken thread library.

In short, I dont claim that Solaris code is bug free. I dont think that Linux developers are bad, I believe they write good code. The problem is Linux develeopment model where he deletes code for new code all the time: "Linux evolves like in biology, it gets slowly better for each generation". This means that source code are always new and buggy. It takes long time to get mature bug free code. This is the reason Linux has lots of problems. With for instance, scalability.

It's the typical mistake people do. You compare Solaris (which is an operating system) development to Linux (a kernel) development. Afaik these, so called stable kernels at kernel.org are just snaphots of the code which is developed continuously. Some people can differ, but I consider (and I'm according to Greg KH talk) there are no more stable releases (I can't define stable term in this case). You should rather compare Solaris to Debian, RHEL or to SLES (they all use well tested kernels and not the newest snapphots). I don't see where Linux has problems in scalability. I hope you're not according to well known problem with GNU's malloc library.

In a few years, Oracle will release a SPARC machine with 16.384 threads and 64TB RAM. Each thread is treated almost like a cpu. This means Solaris runs a 16.000 cpu machine. On one single server. That is good scalability.

No, it doesn't mean Solaris runs a 16.000 CPUs machine, you're not sane. If they will succeed it will run only 128CPU machine.

Linux on the other hand, scales very well horizontally (in a network, such as Google having 10.000 PCs on a network and runs Linux kernel, or super computers doing calculations).

It also scales on incredibly bigger machines then Solaris, because of limitations in Solaris kernel. Vertically Linux scales up to 4096 physical CPUs while Solaris scales only up to 64 physical CPUs.

But vertically, Linux scales very bad. Linux has problems using many cpus on one single machine. I linked to a article where three Linux scalability experts from RedHat, etc - dispelled the FUD from Unix vendors that Linux scales bad. The scalability experts said "Linux scales super, for instance Google has 10.000 computers with Linux. Linux true strength is in horizontal scalability (large network). But also vertically Linux scales good (on one single computer). For instance in this v2.4 Linux uses 4 cores without problems, and in next v2.6 Linux will use 16 cores! Now, that is goooood scalability!". You see that they think that 16 cores in one single computer is good scalability - whereas all Unix vendors considers that very bad scalability. (But no one denies Linux scales good horizontally). Here is the article:
http://searchenterpriselinux.techtar...ux-scalability

Sorry, but this is bull and FUD. This is obsolete, because they're talking about Linux 2.6 and Linux since kernel 2.6.2x scales up to 4096CPUs on a single computer.

And I saw talk about SGI having a computer with 1024 cpus that Linux runs on. That computer is quite special, it behaves like PCs on a network, it is not a single computer. It has something like 128 nodes, and all benchmarks always are using 128 instances of the work load, one for each node. Basically, it is just a cluster, some PCs on a fast switch. Here is more technical information on the SGI computer:

http://www.c0t0d0s0.org/archives/6751-Thank-you,-SGI.html

If you dont understand what he is talking about, it is ok. But try, and you will see why all SGI benchmarks always are parallell tasks that would equally be good in a cluster with 1024 cpus. This is not a vertical scaling machine.

I showed you SGI computer using 256nodes, so four times bigger machine then Solaris can handle.

[kebabbert]

I consider Linus is very credible.

No, you dont. You dont consider Linux to be credible, or Andrew Morton, or researchers or whatever.

If Linus T wrote here "sorry, I have read this thread and unfortunately Kebabbert is correct. Linux scales good horizontally but not so good vertically, but we are working on it" I promise you that Kraftman and all other Linux fan boys would not believe that. When earlier Linus T, Andrew Morton, etc - all said bad things about Linux then the Linux fanboys rejected everything what they said. I showed such negative links from several different Linux developers, and they were all rejected by you.

Therefore, I can not prove anything to you, Kraftman. When even Linus T says something bad about Linux, you reject it. If I where to bring Linus T with me, and we knocked on your door and Linus T explained to you that I was correct - you would not believe me. There is NO WAY I can make you accept any links I provide. Not even from Linus T or Andrew Morton.

Therefore, this debate is over with you, Kraftman. Even when I am correct you reject it. You have showed this time and time again. I have posted several research papers - and you STILL deny I post any research papers - you say I lie, that I never posted any papers at all.

I get a strange feeling:
-Look, it is raining
-No.
-You are wet, you just wiped rain from your face
-No, I didnt
-Yes you did! I saw it! You ARE wet.
-No I am not.
...

Kraftman, you are not serious. I am not going to waste more time on you. You have confessed you FUD. I have showed you lie about me. I dont see why I should debate with someone who FUDs and lies? You are not serious, you are just Trolling.

Good bye. Now I have to deal with the smitty3283 person who also I have quoted when he lies.

But I work and there are lots of posts here, so I will handle posts slowly, on a FIFO basis. I will select some serious posts to answer to. Pure Trollisms and Lies will be ignored. I can not waste too much time on you linux fanboys that confessed you FUD and confessed you make up things, and I have proved you lie.

I will post later. But not to you, Kraftman. You had your time to say something and I listened patiently to you to see if you had something interesting to say, but you didnt care and instead called me names: "idiot", "moron", "troll", etc. Actually, that is not an ok thing to do to people, Kraftman. It is not ok behaviour.

But of course, if you have something serious to say, I will continue debating with you. But please, be serious then.

Stay tuned, the rest of you.

[kebabbert]

Regarding bad code in Solaris.

A) Here is a Linux developer that says that Solaris code is bad. He seems to know what he is talking about, so I suspect he is true. I think the link is credible, and his criticism is valid. (The thread is ~15 years old).

http://cryptnet.net/mirrors/texts/kissedagirl.html

B) Also, Kraftman actually posted a link showing that Solaris had a problem with... forking(?) processes? The solaris user said he could see processes being forked - which indicates something is strange because forking is normally too quick to see text output. The Solaris developer says something like "we have not focused on unimportant things, we have focused on the big picture in Solaris" - so he confesses that Linux may be faster on certain things. I dont doubt that. Many people say that Linux is fast on single threaded things. One guy said something like "Linux completes some threads very fast, and others very slow. Solaris completes all threads slow (not very slow)". But when we ramp up difficulty and many cores, then Solaris continues to work slowly but steadily. Linux chokes and can not scale as well. Some threads take very long time, others finish quickly.

In short, I dont claim that Solaris code is bug free. I dont think that Linux developers are bad, I believe they write good code. The problem is Linux develeopment model where he deletes code for new code all the time: "Linux evolves like in biology, it gets slowly better for each generation". This means that source code are always new and buggy. It takes long time to get mature bug free code. This is the reason Linux has lots of problems. With for instance, scalability.


In a few years, Oracle will release a SPARC machine with 16.384 threads and 64TB RAM. Each thread is treated almost like a cpu. This means Solaris runs a 16.000 cpu machine. On one single server. That is good scalability.

Linux on the other hand, scales very well horizontally (in a network, such as Google having 10.000 PCs on a network and runs Linux kernel, or super computers doing calculations).

But vertically, Linux scales very bad. Linux has problems using many cpus on one single machine. I linked to a article where three Linux scalability experts from RedHat, etc - dispelled the FUD from Unix vendors that Linux scales bad. The scalability experts said "Linux scales super, for instance Google has 10.000 computers with Linux. Linux true strength is in horizontal scalability (large network). But also vertically Linux scales good (on one single computer). For instance in this v2.4 Linux uses 4 cores without problems, and in next v2.6 Linux will use 16 cores! Now, that is goooood scalability!". You see that they think that 16 cores in one single computer is good scalability - whereas all Unix vendors considers that very bad scalability. (But no one denies Linux scales good horizontally). Here is the article:

http://searchenterpriselinux.techtarget.com/news/929755/Experts-Forget-the-FUD-about-Linux-scalability

And I saw talk about SGI having a computer with 1024 cpus that Linux runs on. That computer is quite special, it behaves like PCs on a network, it is not a single computer. It has something like 128 nodes, and all benchmarks always are using 128 instances of the work load, one for each node. Basically, it is just a cluster, some PCs on a fast switch. Here is more technical information on the SGI computer:

http://www.c0t0d0s0.org/archives/6751-Thank-you,-SGI.html

If you dont understand what he is talking about, it is ok. But try, and you will see why all SGI benchmarks always are parallell tasks that would equally be good in a cluster with 1024 cpus. This is not a vertical scaling machine.

[kraftman]

Stupid time limit:

"Why shouldn't I consider Linus isn't credible?"

I consider Linus is very credible.

[kraftman]

Again, it is about credibility. I have told you, if you want to show something bad about Solaris, then you have to give credible links, such as research papers, Solaris developers, etc and other credible links. You can not give links to Solaris competitors - that is simply not credible. That is the reason I quote Linus when I want to show something negative about Linux. Or when I quote Linux scaling experts, when I want to show how bad Linux scales. I would avoid quoting Solaris experts, that is simply not credible when we talk about Linux crappy code.

You quoted lame Bonwick etc. Credible link shows Solaris scales only up to 64CPUs. Linux scales up to 4096CPUs. When Solaris will catch up? Rhetoric question of course, it will probably never catch up. Sun devs lie, FUD and SUN was a company, so its employers couldn't say things which could be bad to company. They're not trustworthy.

So, when you quote Linus Torvalds, who said "But I am in active competition with Sun, so I hope Solaris dies" - do you really think that is credible when he says "Linux has better code than Solaris"?

Yes. You think sun blogs, devs are credible. Why shouldn't I consider Linus isn't credible?

[kebabbert]

Ah, kebabbert - are you even reading the links posted? That test showed that ZFS was faster than BTRFS, once you added in enough disks, but that with a few tweaks and a newer kernel BTRFS was quickly catching up.

I just skimmed the link. I assumed that BTRFS was faster, because ZFS basher Jimbo posted the link. But now you say that ZFS is faster. I dont really understand why ZFS basher would post such a link?

As for performance, I dont really care. I do suspect that BTRFS gets faster than ZFS when it is done. As a research paper explained: to get data safety you need to trade cpu performance and do lots of calculations on checksums. That is one of the reasons ext3 are faster than ZFS - they dont do as many checksum calculations as ZFS. That is also one of the reasons ZFS is much safer. I dont care about speed. If BTRFS is faster - congratulations.

Also, how many disks are required to be in a machine before it counts as "enterprise" in your mind? 16 high-end SSDs sounds like quite a bit to me, they probably have the capabilities of 3 times that many traditional HDDs. More than that is quickly becoming very, very, specialized and marginal market.

Yes, and that very very specilized market we talk about is the Enterprise market. For desktops, BTRFS may be faster. It is ok with me, and I will not dispute it. But go into Enterprise realms, and see how you can get extreme speed with ZFS if you wish.

Because they don't need one?

Sun was the sole developer of ZFS, so they had to devote large amounts of resources to it's development. BTRFS @ Oracle was a completely different situation. Oracle took the lead in developing it, but they were also able to draw upon resources from Red Hat, IBM, and many other places throughout the community, making a larger in-house team by Oracle unnecessary.

As we have seen, Oracle likes to get total control over the products they sell. Oracle is closing parts of Solaris 11, for instance. That is the reason I ask why Oracle does not exert more control over BTRFS?

[smitty3268]

I am only asking, "if Oracle is serious with BTRFS, why dont they assign a large team in-house at Oracle"?

Because they don't need one?

Sun was the sole developer of ZFS, so they had to devote large amounts of resources to it's development. BTRFS @ Oracle was a completely different situation. Oracle took the lead in developing it, but they were also able to draw upon resources from Red Hat, IBM, and many other places throughout the community, making a larger in-house team by Oracle unnecessary.

[smitty3268]

Ok, so what are you trying to prove here? That ZFS is slower? Ok, it may be slower, according to this test. It is not really important to me, who is fastest.

And also, your post uses only 16 drives. That is nothing when we talk about Enterprise storage halls. You dont see such tiny machines in Enterprise halls. ZFS is for Enterprise halls, with many discs. Then it excels. It may be slow on desktop. That is ok with me. I dont mind if ZFS is slow on desktop, as long as my data are safe.

Ah, kebabbert - are you even reading the links posted? That test showed that ZFS was faster than BTRFS, once you added in enough disks, but that with a few tweaks and a newer kernel BTRFS was quickly catching up.

Also, how many disks are required to be in a machine before it counts as "enterprise" in your mind? 16 high-end SSDs sounds like quite a bit to me, they probably have the capabilities of 3 times that many traditional HDDs. More than that is quickly becoming very, very, specialized and marginal market.