FreeBSD 8.0 vs. Ubuntu 9.10 Benchmarks

[kraftman]

Allow me to correct you:

"As of Linux 2.6.22 (Fedora 8) they have fixed the most serious scaling problem [...]. Linux 2.6.23 incorporated a new "Completely Fair" Scheduler which performs significantly worse than 2.6.22 on this workload. 2.6.24 has not yet been evaluated."

I was according to this:

http://people.freebsd.org/~kris/scaling/os-mysql.png

If 2.6.22 was tested here and there (also in benchmark I posted) I wonder what they base on saying some scaling problems were fixed in 2.6.22?

Ok, now I know:

http://jeffr-tech.livejournal.com/5705.html

There's gnu's malloc used and that's why 2.6.21* drops down.

And with Google's malloc:

http://ozlabs.org/~anton/linux/sysbench/

Seriously, it seems a fair test to me, though it's rather old and many things might have changed in both OSes. According to the link you posted 2.6.25 was already much improved.

It could be like this. However, there's/was known problem with GNU's malloc library and I wonder what library was used (it seems it was used). There's also 2.6.22 test which I'm according to.

[clau]

Don't forget to use Google's malloc in this case. Hehe I thought about this benchmark before :P

http://www.kernel.org/pub/linux/kernel/people/npiggin/sysbench/

Bsd guys didn't even post configs, apps versions, they write linux not Linux, not nice :P

Allow me to correct you:

"As of Linux 2.6.22 (Fedora 8) they have fixed the most serious scaling problem [...]. Linux 2.6.23 incorporated a new "Completely Fair" Scheduler which performs significantly worse than 2.6.22 on this workload. 2.6.24 has not yet been evaluated."

They *have* written Linux with capital L

"MySQL 5.0.51 is currently used for performance comparisons."

There you have the MySQL version.

Seriously, it seems a fair test to me, though it's rather old and many things might have changed in both OSes. According to the link you posted 2.6.25 was already much improved.

Arguably, the omitted sysbench version is less important, since the same version was used.

Bottom line, I'd like to see such a test on newer releases.

[kraftman]

Anyway, if you ever get them up and running, please consider running the benchmark presented here:
http://people.freebsd.org/~kris/scaling/mysql.html

Don't forget to use Google's malloc in this case. Hehe I thought about this benchmark before :P

http://www.kernel.org/pub/linux/kernel/people/npiggin/sysbench/

Bsd guys didn't even post configs, apps versions, they write linux not Linux, not nice :P

[kraftman]

I can't comprehend how you could "cheat" with a VM, but fine.

Like here in SQLite:

http://www.phoronix.com/scan.php?page=article&item=linux_2631_kvm&num=3

I could do a bare metal then, but wouldn't be able to do a plethora of options. I'd only have time to do one FreeBSD test and one optimized Ubuntu test (9.10 desktop or 9.04 server?)

Maybe compare something recent? If you want test FreeBSD8 then test Ubuntu 9.10 maybe?

[clau]

First, you misunderstand. You can't run anything in a host OS of ESXi, since it doesn't have anything more swift than a busybox of the needed programs to allow you to manage the ESXi box. So every test (FreeBSD, Ubuntu, etc) would be running as a guest OS in the VMWare server and only one VM would be running (the one testing) simultaneously.

Second, pretty much no one uses bare metal now. Do you? For servers? I've got two racks of machines that I am responsible and not a one is bare metal. By your comments, it sounds like you do. But I don't at work or home.

Thanks for the explanation. And yes, I usually use bare metal, but on the other hand I'm not dealing with lots of servers and neither with high performance hw.

Anyway, if you ever get them up and running, please consider running the benchmark presented here:

http://people.freebsd.org/~kris/scaling/mysql.html

Instead of running FreeBSD, Linux, DragonflyBSD and NetBSD, you could try just FreeBSD 7.2, 8.0 and a Linux distro with a recent kernel.

[risner]

What I know OS'es in vm show better numbers in Phoronix Test Suite in some benchmarks. Maybe they're cheating there somehow (and maybe one can cheat more then another? ; D)? It will be probably hard to setup a fair test - Mtippett post.

I can't comprehend how you could "cheat" with a VM, but fine.

I could do a bare metal then, but wouldn't be able to do a plethora of options. I'd only have time to do one FreeBSD test and one optimized Ubuntu test (9.10 desktop or 9.04 server?)

I can hand someone remote console to the system (IPMI IP-KVM) bare metal with the CD in the cdrom. Anyone want to help by doing the Ubuntu side?

[kraftman]

First, you misunderstand. You can't run anything in a host OS of ESXi, since it doesn't have anything more swift than a busybox of the needed programs to allow you to manage the ESXi box. So every test (FreeBSD, Ubuntu, etc) would be running as a guest OS in the VMWare server and only one VM would be running (the one testing) simultaneously.

What I know OS'es in vm show better numbers in Phoronix Test Suite in some benchmarks. Maybe they're cheating there somehow (and maybe one can cheat more then another? ; D)? It will be probably hard to setup a fair test - Mtippett post - and thus raw benchmark numbers like in the current comparison are rather meaningless.

@Energyman

http://www.cyberciti.biz/tips/compar...-bsd-oses.html

Yeah, such things make me laugh

[risner]

Comparing a benchmark ran in a VM with one ran in the host OS is completely pointless.
If you really want to do the benchmark yourself, you'd have to install both OSes directly on the hardware, no VM involved.

First, you misunderstand. You can't run anything in a host OS of ESXi, since it doesn't have anything more swift than a busybox of the needed programs to allow you to manage the ESXi box. So every test (FreeBSD, Ubuntu, etc) would be running as a guest OS in the VMWare server and only one VM would be running (the one testing) simultaneously.

Second, pretty much no one uses bare metal now. Do you? For servers? I've got two racks of machines that I am responsible and not a one is bare metal. By your comments, it sounds like you do. But I don't at work or home.

[clau]

Anyone want to help me do any of these:
1) Install Unbuntu/Gentoo/Debian in a VM and optimize it.
2) Install and run the Phoronix Test Suite inside FreeBSD/Ubuntu so I don't have to learn how?

Comparing a benchmark ran in a VM with one ran in the host OS is completely pointless.
If you really want to do the benchmark yourself, you'd have to install both OSes directly on the hardware, no VM involved.

[risner]

Help

I have a new Quad Core Xeon (Nethalm/Core i7) ESXi box with 6 gb ram and 1.5 TB disk for a week.

I've already set up a FreeBSD properly in it (ZFS filesystem, optimized kernel/system, still gcc 4.2.1 but gcc 4.4 installed for ports.)

Anyone want to help me do any of these:
1) Install Unbuntu/Gentoo/Debian in a VM and optimize it.
2) Install and run the Phoronix Test Suite inside FreeBSD/Ubuntu so I don't have to learn how?

I'll work with anyone, provide access to the ESX vSphere client, do what I know how to do (install FreeBSD/ESXi/Gentoo.)

Would need to only have one VM running at one instant in time during testing to just get reasonable results.

Any takers?