Multi-Core, Multi-OS Scaling Performance

[phoronix]

Phoronix: Multi-Core, Multi-OS Scaling Performance

In this article we are looking at how Linux, OpenSolaris, and FreeBSD scale across multiple cores. Benchmarked are CentOS 5.5, Fedora 14, PC-BSD/FreeBSD 8.1, and OpenIndiana b148 as we see how the performance differs when running on one, two, three, four, and six cores, plus when Intel Hyper Threading is enabled.

http://www.phoronix.com/vr.php?view=15706

[baryluk]

So was PC-BSD devlopers notified about this important problem? It looks like not well tested scheduler on multicore.

[baryluk]

HT is a real nightmare. If you have lots of data to calculate and wouldn't even run into the cache hell, it is fine.

However if a high priority thread is the HT you're screwed. HT is basicaly a technique that lets a second thread utilize a part of the CPU that isn't being consumed by another thread.

Great for round robin scheduling and number crunching. For everything else... not so much...

For number crunching? I will say in most cases not. It is good for number crunching with irregular structure of memory access and working set of memory is bigger than L2 cache. Then using HT will help, as it will hide memory latencies. But for pure number crunching (like computing complicated function of 100 floating operations on array of 1000000 doubles, for example when solving PDE), it will not help that much (even if 1000000 doubles is 8MB - much more than L2), as cache prefetching and actual FPU computations are longer than memory transfer time. For simple computations, or irregular access yes - for example bigmultiuser databases, or raytracing with accelerations strctures (BVH or kd-tree) - it can give even 20-30% speed increase.

[baryluk]

Looks like kind of scheduler bug in PC-BSD. It cannot be so bad. Obviously wasn't tested well on multicore.

[V!NCENT]

Common wisdom in the sysadmin field about 6 years ago or so was that HT was a real mixed bag.

HT is a real nightmare. If you have lots of data to calculate and wouldn't even run into the cache hell, it is fine.

However if a high priority thread is the HT you're screwed. HT is basicaly a technique that lets a second thread utilize a part of the CPU that isn't being consumed by another thread.

Great for round robin scheduling and number crunching. For everything else... not so much...

[Garp]

Common wisdom in the sysadmin field about 6 years ago or so was that HT was a real mixed bag. At the time the HT thread shared the CPU cache with the main thread. For highly repetitive work on the same data, e.g. multimedia processing, that was fine. However if the HT thread picked up an instruction that didn't have data in cache it would flush and reload. That could end up wasting a lot of cycles and could noticeably slow things down. Safest was to assume that it would be bad, but if you could, do benchmarking. I haven't seen any technical details to suggest whether that weakness still remains or not.

[kebabbert]

Gah, the edit limit here is truly microscopic. Anyway.
The only way to really isolate that effect would of course be to run the linux distros through twice, with different schedulers ... but then it'd only be fair to do the same with PC-BSD (BSD vs the default ULE) and openindiana (if they have more than one). It would more than double the work, of course.

Solaris has several schedulers, that you can change on-the-fly without rebooting. Most probably OpenIndiana also has those schedulers. OpenIndiana is in alpha stage, and has not hit beta version yet. OpenIndiana is the fully open sourced version of OpenSolaris, so it should have every feature that OpenSolaris had (which also was in alpha version).

This year Solaris 11 will hit beta version (it is in alpha now), and later this year Solaris 11 will finally be done and released for production! Yey!

[dnebdal]

That would be a completely different test - of the schedulers. This was a test of the operating systems. A more useful one, in my opinion. Though i do agree that determing whether the FS was to blame might be nice.

Well, in this case just listing the respective schedulers would be nice - my point was that scheduler choice is as interesting in this benchmark as the filesystem used is for DB benchmarks.

[smitty3268]

Gah, the edit limit here is truly microscopic. Anyway.
The only way to really isolate that effect would of course be to run the linux distros through twice, with different schedulers ... but then it'd only be fair to do the same with PC-BSD (BSD vs the default ULE) and openindiana (if they have more than one). It would more than double the work, of course.

That would be a completely different test - of the schedulers. This was a test of the operating systems. A more useful one, in my opinion. Though i do agree that determing whether the FS was to blame might be nice.

[dnebdal]

It would be explanatory information, though - much like how the filesystem is often relevant when there are large gaps in database performance. (The different kernel versions here are of course a confounding effect - but it the two linuxes are also using different schedulers, that might be part of the explanation.)

Gah, the edit limit here is truly microscopic. Anyway.
The only way to really isolate that effect would of course be to run the linux distros through twice, with different schedulers ... but then it'd only be fair to do the same with PC-BSD (BSD vs the default ULE) and openindiana (if they have more than one). It would more than double the work, of course.