Dear Phoronix benchmarkers. I guess this is a futile attempt, but anyway:
It's ok when you try to compare operating systems in a workloads that don't really stress operating systems - e.g. by running programs that spend almost all of their cpu time in userland. Doing real benchmarks - with, say, PostgreSQL or Apache or Postfix - is much harder, and you're not being paid for this, so thanks for doing at least _some_ benchmark.
_However_ - when you benchmark something that depends mostly on a compiler version and flags, and then say that operating system A using gcc version N performs better than operating system B using gcc version M, then this is no longer a benchmark - it's something between FUD and pure stupidity.
Announcement
Collapse
No announcement yet.
AMD Shanghai Opteron: Linux vs. OpenSolaris Benchmarks
Collapse
X
-
Originally posted by flice View PostOK, so I took the time and benchmarked several of the tests as they appear in the article. The benchmarks were done inside VirtualBox VMs running in Ubuntu 8.10. One VM was OpenSolaris x64 2008.11, the other was Ubuntu x64 8.10. All the OpenSolaris programs were compiled with flags taken from here.
So here's what I have:
Code:SOLARIS LINUX lame 23.17 23.02 oggenc 23.51 19.11 GraphicsMagic - HWB 74 50 GraphicsMagic - LAT 17 10 GraphicsMagic - resize 47 33 GraphicsMagic - sharpen 20 10
Overall, the article results seem to be completely detached from reality.
Flice I'm assuming you used SunCC for the above correct? It would be interesting to see how gcc3.4 behaves in your VM. At least then people could draw parallels between it and SunCC when it comes to Phoronix test suite.
Thanks for the results.
Leave a comment:
-
OK, so I took the time and benchmarked several of the tests as they appear in the article. The benchmarks were done inside VirtualBox VMs running in Ubuntu 8.10. One VM was OpenSolaris x64 2008.11, the other was Ubuntu x64 8.10. All the OpenSolaris programs were compiled with flags taken from here.
So here's what I have:
Code:SOLARIS LINUX lame 23.17 23.02 oggenc 23.51 19.11 GraphicsMagic - HWB 74 50 GraphicsMagic - LAT 17 10 GraphicsMagic - resize 47 33 GraphicsMagic - sharpen 20 10
Overall, the article results seem to be completely detached from reality.Last edited by flice; 12 February 2009, 03:17 AM.
Leave a comment:
-
Originally posted by kebabbert View PostTo me it seems that Linux people thinks that scalability is about 4-8 CPUs. The Solaris people talks about hundreds of CPUs. With the very same kernel, unmodified. Now THAT is scalability.
I dont get it. What is wrong with the Linux people? ... I really really wonder, how can the Linux people call this fair?
So, just because lots of people say dumb stuff doesn't mean that all Linux people agree with them. Some of the examples you give are pretty embarrasing.
edit: I forgot to say anything about different kernel binaries for server vs. desktop. Linux doesn't have different implentations of anything important (scheduler, vm, networking) for small vs. large machines, but Linux is designed to be configured. Linux has always been open source, so it's not surprising that it's well set up to be compiled by the end user according to their customized config. e.g. if you compile for a uniprocessor machine, some of the lock/unlock functions become macros that expand to do { } while(0). Even if openSolaris is only designed to have a single kernel binary for all systems, I don't think it's fair to hold Linux to that standard. But like I said, there aren't kernel config options that select many-core tuned versions of algorithms vs. few-core tuned versions. There are kernel config options for some specific bigiron hardware, i.e. amd64 kernels can be built for standard PC-compatible (CONFIG_X86_PC) or ScaleSMP vSMP systems (CONFIG_X86_VSMP), whatever they are. ia32 kernels have config options for a few different kinds of older big-iron hardware.
These days, Ubuntu has a -generic kernel and a -server kernel. -generic, which is tuned for desktops, uses voluntary kernel preemption of kernel threads for lower latency but worse throughput. Before tick-less kernels, it use timer interrupt HZ=250. The -server kernel doesn't preempt the kernel, and used HZ=100 before Linux went tick-less. The only other difference is defaulting to the deadline iosched instead of cfq. (I'm looking at debian/configs/amd64/config.server and config.generic in Ubuntu's kernel git tree.)
There are a lot more differences between the i386 generic and server kernels. e.g. -server uses PAE to handle up to 64GB on a 32bit machine, and is built with CONFIG_M686=y instead of CONFIG_M586=y. And it builds in more Xen stuff...
server vs. non-server seems like a good place to have different kernels. There is even a different installer installer for Ubuntu's server flavour (which installs a different set of packages by default).
Anyway, Linux has gained some features which make one-binary-everywhere work better. Since that's what distros want to ship to avoid confusing newbies. e.g. on a uniprocessor machine, the kernel will, at runtime, patch the x86 lock prefix in all the inline spinlocks to a nop, so the spinlocks just increment/decrement a counter without serializing the CPU. It can even patch it back if a CPU is hot-plugged later. (This is most likely to happen inside a vm, I imagine.)
So my point here is that scalability from a single code base is what counts, not from a single binary from that one code base. But it only counts if scalability-enabling alternate bits of code don't change the API for drivers and stuff. So I think it would be fair to have a few different algorithms in the VM or something that could be selected, but only if drivers that allocated memory didn't have to have two code paths for the different settings of the config option. That would be scalability at the cost of huge maintenance overhead for other code. In general, the kernel devs aren't keen on even that. AFAICT, they always choose more scalable algorithms and default settings of tuning params.
Notice that I'm not claiming that Linux scales well up to 256 cores or anything like that. I just don't know. I think it does ok, but I have no experience to back that up. Scalability is definitely one of the ongoing development goals, though, so Linux still has plenty of room for improvement, to put it in a glass-half-full light. I've read a paper from 2006 on tuning XFS for a 24 CPU ia64 machine with hundreds of disks, so Linux does at least have a pretty scaleable filesystem thanks to SGI. http://oss.sgi.com/projects/xfs/pape...2006-paper.pdf (or slides from the talk http://oss.sgi.com/projects/xfs/pape...esentation.pdf). Other good links on some guy's web site: http://www.sabi.co.uk/Notes/linuxFS.htmlLast edited by Peter_Cordes; 11 February 2009, 11:14 AM.
Leave a comment:
-
Originally posted by Kano View PostSome of the benchmarks are just are a bit pointless. ... Using compile benchmarks between os is like comparing apples to pears.
Sometimes you want to know whether you like apples or whether you like pears, not which kind of apples you like best. Err, probably nobody will choose a platform because the default compiler is fast, but it's still interesting to do things that compare how all the differences between compiling and running something on Ubuntu vs. OpenSolaris stack up. Or it would have been if the non-Java stuff had been compiled with a useful compiler.
Leave a comment:
-
Originally posted by kraftman View PostThose are not the results, but Sun guy's talk. It will be great if some of you guys will do benchmarks itself. Same hardware, same compilers, same flags etc. I can do tests, but only in vbox. I wrote in my first post in this thread that I don't consider this Phoronix benchmark was fair.
Originally posted by kraftman View PostIf modified Linux kernel scales better than modified/unmodified Solaris kernel... I bet you can download this modified kernel from somewhere. Btw. what are those modifications? Modified settings, code?
Originally posted by kraftman View PostRight, normal people don't care too much about scaling. Do you think that companies are bunch of idiots who don't know how to properly setup Linux system for their needs? :> Andrew is objective. He's not a lier like others who claim that their system is 15% better in something, but it looks like it's not or there are NO bugs, NO regressions, it's most secure etc. I don't expect such objectiveness from Sun guys - Solaris is better Linux.... Only marketing crap, they won't tell you a word about bugs or performance problems (at least not so sincerely like Linux devs).
Kernighan and Richie, the programming Gurus, had studied Linux source code and said that the quality was low. It was not good code.
You know, Linus Torvalds sees the Linux kernel development as biology evolution, he explained. If there are some problems, he will redesign that part. This way the kernel evolves and becomes better and better.
I dont agree with that. If you redesign everything all the time, then you get... Windows. You know, each new version of Windows sucks, is error prone and full of bugs. After a few Service Packs, then Windows becomes stable. Linus redesigns big parts all the time and this causes all these regressions Andrew talks about. Linux kernel is over 10 million lines of code. That is quite sick. The whole Windows NT was 10 millions LOC. Now you have ONE kernel that big. It must be impossible to get that huge monolithic piece of code, stable. Just as Andrew explains.
As I said, several startup companies are using Linux. And everything is fine. But sooner or later, their workload increases a lot beyond what them people are used to. And they venture into the realms of Enterprise. Then they notice that Linux doesnt cut it anymore.
Whatever. I am not going to argue with you over this matter anymore. If you prefer Linux and I prefer Solaris, then we both are happy.
PS. I am interested in SUNs new machines. Rumours say that it will have 2048 threads. I think lots of OS will have problems handling 2048 threads.Last edited by kebabbert; 11 February 2009, 09:36 AM.
Leave a comment:
-
Originally posted by kebabbert View Posthttp://www.phoronix.com/forums/showt...t=14017&page=5
"It is possible some of the performance differences are due to the
gcc version being used as Solaris bundles gcc 3.4.3 and other distros
may bundle 4.x, but it is much more likely due to the default ABI
used by the bundled gcc. "gcc -O" on Solaris will default to ia32/x87,
whereas on the other "64 bit" distros tested it will default to amd64.
The performance difference can be seen in the two Byte Computational
benchmarks on page 7 where Solaris appears to lag: Dhrystone 2
(./Run dhry2) and Floating-Point Arithmetic (./Run float). These
tests are compiled with "gcc -O" which produces ia32/x87 code.
When adding "-m64" which puts Solaris on par with the other distros,
the performance jumps quite a bit. Measured on a Intel QX6700,
dhry2 goes from 9307771.4 to 13421763.5 and float goes from
707932.5 to 1477185.6.The ABI used can make a big difference. Solaris allows you to
choose either, but the default for the bundled gcc is still the
slower ia32/x87."
I doubt that Linux scales well. Ive seen numerous articles where companies say that when their workload increases beyond a certain point, Linux doesnt cut it anymore. And then they switch to Solaris on the same hardware, and everything is fine. Ive posted several links showing this in my posts, on my link above.
And, in your link that moron brags about Linux v2.6 being able to handle 16 cpus. I assume he talks about standard linux kernel. At the same time he talks about large Linux clusters, but fail to tell they are modified Linux kernels.
If Solaris scales well on hundreds of CPUs and many more threads, it is not something that affects an ordinary user, right? So normal people which are using dual cpus or so, doesnt notice bad Linux scaling. It is in companies, when their workload increases they have to switch to a real Unix. In my link above, I post an interview to Linux kernel hacker Andrew Morton, who claims that Linux code is full of bugs and regressions. Read the interview if you dont believe me.
And if Linux scales better than Windows, why people prefer using Windows?
Leave a comment:
-
Originally posted by drag View PostVirtuozzo has been used for years to create container-style virtulaization for web serving companies. The open source part is OpenVZ.
Plus there is Vserver that also does container-style virtualization.
Not to also mention other virtualization options for Linux... For example Linux now has native built-in ability to be a hypervisor (like Xen or Vmware) in the form of KVM. Every Linux distribution ships with that. There are then a half a dozen virtualization options with various plusses and minuses for Linux.
I think you've missed the point of zones. Zones aren't about virtualization they are about limiting access to a CPU's cores. This comes in extremely handy with Oracle that has a per thread license allowing it to be installed on say an 8 core machine but at a 1 or 2 core price (assuming you limit the Oracle zone to 1 or 2 cores). Container virtualization doesn't limit the access to the CPU's cores thus you would have to pay far more for the Oracle license.
That will upset your CFO which in turn will upset the CEO and in this market probably cost you your job
Leave a comment:
-
Sheesh, have you tried OpenSolaris at all ?
Originally posted by Farrell View PostWell, all I know is having used Solaris (and before that, SunOS) and Linux on Sun hardware, the system is almost always faster running Linux, be it SparcStation 10/20 (some with Ross processors), and Ultra/Enterprise beasties of various types. Thus Solaris's offspring, OpenSolaris being slow on x86/64 systems is no surprise to me.
The don't call it Slowaris for nothing!
ttyl
Farrell
Leave a comment:
-
Originally posted by etacarinae View PostOnce again, they're quite divergent. Only some selected features and enhancements are backported to Solaris from OpenSolaris. And if you think that it takes rewriting "all of the [..] code" to speed up someting than I don't know how to respond to that. It would just mean that any given project can never be optimized.
The don't call it Slowaris for nothing!
ttyl
Farrell
Leave a comment:
Leave a comment: