If this is your first visit, be sure to
check out the FAQ by clicking the
link above. You may have to register
before you can post: click the register link above to proceed. To start viewing messages,
select the forum that you want to visit from the selection below.
It is a bad idea to have compilation benchmarks with different versions of a compiler. Because the speed of different compilers is an added variable to determining the speed of the OS. In fact, the difference of speed between different compilers might be the only reason for a speed difference between the OSes.
Also, as a developer, the speed of compilation is one of the last factors for consideration when choosing a compiler. I'd be primarily concerned with the speed of generated executable and supported languages, standards, and libraries.
Most (casual) developers don't look to closely at the compiler that ships with the system, if it has gcc they start using. It is only in more formal environments where there is tool selection (or an individual developer is really focused on some metric like above.
I have built *way* too many compilers myself (crosstool really rocks), but for most purposes, I don't bother looking to closely at the compiler for general ad-hoc development tasks.
Finally, remember that a lot of people _like_ to stay bleeding edge with end user functionality, like kernels, gnome, firefox, etc. They aren't focused on speed or size of the result, but they know they will be building on a regular basis. They too don't focus on selecting the right compiler, but instead grab what is easily available.
Maybe better give a gui rating not a performance rating I prefer solutions executed by scripts - run it, then something works. Of course lots of distros provide extra guis for this and that. I did not try Mandriva nor Fedora lately, but from history usually Mandriva is maybe just behind SuSE's yast from gui tools. When you like the tools a distro provides additionally to its preconfig/stability that's usally the logical reason to choose it. Nobody would use Mac OS X because it is faster in a few benchmarks - same applies for any distro. I don't know of anybody who selects a distro because some apps run slightly faster.
As PTS compares mainly selfcompiled binaries it would not be that hard to bootstrap a newer compiler if needed. I never did that because the default compiler was slower or generated slower binaries. The only time i definitely had to compile even an older gcc (2.95) instead of using the default (some 2.96 prerelease) of old Red Hat 7.x systems was because the compiler was so broken that it was not able to generate binaries from standard source code, i still have got no idea how many changes red hat did to compile everything they shipped precompiled Well i did not prefer to fix the code, i just added another compiler - all in my home only, so nothing could hurt the rest of the system.
As the article says in the ifrst page: "The x86_64 builds of both Fedora 11 and Ubuntu 9.04 were used."
Arrg, how lame of me, thanks for pointing it out. Then it would be nice to add the 32 bit ;-) Seriously, this is the most FAQ I see in forums from people who just bough a 64 bit system (namely: "Does 64 bit perform better?").
I agree with the objections, but I think compilation of a rather large project has historically been used as an overall system benchmark because it exercises CPU, memory and disk read/writes. So, besides the caveats you all provided, it's still a nice benchmark if taken with a grain of salt
In summary, it's the best benchmark ever, except that it doesn't mean anything.
You guys are overlooking the biggest reason Fedora performs worse in some of these tests (and frankly I am disappointed no one has mentioned it).
Fedora's performance is impacted by the fact that many of its binaries are compiled with options like FORTIFY_SOURCE and -fstack-protector as well as ASLR hardening. Further, exec-shield and Selinux are on by default. All of these protections will put a small hit on performance (SELinux alone can sometimes result in as much as 7% overhead on some benchmarks) Bottome line: when all of these protections are combined, the results in this benchmark seem about right. You can read about Fedora's security features here.
No other desktop Linux distro I am aware of offers the security that Fedora does out of the box. So, really, we're comparing apples to oranges here. Essentially, it's a trade off (just like many things are when choosing a distro), and in this case it happens to be performance vs. security. Personally, I am willing to give up some performance in some areas for the increased security.
(I already posted my reply once and it didn't show up, so if it does show up later, I apologize for the double post).
I had a long reply as to why this benchmark showed the results it did, but I will summarize:
Fedora has increased security that can only be achieved through some compiler options like FORTIFY_SOURCE, -fstack-protector, PIE, and exec-shield. Most of these options will result in a performance hit. And that doesn't even count SELinux, which in itself has shown overhead as high as 5-7% in some tests. It's obvious to me that (especially in the Apache tests) this is why Ubuntu performed so much better. Ubuntu is not protected like Fedora is! (AppArmor on Ubuntu only protects CUPS by default and has no impact on overall system performance).
Usally all apps are compiled from source if they are not binary only. That usally does not affect it, but you might have installed more build-deps on one system that would enable additionally features and compile longer for example.
Please try to figure out the reasons for the differences
Just reporting benchmark results isn't all that useful, please try to figure out the reasons for the discrepancies in performance. For CPU bound programs the differences in performance between Fedora and Ubuntu should be in the noise, when you find large differences it's worth digging deeper to find out the reason. Are the CPU Speed Governors set the same? Is it SeLinux? are there background processes that are running on one and not the other. Is it compiler switches? Is it the kernel rev? It's easy to enable/disable/change SeLinux, daemons, speed governor modes, firewall and even file systems. It's also easy to compile a standard kernel using identical switches for both distros, that would allow you to isolate the effects of the kernels.