Why does Phoronix use Ubuntu for Benchmarking ?

[Apopas]

The desktop is more than fast enough for me already. I don't care if a text box draws 1.4% faster on one distribution than another.

But benchmarks have rarely practical use. Their existence is only to prove that my distro/OS is better than yours for no real reason

[frantaylor]

But benchmarks have rarely practical use. Their existence is only to prove that my distro/OS is better than yours for no real reason

Well that is just bogus because there are many issues. Some ethernet cards just suck and you cannot see for sure unless you benchmark. If you tweak on the block size for your disk partitions it will have impact, but you will not know unless you benchmark. Often there are multiple ways to write an algorithm and you cannot tell which is faster unless you benchmark. There is a bewildering array of optimizations for gcc and you cannot tell which will make your program run faster unless you benchmark.

[lordmozilla]

Centos 5 in VMware versus Centos 5 in qemu, for example.

Releasing Vmware benchmarks is illegal. sorry ;-)

Benchmarks on a rolling release such as Arch/gentoo would show more what unpatched linux does. The problem with ubuntu is the patching of patches which end up with for example Intel gfx issues.

But phoronix is doing a great job, and I'm sure using ubuntu gets a lot of readers in.

[Apopas]

Well that is just bogus because there are many issues. Some ethernet cards just suck and you cannot see for sure unless you benchmark. If you tweak on the block size for your disk partitions it will have impact, but you will not know unless you benchmark. Often there are multiple ways to write an algorithm and you cannot tell which is faster unless you benchmark. There is a bewildering array of optimizations for gcc and you cannot tell which will make your program run faster unless you benchmark.

If I didn't like benchmarks I wouldn't be here and their usefulness is obvious. I had in mind some desktop benchmarks like lame encoding which mainly differs by 1-2% between the dists when I made the post.

[frantaylor]

Releasing Vmware benchmarks is illegal. sorry ;-)

Benchmarks on a rolling release such as Arch/gentoo would show more what unpatched linux does. The problem with ubuntu is the patching of patches which end up with for example Intel gfx issues.

But phoronix is doing a great job, and I'm sure using ubuntu gets a lot of readers in.

Is it publishing the benchmark that is "illegal", or is the publishing of the RESULTS of the benchmark that is "a contractual violation"?

It would be nice to have a script that runs the benchmark for us and we can view the results for ourselves. I can't imagine that that would be "illegal", or even a "contractual violation"

I wonder if there is any way to procure a copy of VMware Player without agreeing to the EULA. In this case one could publish benchmarks, because one did not enter into any sort of contract.

[MetalheadGautham]

Releasing Vmware benchmarks is illegal. sorry ;-)

Releasing VMWare benchmarks is illegal ? Since when ? How ? Why ?

Benchmarks on a rolling release such as Arch/gentoo would show more what unpatched linux does. The problem with ubuntu is the patching of patches which end up with for example Intel gfx issues.

And what is wrong with showing what unpatched linux does ? And btw, Gentoo too can be easily patched.

[frantaylor]

Releasing VMWare benchmarks is illegal ? Since when ? How ? Why ?

It's not "illegal", it's a "contract violation" to publish benchmark results for many products. Read the EULA. Oracle is another example of this.

EMC would probably work their legal wiles and take down any web site that published VMware benchmark results. However, there is nothing wrong with publishing the benchmark program itself. The curious can run the benchmark themselves, and use the results for decision-making, as long as they don't publish the results.

The problem with benchmarks of big complex software is that the results are often dependent on hardware or software setup or whatever, and the results are not very useful unless you just happen to have exactly the same setup.

[deanjo]

Releasing VMWare benchmarks is illegal ? Since when ? How ? Why ?

It's been in the EULA for as long as I remember.

See here why:

http://blogs.vmware.com/virtualreality/2009/03/index.html

Look at the March 12th at the bottom of the page.

You may use the Software to conduct internal performance testing and benchmarking studies, the results of which you (and not unauthorized third parties) may publish or publicly disseminate; provided that VMware has reviewed and approved of the methodology, assumptions and other parameters of the study. Please contact VMware at [email protected] to request such review.

[Kano]

Basically all VMs have got problems with timers. If you really want to benchmark then use a manual stop watch. Timers tend to run pretty much like the moon inside VMs

[frantaylor]

Basically all VMs have got problems with timers. If you really want to benchmark then use a manual stop watch. Timers tend to run pretty much like the moon inside VMs

There is nothing wrong with the timer on the bare metal machine in which the VMs are running. Processes inside of VMs can easily communicate with the host system, and vice versa.

The guests' real time clock in Vmware Workstation 6.5.2 is really quite accurate. It uses the timer in the host system. The clocks in my VMs stay accurate to within a second over days of use despite heavy loading of both the guest and host operating systems.