The scientific approach

It happend so many times before but this time I really had to register and write this post. Phoronix should really pay more attention to numbers and their math. For example, in the "Unigine Heaven v2.0 1980x1080" we see that the Metacity result is 74% faster than Compiz, not 63%. I haven't found a rounding of the numbers that would actually lead to 63%. If Metacity leads to 25.34 fps and Compiz to 14.56, then Metacity is (25.34-14.56)/14.56=0.74... times faster than Compiz. On the contrary, Compiz reaches just (25.34-14.56)/25.34=0.425=42.5% of the performance of Metacity. Now question to the reader: By what percentage is Compiz slower than Metacity?

Overall, Phoronix should definitly come in contact with someone with a more academic background. I'm happy that the Phoronix Benchmark Suite incorporated variance measurements to help with the significance. But this is definitely not enough. The test scenarios for the article should be designed more according to the "scientific approach" to do evaluation. Otherwise, Phoronix just targets readers who are just able to understand black/white messages. "Scientific depth" is the keyphrase here.

It happend so many times before but this time I really had to register and write this post. Phoronix should really pay more attention to numbers and their math. For example, in the "Unigine Heaven v2.0 1980x1080" we see that the Metacity result is 74% faster than Compiz, not 63%. I haven't found a rounding of the numbers that would actually lead to 63%. If Metacity leads to 25.34 fps and Compiz to 14.56, then Metacity is (25.34-14.56)/14.56=0.74... times faster than Compiz. On the contrary, Compiz reaches just (25.34-14.56)/25.34=0.425=42.5% of the performance of Metacity. Now question to the reader: By what percentage is Compiz slower than Metacity?

Overall, Phoronix should definitly come in contact with someone with a more academic background. I'm happy that the Phoronix Benchmark Suite incorporated variance measurements to help with the significance. But this is definitely not enough. The test scenarios for the article should be designed more according to the "scientific approach" to do evaluation. Otherwise, Phoronix just targets readers who are just able to understand black/white messages. "Scientific depth" is the keyphrase here.

## Comment