The Performance Impact To AMD Zen 2 Compiler Tuning On GCC 9 + Znver2

[foobaz]

I like the geometric mean because it gives an equal weight to each ratio of results. In benchmarking, we care about relative performance, not absolute. Using a geometric mean is appropriate because the absolute value of each benchmark have no effect on the result, only the relative values do.

[carewolf]

Please don't use -march=x86-64 it is equivalent to writing -march=k8, it specifies the minimum architecture for x86-64 but also tunes for it, which leads to it being worse than writing nothing or -mtune=generic (which is the same as writing nothing).

Edit: I checked the documentation and actually couldn't find that information there, so wondering where I got the idea I checked the gcc source code, https://code.woboq.org/gcc/gcc/confi...86.c.html#3550

Yeah.. Tunes for K8.. Please don't use it

[oibaf]

And as explained more elegantly via Wikipedia:

Why don't you normalize the input data before doing the arithmetic mean?
Set the lower value of every benchmark to 100 and the other proportionate to it. Then the value are comparable, you can properly do an arithmetic mean.

[carewolf]

Why don't you normalize the input data before doing the arithmetic mean?

Please, just read the wikipedia article already https://en.wikipedia.org/wiki/Geometric_mean .. When normalizing and using arithmetic mean, you can prove anything you want by how you decide to normalize it.

[xorbe]

... you can prove anything you want by how you decide to ...

Well that's every review in a nutshell, which is why reviewers have reputations.

[profoundWHALE]

I asked this before and got no answer.

So, once again - what is the reason to use the geometric mean instead of the arithmetic one?

Hint: If you're not sure why, it's certainly wrong.

I'm pretty sure if you knew why, you wouldn't have asked the question. You would've been like:

What is the reason to use the geometric mean instead of the arithmetic one?

Arithmetic mean is good because XYZ, and geometric mean is bad because ABC.

And after all of this it turns out there's a wikipedia page on that very topic. I think Michael's choice is particularly good with the geometric mean because the Phoronix Test Suite compares so many different things with many different types of measurement units and different scales.

[entropy]

Okay, I bite.

And after all of this it turns out there's a wikipedia page on that very topic. I think Michael's choice is particularly good with the geometric mean because the Phoronix Test Suite compares so many different things with many different types of measurement units and different scales.

Please show me an example where Phoronix benchmarks compose a result from "different types of measurement units and different scales".

[thebear]

Here's a paper on why geometric mean is to be preferred to arithmetic mean (I'm not familiar with the journal so I cannot speak to it's peer review process though):

"How not to lie with statistics: the correct way to summarize benchmark results", Communications of the ACM - The MIT Press scientific computation series CACM Homepage archive Volume 29 Issue 3, March 1986 Pages 218-221 ACM New York, NY, USA

https://dx.doi.org/10.1145/5666.5673

or

https://www.cse.unsw.edu.au/~cs9242/18/papers/Fleming_Wallace_86.pdf

[entropy]

Here's a paper on why geometric mean is to be preferred to arithmetic mean (I'm not familiar with the journal so I cannot speak to it's peer review process though):

"How not to lie with statistics: the correct way to summarize benchmark results", Communications of the ACM - The MIT Press scientific computation series CACM Homepage archive Volume 29 Issue 3, March 1986 Pages 218-221 ACM New York, NY, USA

https://dx.doi.org/10.1145/5666.5673

or
https://www.cse.unsw.edu.au/~cs9242/...Wallace_86.pdf

I stand corrected.
If he takes the geometric mean only in the final comparison, that's fine indeed.

For whatever reason I thought, the geometric mean was applied to the partial measurements as well.

Sorry for the fuss.

[nivedita]

Okay, I bite.



Please show me an example where Phoronix benchmarks compose a result from "different types of measurement units and different scales".

This article?