Open-Source Performance-Per-Dollar Benchmarking

[phoronix]

Phoronix: Open-Source Performance-Per-Dollar Benchmarking

Recently within our forums there was a request to do performance-per-dollar benchmarks and reporting. Today I came up with a way to make this process very easy and trivial, that you can do too when carrying out your own open-source Linux tests...

http://www.phoronix.com/scan.php?pag...nce-Per-Dollar

[mitcoes]

Thanks a lot

[Ploplop]

Wouldn't it be far more useful to integrate power data over time instead of calculating an average?

That would give the total power used to complete one task (i.e. one series of benchmarks) instead of something vaguely correlated with the maximum power consumption and the particular timing between benchmarks. Such an integral would better represent the power consumption of the chips: this way, you wouldn't penalize powerful chips that draw a lot of power for a short while compared to a far less powerful chip drawing half the power but taking 10 times longer to perform the same task.

Said otherwise, with you current metrics, datacenter should be filled with raspberry pi's, not by high-end intel/amd chips.

[maarten]

I concur with Ploplop.

It would be interesting to see how much energy each test consumed. Preferably In in J or Wh units.

It would be nice to see how much energy a full-blown comparison of graphics cards used.
In addition you'll have an idea how much Dollars you spend on each test/article.

Add an estimation of the time you've allocated to running and analyzing the tests + writing the article and you'll see how much everything costs., for bookkeeping purposes.

I also have a comment on the 'Performance per Dollar'.
I find the name confusing. It's not clear whether you're referring to the cost of the hardware or the cost of the energy.
I would call the table 'Variable cost' and let the y-axes-unit be 'Dollar per Operation' or "Dollar per second gameplay'.

Thanks for the well-written articles!

[Shimon]

Also PERFORMANCE_PER_WATT greater than 1 could mean actual power. In absence of sensors this could provide a decent estimate.