What Shall We Benchmark Next? Let Us Know!

[Michael]

Please keep the one-time bench requests to a separate thread, this one is in regards to Phoromatic Tracker and on-going performance measurements.

[Raine]

SysBench and Stream

SysBench over Percona XtraDB (MySQL fork) vs Postgres:

http://sysbench.sourceforge.net/

http://www.percona.com/percona-lab.html

Or even MariaDB:

http://askmonty.org/wiki/MariaDB

Stream: memory bandwith

http://www.cs.virginia.edu/stream/

[Michael]

Stream is already in the Phoronix Test Suite...

[curaga]

Adding my vote for tracking Wine and the graphics stack (both Mesa and 2d).

[avilella]

I want to see more OSX vs Linux tests.

[Mez']

Why not actually concentrate on giving us news instead of the eternal same and uninformative benchmarks? I mean, ok, you want to promote your own Phoronix Test Suite, but all these benchmarks are really annoying and it doesn't give the user much information at the end of the day. Throwing at our face all these graphs and figures about every little alpha and beta seems to be an easy way to deal with information, but certainly not a qualitative one. Such a pity to use it the wrong way. Phoronix used to be a very good website to be up to date with Unix/linux news, but now it's all about those same always benchmarks.

[some-guy]

Why not actually concentrate on giving us news instead of the eternal same and uninformative benchmarks? I mean, ok, you want to promote your own Phoronix Test Suite, but all these benchmarks are really annoying and it doesn't give the user much information at the end of the day. Throwing at our face all these graphs and figures about every little alpha and beta seems to be an easy way to deal with information, but certainly not a qualitative one. Such a pity to use it the wrong way. Phoronix used to be a very good website to be up to date with Unix/linux news, but now it's all about those same always benchmarks.

Phoronix is a hardware-review/benchmarking site.

Go to the front page and read the title.

[krausest]

A regular OpenGL benchmark on windows(!), linux and freebsd for both nvidia and ati would be nice.

But: Please consider to add some variance based metrics to the benchmarks (e.g. confidence intervals which are easy to understand). Currently you don't provide any answer whether your performance findings are significant.

[unix_epoch]

Latency

While I don't know the specific benchmarks to use, anything that measures latency / responsiveness would be awesome.

Linux still doesn't feel quite right under some workloads, so anything that helps devs get to the bottom of the responsiveness issues is a good thing.

Thanks for making such a great platform, its growing continually more impressive fast

cyclictest, interbench, and framerate jitter measurements. cyclictest measures scheduler latency, interbench is supposed to measure interactive latency, and better framerate stats will show actual game playability/video watchability a lot better than just the average framerate.

Specifically, for meaningful latency results, cyclictest reports should show min, avg, max, and stddev. Games should have their framerate checked multiple times per second (or ideally just measure the interval between every call to glSwap()), with min, avg, max, and stddev frame times (single frame time is 1/fps, so instantaneous fps is 1/frame time).

[unix_epoch]

cyclictest, interbench, and framerate jitter measurements. cyclictest measures scheduler latency, interbench is supposed to measure interactive latency, and better framerate stats will show actual game playability/video watchability a lot better than just the average framerate.

Specifically, for meaningful latency results, cyclictest reports should show min, avg, max, and stddev. Games should have their framerate checked multiple times per second (or ideally just measure the interval between every call to glSwap()), with min, avg, max, and stddev frame times (single frame time is 1/fps, so instantaneous fps is 1/frame time).

Even though nobody in the mainstream talks about latency benchmarks, latency is the most important factor in how people feel when they're using a UI. If it is responsive (i.e. low average latency, low jitter, low max latency), they'll enjoy using it more both consciously and subconsciously.

And we're really not getting 1 minute to edit posts...