Announcement

Collapse
No announcement yet.

What Shall We Benchmark Next? Let Us Know!

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • #31
    Please keep the one-time bench requests to a separate thread, this one is in regards to Phoromatic Tracker and on-going performance measurements.
    Michael Larabel
    http://www.michaellarabel.com/

    Comment


    • #32
      SysBench and Stream

      SysBench over Percona XtraDB (MySQL fork) vs Postgres:
      http://sysbench.sourceforge.net/
      http://www.percona.com/percona-lab.html

      Or even MariaDB:
      http://askmonty.org/wiki/MariaDB

      Stream: memory bandwith
      http://www.cs.virginia.edu/stream/

      Comment


      • #33
        Stream is already in the Phoronix Test Suite...
        Michael Larabel
        http://www.michaellarabel.com/

        Comment


        • #34
          Adding my vote for tracking Wine and the graphics stack (both Mesa and 2d).

          Comment


          • #35
            I want to see more OSX vs Linux tests.

            Comment


            • #36
              Why not actually concentrate on giving us news instead of the eternal same and uninformative benchmarks? I mean, ok, you want to promote your own Phoronix Test Suite, but all these benchmarks are really annoying and it doesn't give the user much information at the end of the day. Throwing at our face all these graphs and figures about every little alpha and beta seems to be an easy way to deal with information, but certainly not a qualitative one. Such a pity to use it the wrong way. Phoronix used to be a very good website to be up to date with Unix/linux news, but now it's all about those same always benchmarks.

              Comment


              • #37
                Originally posted by Mez' View Post
                Why not actually concentrate on giving us news instead of the eternal same and uninformative benchmarks? I mean, ok, you want to promote your own Phoronix Test Suite, but all these benchmarks are really annoying and it doesn't give the user much information at the end of the day. Throwing at our face all these graphs and figures about every little alpha and beta seems to be an easy way to deal with information, but certainly not a qualitative one. Such a pity to use it the wrong way. Phoronix used to be a very good website to be up to date with Unix/linux news, but now it's all about those same always benchmarks.
                Phoronix is a hardware-review/benchmarking site.

                Go to the front page and read the title.

                Comment


                • #38
                  A regular OpenGL benchmark on windows(!), linux and freebsd for both nvidia and ati would be nice.

                  But: Please consider to add some variance based metrics to the benchmarks (e.g. confidence intervals which are easy to understand). Currently you don't provide any answer whether your performance findings are significant.

                  Comment


                  • #39
                    Latency

                    Originally posted by sifnt View Post
                    While I don't know the specific benchmarks to use, anything that measures latency / responsiveness would be awesome.

                    Linux still doesn't feel quite right under some workloads, so anything that helps devs get to the bottom of the responsiveness issues is a good thing.

                    Thanks for making such a great platform, its growing continually more impressive fast
                    cyclictest, interbench, and framerate jitter measurements. cyclictest measures scheduler latency, interbench is supposed to measure interactive latency, and better framerate stats will show actual game playability/video watchability a lot better than just the average framerate.

                    Specifically, for meaningful latency results, cyclictest reports should show min, avg, max, and stddev. Games should have their framerate checked multiple times per second (or ideally just measure the interval between every call to glSwap()), with min, avg, max, and stddev frame times (single frame time is 1/fps, so instantaneous fps is 1/frame time).

                    Comment


                    • #40
                      Originally posted by unix_epoch View Post
                      cyclictest, interbench, and framerate jitter measurements. cyclictest measures scheduler latency, interbench is supposed to measure interactive latency, and better framerate stats will show actual game playability/video watchability a lot better than just the average framerate.

                      Specifically, for meaningful latency results, cyclictest reports should show min, avg, max, and stddev. Games should have their framerate checked multiple times per second (or ideally just measure the interval between every call to glSwap()), with min, avg, max, and stddev frame times (single frame time is 1/fps, so instantaneous fps is 1/frame time).

                      Even though nobody in the mainstream talks about latency benchmarks, latency is the most important factor in how people feel when they're using a UI. If it is responsive (i.e. low average latency, low jitter, low max latency), they'll enjoy using it more both consciously and subconsciously.

                      And we're really not getting 1 minute to edit posts...

                      Comment


                      • #41
                        Originally posted by unix_epoch View Post
                        cyclictest, interbench, and framerate jitter measurements. cyclictest measures scheduler latency, interbench is supposed to measure interactive latency, and better framerate stats will show actual game playability/video watchability a lot better than just the average framerate.

                        Specifically, for meaningful latency results, cyclictest reports should show min, avg, max, and stddev. Games should have their framerate checked multiple times per second (or ideally just measure the interval between every call to glSwap()), with min, avg, max, and stddev frame times (single frame time is 1/fps, so instantaneous fps is 1/frame time).
                        Also, a histogram of the game frame times and cyclictest/interbench latency measurements will make it really easy to see what the typical latency will be like. A 3D plot where histograms from various kernels are stacked next to each other will allow comparing kernel versions as part of the ongoing tracking.

                        Comment


                        • #42
                          Wine would be cool…

                          I also think tracking wine's performance would be really interesting and could potentially benefit the project in the long run.

                          However, at least for me, using wine together with the xf86-video-ati driver (r300) is basically near impossible, as far as running games(3d) is concerned, especially since switching over to the gallium/dri2/kms stack.
                          I'd assume that any performance improvements or regressions that could be measured when rendering direct3d/opengl would be more down to the driver side of things than to wine's code.
                          Then again, a comparison to using fglrx under the hood would levitate this discrepancy I guess...
                          Probably, other, non-render related tests would be more showing.

                          Comment


                          • #43
                            Firefox vs. Chromium vs. Konqueror? Maybe just the javascript engines?

                            Comment


                            • #44
                              Originally posted by Michael View Post
                              We are seeking requests for trackers to continuously monitor performance, not individual performance article benchmarks. Post those requests in another thread as I will just be ignoring them in this thread.
                              I think asking people to stick to tracker suggestions is an exercise in futility considering the title of the news article.

                              Many people, myself included, upon reading the title "Default What Shall We Benchmark Next? Let Us Know!" think you were asking for suggestions on what you will benchmark (as a "one time" thing) and immediately rushed to post with an opinion, skipping the actual article contents in the process.

                              Comment


                              • #45
                                Originally posted by grantek View Post
                                Firefox vs. Chromium vs. Konqueror? Maybe just the javascript engines?
                                Sounds good but don't forget epiphany

                                Comment

                                Working...
                                X