No announcement yet.

What Shall We Benchmark Next? Let Us Know!

  • Filter
  • Time
  • Show
Clear All
new posts

  • #41
    Originally posted by unix_epoch View Post
    cyclictest, interbench, and framerate jitter measurements. cyclictest measures scheduler latency, interbench is supposed to measure interactive latency, and better framerate stats will show actual game playability/video watchability a lot better than just the average framerate.

    Specifically, for meaningful latency results, cyclictest reports should show min, avg, max, and stddev. Games should have their framerate checked multiple times per second (or ideally just measure the interval between every call to glSwap()), with min, avg, max, and stddev frame times (single frame time is 1/fps, so instantaneous fps is 1/frame time).
    Also, a histogram of the game frame times and cyclictest/interbench latency measurements will make it really easy to see what the typical latency will be like. A 3D plot where histograms from various kernels are stacked next to each other will allow comparing kernel versions as part of the ongoing tracking.


    • #42
      Wine would be cool?

      I also think tracking wine's performance would be really interesting and could potentially benefit the project in the long run.

      However, at least for me, using wine together with the xf86-video-ati driver (r300) is basically near impossible, as far as running games(3d) is concerned, especially since switching over to the gallium/dri2/kms stack.
      I'd assume that any performance improvements or regressions that could be measured when rendering direct3d/opengl would be more down to the driver side of things than to wine's code.
      Then again, a comparison to using fglrx under the hood would levitate this discrepancy I guess...
      Probably, other, non-render related tests would be more showing.


      • #43
        Firefox vs. Chromium vs. Konqueror? Maybe just the javascript engines?


        • #44
          Originally posted by Michael View Post
          We are seeking requests for trackers to continuously monitor performance, not individual performance article benchmarks. Post those requests in another thread as I will just be ignoring them in this thread.
          I think asking people to stick to tracker suggestions is an exercise in futility considering the title of the news article.

          Many people, myself included, upon reading the title "Default What Shall We Benchmark Next? Let Us Know!" think you were asking for suggestions on what you will benchmark (as a "one time" thing) and immediately rushed to post with an opinion, skipping the actual article contents in the process.


          • #45
            Originally posted by grantek View Post
            Firefox vs. Chromium vs. Konqueror? Maybe just the javascript engines?
            Sounds good but don't forget epiphany


            • #46
              Wine is an interesting idea, I agree. +1

              I also like the idea of monitoring the performance of some non-Ubuntu distribution, especially one which is specifically designed to be a rolling release (Arch or Gentoo or even Debian Sid), with latest STABLE software, not the way Ubuntu fluctuates between buggy new untested software and old and therefore slow software.

              Arch would be awesome Except there is the slight problem that sometimes it needs user input and configuration to upgrade or continue running after an upgrade. I don't know anything about Gentoo, but I think this could be a good option. Sid would be too much like Ubuntu.

              These are my thoughts


              • #47
                My vote is on oss radeon and mesa. Just like the kernel tracker, but maybe on a per commit basis. And please do include 2D benchmarks like gtkperf.


                • #48

                  My top pick would be a tracker for the open video drivers (specifcally intel, radeon and nouveau) but I understand the difficulty considering libdrm and kernel compatibility issues to which you alluded.

                  My second pick is a tracker for WebKit (that measures the performance of both WebCore and JavaScriptCore). It would be great to catch regressions on something like this since it is used by so many browsers (Chrome, Konqueror, Epiphany, Midori, Safari, etc).


                  • #49
                    zen-kernel vz linux-kernel vs BSD?

                    In that case with all the good stuff enabled that the linux-kernel doesn't have?

                    And the rolling updates shootout


                    • #50
                      oss nouveau vs radeon would be also cool...