Announcement

Collapse
No announcement yet.

Running The Latest Windows 10 vs. Ubuntu Linux OpenGL/Vulkan Benchmarks

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • Running The Latest Windows 10 vs. Ubuntu Linux OpenGL/Vulkan Benchmarks

    Phoronix: Running The Latest Windows 10 vs. Ubuntu Linux OpenGL/Vulkan Benchmarks

    Now that my Linux reviews of the GeForce GTX 1070 and GeForce GTX 1080 have been published, next on my agenda this week are running some fresh Windows vs. Linux graphics benchmarks with these Pascal graphics cards...

    http://www.phoronix.com/scan.php?pag...l-Win-vs-Linux

  • #2
    Michael, I'd love to see x-plane included in the test, I remember you used to test it a few years ago

    Comment


    • #3
      Originally posted by kbios View Post
      Michael, I'd love to see x-plane included in the test, I remember you used to test it a few years ago
      Unfortunately X-Plane hasn't updated their benchamrk mode in a long time.
      Michael Larabel
      http://www.michaellarabel.com/

      Comment


      • #4
        Can Quake-3 break 3,000 FPS? :-)

        Nah, just kidding.

        Comment


        • #5
          Originally posted by Zan Lynx View Post
          Can Quake-3 break 3,000 FPS? :-)

          Nah, just kidding.
          The funny thing is that you connections problems and things start to lag, when you uncap the FPS on Quake 3 and go into the 4 digits FPS counts.

          Comment


          • #6
            Hi Michael,

            Something I'd really like to see in any of your future Vulkan benchmarks is the quantification of 'smoothness' or "absence of (micro)stuttering/jitter" many users seem to be particularly enthusiastic about.
            Sometimes your benchmark results include "SE +/- ..." labels for the "Frames Per Second" metric, but I suppose these numbers indicate the inter-testrun stddev of the average framerate, rather than the inverse of the inter-frame stddev of the frametime.
            There are several possible metrics to try to quantify this 'smoothness', of which some I list here:

            - "StdDev of Frametime In Milliseconds, Less is Better":
            Calculate the standard deviation of the frametimes of all frames to obtain the metric.
            Advantage: might capture lower frequency stuttering, such as slowly fluctuating/oscillating framerates
            Disadvantage: breaks down when multiple rendering scenes with very different average framerates are present in the testrun

            - "StdDev of Inter-frametime Difference In Milliseconds, Less is Better":
            For each two successive frames, calculate the difference between their frametimes,
            then calculate the standard deviation of all those differences to obtain the metric.
            Advantage: some not-frequently occurring stuttering will be captured in the metric
            Disadvantage: since it uses the average (of squares), sudden drops in frametime (at the moment when rendering scene changes) will be captured in the metric as well

            - "Median Absolute Inter-frametime Difference In Milliseconds, Less is Better":
            For each two successive frames, calculate the absolute difference between their frametimes,
            then calculate the median of all those absolute differences to obtain the metric.
            Advantage: since it uses the median, sudden drops in frametime (at the moment when rendering scene changes) will be filtered out
            Disadvantage: some not-frequently occurring stuttering could be filtered out as well
            Another problem with the Inter-frametime metrics, is that we only consider two successive frames,
            rather than a larger window (which better matches our human perception of an image sequence).

            - "Median Rolling StdDev of Frametime In Milliseconds, Less is Better":
            For all W successive frames, where W denotes the number of frames in a rolling window, calculate the standard deviation of their frametimes,
            then calculate the median of all those standard deviations to obtain the metric.
            For W, one can use 30: 500ms @ 60fps:
            500ms because frametime is approximately constant during this period for the scenes of most benchmarks (see test-data for your reference);
            60fps because I assume the (fixed!) timestep used in the benchmarks' renderers is 1 / 60fps.
            Advantage: now we consider a larger window, and we capture both (fairly) low and high frequency stuttering
            (Disadvantage: ) scene changes will be captured as well (undesirable), however this is filtered out by taking the median

            - "Percentile Low-High Gap of Frametime In Milliseconds, Less is Better":
            Calculate the difference between the 99% and 1% percentile of the frametimes of all frames to obtain the metric.
            This method originates from a hint at the end of http://techreport.com/review/21516/i...benchmarking/6 for defining a 'jitter' metric,
            however I modified the 50-99% percentile to 1-99% percentile for symmetry purposes.
            Advantage: this is almost the same as difference between min and max frametime, and thus represents the total span of the frametime
            Disadvantage: breaks down when multiple rendering scenes with very different average framerates are present in the testrun

            - "Median Rolling Percentile Low-High Gap of Frametime In Milliseconds, Less is Better":
            For all W successive frames, where W denotes the number of frames in a rolling window, calculate the difference between the 99% and 1% percentile of their frametimes,
            then calculate the median of all those differences to obtain the metric.
            Again, for W, we can use 30.
            Advantage: now we consider a larger window, and we capture both (fairly) low and high frequency stuttering
            (Disadvantage: ) scene changes will be captured as well (undesirable), however this is filtered out by taking the median
            You'll notice this metric is very similar to the "Median Rolling StdDev of Frametime In Milliseconds, Less is Better" metric,
            in most cases this one is simply a multiple of the latter.

            The reason why I don't include metrics such as "#Frames with Frametime Higher than 1 / 60fps, Less Is Better",
            is because for most games on PS3 or XBox 360 consoles, a consistent framerate of 30fps is used,
            and this still looks better than a game having an average framerate of 75fps with frequent drops to 30fps.

            I think "Median Rolling StdDev of Frametime In Milliseconds, Less is Better" is the best one,
            in a new post I'll test these metrics and try to convince you, using this test-data:
            https://www.phoronix.com/forums/foru...624#post883624
            Feedback is of course encouraged.
            Last edited by Eliasvan; 07-09-2016, 04:40 AM.

            Comment


            • #7
              Sascha Willems has written examples for Vulkan, quite nice ones, in fact.

              https://github.com/SaschaWillems/Vulkan

              These compile under Windows and Linux. When you compare their performance for each OS then you might be able to get a more direct comparison of the Vulkan implementations themselves, as well as possible clues where there to find differences in detail.

              Comment


              • #8
                Collected so far.... $35.....
                Michael Larabel
                http://www.michaellarabel.com/

                Comment


                • #9
                  I'd just like to see as many actual games as possible, at realistic speeds (under 150fps?)

                  So no Quake 3, no Furmark, no Unigine, etc. (you can throw those on additionally, but real games are more interesting)
                  Last edited by smitty3268; 06-15-2016, 07:15 PM.

                  Comment


                  • #10
                    Originally posted by Michael View Post
                    Collected so far.... $35.....
                    You better get some food stamps...

                    Comment

                    Working...
                    X