Announcement

Collapse
No announcement yet.

Dota 2 Test broken

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Dota 2 Test broken

    I think some recent game update broke how pts received the numbers from the benchmark.

    Before I got results between 110-120 fps now its much lower.
    When you watch the benchmark you can clearly see that fps is still in that range. So there is no system regression.

    Resolution: 1920 x 1080 - Renderer: Vulkan:
    14.8
    15.4
    14.4

    Looks almost like its returning minimal fps now instead of average.

  • #2
    I noticed the same thing. Based on the csv output that number is the frame variability. I'm not sure how the results parser works to tell whether this is a bug but the fix is to edit the results-definition.xml of the test and remove UNKNOWN i.e. :

    Code:
    <OutputTemplate>pts2.dem,UNKNOWN,#_RESULT_#, 17.5, 30.1,3840,2160 (...)
    changed to

    Code:
    <OutputTemplate>pts2.dem,,#_RESULT_#, 17.5, 30.1,3840,2160 (...)

    Comment


    • #3
      Apologies for the delay, I've now been able to test it out. Indeed, the latest Steam updates to Dota 2 tweak the output of the FPS information from the game... The test profile has now been adjusted to account for the latest dota2. This should be fixed by the latest test profile update, pts/dota2-1.2.5.
      Michael Larabel
      https://www.michaellarabel.com/

      Comment


      • #4
        Thx for fixing this. Now I can continue my regression testing.

        Comment


        • #5
          Dont know if thats intended behavior but it seems that the new test isnt used on old benchmark presets.
          I have many datapoints from old mesa versions and want to compare to the current one. But pts uses everytime 1.2.2, only if I create a new preset its using 1.2.6.

          Comment


          • #6
            Originally posted by Strunkenbold View Post
            Dont know if thats intended behavior but it seems that the new test isnt used on old benchmark presets.
            I have many datapoints from old mesa versions and want to compare to the current one. But pts uses everytime 1.2.2, only if I create a new preset its using 1.2.6.
            That's intended to ensure past results are 1:1 comparable with new runs, but in your old results you can simply sed/find-and-replace the version with the new version.
            Michael Larabel
            https://www.michaellarabel.com/

            Comment

            Working...
            X