Announcement

Collapse
No announcement yet.

Portal & HL2 Lost Coast Linux Benchmarks Added

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • Portal & HL2 Lost Coast Linux Benchmarks Added

    Phoronix: Portal & HL2 Lost Coast Linux Benchmarks Added

    Users of the Phoronix Test Suite and OpenBenchmarking.org can now easily fire-up Half-Life 2: Lost Coast and Valve's Portal for some new Linux game benchmarking...

    http://www.phoronix.com/vr.php?view=MTY0ODU

  • #2
    Can't you use a single Steam game library that you keep on a harddrive or similar.
    Then you just can tell Steam to not update those games.

    Comment


    • #3
      Honestly, I don't think it matters that Valve patches their games and you can't roll back to older releases. Nobody cares about the performance of an older version of software anyway, and it's not like the performance in a game from a studio that only releases games that are heavily optimized, and all the patches henceforth don't really affect performance anyway, is going to change performance at all just because a patch is released that fixes non-performance-related bugs. In fact, I can't recall any articles on here where you compared performance of an older benchmark with a fresh benchmark -- they are always carried out with fresh benchmarks and references to prior benchmarks you performed. In that case, it really makes no difference at all.

      Comment


      • #4
        Originally posted by Pajn View Post
        Can't you use a single Steam game library that you keep on a harddrive or similar.
        Then you just can tell Steam to not update those games.
        That's no use for fair cross-platform testing, unless you keep such a library for each platform and take care to update them at the same time. Even if you do that, it's still very hard for anyone else to replicate your test because they'd need to have saved the same version (it's proprietary software, so you can't send your copy to them).

        Comment


        • #5
          Originally posted by mmstick View Post
          Honestly, I don't think it matters that Valve patches their games and you can't roll back to older releases. Nobody cares about the performance of an older version of software anyway, and it's not like the performance in a game from a studio that only releases games that are heavily optimized, and all the patches henceforth don't really affect performance anyway, is going to change performance at all just because a patch is released that fixes non-performance-related bugs. In fact, I can't recall any articles on here where you compared performance of an older benchmark with a fresh benchmark -- they are always carried out with fresh benchmarks and references to prior benchmarks you performed. In that case, it really makes no difference at all.
          Exactly: who cares about benchmarks run on old, crusty ass unpatched versions of software? Suppose there's some poor ass performance in some game X where you're getting 16fps on card Y. It's patched so that one now gets 48fps on the same card. Would anyone care that the same card performs just as miserably on a newer kernel on the crusty old version of the game as it did back then? Or that some newer card Z on the same architecture (e.g. Pitcairn) performs also miserably on the crusty old version of the game? I doubt it! I sure as hell don't!

          Comment


          • #6
            http://openbenchmarking.org/result/1...SO-1403310PT15

            NVIDIA GEFORCE GTX 770 vs radeon 7790 + Mesa 10.2-git

            Comment


            • #7
              Originally posted by molecule-eye View Post
              Exactly: who cares about benchmarks run on old, crusty ass unpatched versions of software? Suppose there's some poor ass performance in some game X where you're getting 16fps on card Y. It's patched so that one now gets 48fps on the same card. Would anyone care that the same card performs just as miserably on a newer kernel on the crusty old version of the game as it did back then? Or that some newer card Z on the same architecture (e.g. Pitcairn) performs also miserably on the crusty old version of the game? I doubt it! I sure as hell don't!
              When you are comparing the same card on two different version of the kernel you do care if it was a fix in the actual game or not. Phoronix is striving to make comparisons betwean benchmark results valid wich puts this restraint on how the software has to be set up.

              Comment


              • #8
                Originally posted by Pontostroy View Post
                http://openbenchmarking.org/result/1...SO-1403310PT15

                NVIDIA GEFORCE GTX 770 vs radeon 7790 + Mesa 10.2-git
                What driver did you used with 7790 , the OSS RADEON ?
                IF so, it wasn't a fair comparison....or complete...you should have done it with Catalyst also....

                Comment


                • #9
                  Originally posted by AJenbo View Post
                  When you are comparing the same card on two different version of the kernel you do care if it was a fix in the actual game or not. Phoronix is striving to make comparisons betwean benchmark results valid wich puts this restraint on how the software has to be set up.
                  Are you telling me that the sole or main purpose of Phoronix benchmarking is troubleshooting? To see whether there's a kernel regression/improvement or whether it was in the game? That's an interesting goal no doubt, but probably more people are just interested in pure linux performance (and relative performance, i.e. to other OSes) given the latest software on the latest kernel and drivers.

                  Comment


                  • #10
                    Yeah, it would be really nice if you could specify what version (inc "current") that you would like the benchmark to run. But i guess that is the only option you get with the Steam games, so they are a good pick when you are doing a batch of benchmarks to see the current state of things.

                    Comment


                    • #11
                      Originally posted by Pajn View Post
                      Can't you use a single Steam game library that you keep on a harddrive or similar.
                      Then you just can tell Steam to not update those games.
                      No, that's only good for local testing... What happens if User D or Company X wants to reproduce the results remotely?
                      Michael Larabel
                      http://www.michaellarabel.com/

                      Comment


                      • #12
                        Originally posted by molecule-eye View Post
                        Exactly: who cares about benchmarks run on old, crusty ass unpatched versions of software? Suppose there's some poor ass performance in some game X where you're getting 16fps on card Y. It's patched so that one now gets 48fps on the same card. Would anyone care that the same card performs just as miserably on a newer kernel on the crusty old version of the game as it did back then? Or that some newer card Z on the same architecture (e.g. Pitcairn) performs also miserably on the crusty old version of the game? I doubt it! I sure as hell don't!
                        It's all a matter of reproducibility, not a matter of wanting to intentionally run older versions of software. With PTS testing, all tests are version controlled, configurations saved, etc. Any independent user or company can reproduce the exact tests I did (or any other party did for that matter). With Steam, you lose that ability due to having the game version handling being outside of my control. It also matters when running constant Mesa/Linux kernel benchmarks and wanting to keep the version constant -- and then when adding new test systems to that harness, you lose the ability unless manually copying files from one system to the next.
                        Michael Larabel
                        http://www.michaellarabel.com/

                        Comment


                        • #13
                          Excellent news!

                          If the version of the games can be automatically retrieved, then maybe appending it to the game’s name could sort of fix the problem of comparing different versions… If the version is different you know you can’t 100% trust the comparision, but you still know the current performance, which is after all quite important .

                          One thing missing from the benchmarks is the game’s settings.

                          Comment


                          • #14
                            Originally posted by stqn View Post
                            Excellent news!

                            If the version of the games can be automatically retrieved, then maybe appending it to the game’s name could sort of fix the problem of comparing different versions… If the version is different you know you can’t 100% trust the comparision, but you still know the current performance, which is after all quite important .

                            One thing missing from the benchmarks is the game’s settings.
                            Unfortunately I am not even aware of any universal Steam commands or files storing a game's revision/version information... Short of just trying to MD5 the main binary or something to get some sort of identifier that would only be rather useless.
                            Michael Larabel
                            http://www.michaellarabel.com/

                            Comment


                            • #15
                              Originally posted by AJSB View Post
                              What driver did you used with 7790 , the OSS RADEON ?
                              IF so, it wasn't a fair comparison....or complete...you should have done it with Catalyst also....
                              The NVIDIA 770 is a much more powerful card than the 7790, even if the drivers were equal. In fact it's about twice as fast, looking at Passmark's MS Windows results (probably not the best benchmark in the world from what I've heard, but a reasonable comparison point). That difference is even larger in these benchmarks, but not really surprising given the additional performance hit of the FOSS drivers. Catalyst benchmarks would be interesting, but I don't blame him for not testing as the driver is mostly a pain in my experience.

                              Finally, there are some system differences that could be affecting things, such as a somewhat stronger CPU in the 770 system. Probably not enough to dramatically skew the results but something to take into account.
                              Last edited by AnonymousCoward; 03-31-2014, 12:13 PM.

                              Comment

                              Working...
                              X