Announcement

Collapse
No announcement yet.

NVIDIA vs. AMD OpenGL & Vulkan Benchmarks With Valve's Dota 2

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • dang_valve
    replied
    This is Dan, one of the developers of the Source 2 Vulkan render backend at Valve. I'm trying to replicate the performance results here. I downloaded the pts.dem file from http://www.phoronix-test-suite.com/b...ts-dota2-2.zip and I believe I am using a similar command-line based on http://openbenchmarking.org/innhold/...a1110a2808cf00 .

    Command-line:
    [-vulkan | -gl ] +timedemoquit pts2 +demo_quitafterplayback 1 +cl_showfps 1 +fps_max 0 -novconsole -fullscreen +timedemo_start 1 +timedemo_end 1000

    My system is Intel i7-3770k NVIDIA TITAN X and I tested using the 367.18 driver on Ubuntu x64 15.10 4.2.0.27. I don't have a 4k monitor, but I see:
    • Vulkan: 138.8fps GL: 139.6fps (2560x1440 Best Looking)
    • Vulkan: 149.7fps GL: 142.0fps (800x600 Best Looking)
    I need to test at 4k to compare results closer, but some thoughts from me:
    • I have some suspicion that perhaps the game/dota/cfg/video.txt settings used for each of these tests was not identical. Dota 2 will reconfigure settings automatically when there is a new PCI deviceID/vendorID detected and that would skew results massively if the settings are not the same. You could add -autoconfig_level 3 to your command-line for example to test Best Looking settings and not have Dota 2 autodetect. If you are not verifying the settings are the same between GPU changes then that could explain why the results are so strange.
    • We have spent most of the effort improving perf for CPU limited situations so if you could include some lower res scores (still at Best Looking) that would be good.
    • I do not expect (and have not been able to replicate) such large GPU perf drops so I'm not sure yet why I'm not replicating it. I would have expected GPU perf to be close between Vulkan + GL. If the video settings did not match between the GL and Vulkan tests that might explain it.
    • Your timedemo is a fair test, but it is fairly low in terms of activity. The more particles/models/etc. that are on-screen during the timedemo the better I expect Vulkan to do since that is when we tend to be come renderthread CPU bound on the other APIs (GL/DX9/DX11)
    • In CPU limited situations I am seeing very large gains on AMD Vulkan vs GL.

    Leave a comment:


  • jntesteves
    replied
    Originally posted by schmidtbag View Post
    Seems people here really aren't paying attention... Did you not read the page regarding CPU usage? Sure, he's using a high resolution, but Vulkan is already proving it's worth. The drop in CPU usage, even at 4K, is hugely significant. If you have a 1080p display and any of the GPUs that were tested, you don't need Vulkan.
    I think the drop in CPU usage can be directly attributed to the drop in framerate in this case. But even at 1080p and with said GPUs a load can still be CPU bound on weaker AMD CPUs or 15W laptop CPUs from Intel. I believe these would be much more interesting hardware to compare Vulkan with older APIs. I think the big selling point of Vulkan over OpenGL is in the mobile space, and gaming on small laptops will eat away desktops' market, eventually. Some smaller laptops are already out with 15W CPUs and still dubbed "gaming". I had a Lenovo Y40 two years ago, it had great graphics from AMD (10 CUs, 2GB), but the 15W CPU meant it would never stress the GPU. My current laptop with a much weaker GPU from AMD (6CUs, 1GB) always outperforms that one due to the 45W CPU alone.

    Leave a comment:


  • justmy2cents
    replied
    Originally posted by slacka View Post
    Why are you benchmarking a new API whose primary purpose is to reduce CPU at 4K? All you are doing is measuring fill-rate and ensuring that the app is 100% GPU bound and CPU sitting mostly idle. If you actually want to see how Vulkan improves over OpenGL, you need to benchmark at much lower resolutions.
    you should realize that if benchmark was done at scenario where cpu was at 100% it would be unreliable. once you hit 100% it will stay there and only lower frame rate. you can only reliably see the difference when neither scenario hits 100%. once you see how much lower cpu usage is, it is simple to know which cpu will now suffice

    Leave a comment:


  • boxie
    replied
    Also - as a second thought. Since Vulkan renderers are multithreaded, and CPU overhead is lower - I wonder if the CPU governor has a bit more to play?

    I might be blind but I could not see anything in the article about which CPU governor was used. It would be interesting to how different power states affect the game.

    Leave a comment:


  • boxie
    replied
    Awesome to see some Vulkan tests I am pretty sure that Valve will be further optimising their game and learning lots about how to implement a Vulkan engine.

    As for those that are complaining about "invalid tests" - STFU and quit your whining and whinging. Vulkan is supposed to help when CPU bound, but NOT regress otherwise.

    Leave a comment:


  • schmidtbag
    replied
    Seems people here really aren't paying attention... Did you not read the page regarding CPU usage? Sure, he's using a high resolution, but Vulkan is already proving it's worth. The drop in CPU usage, even at 4K, is hugely significant. If you have a 1080p display and any of the GPUs that were tested, you don't need Vulkan.

    Leave a comment:


  • SaucyJack
    replied
    Originally posted by eydee View Post
    Maybe the 980 Ti is the one that is "misbehaving", not all the others. It is better of course, but not that much better, especially compared to the 980. (We can see almost 200% performance.) I suggest running image quality tests too, if PTS can do it. This is all too fishy. Seeing Windows Direct3D results right next to these one could also be interesting.
    Something fishy with a nvidia driver? Say it ain't so...

    Leave a comment:


  • SaucyJack
    replied
    Not sure what all of the hubbub is about, valve accomplished exactly what they were going for. Something PTS doesn't seem to cover is frame time variance, which they knocked down to ~15ms from ~45ms.

    Leave a comment:


  • darkbasic
    replied
    WTF Michael, the whole point about Vulkan are CPU-bound scenarios, why did you test only 4K which is the less interesting resolution?

    Leave a comment:


  • eydee
    replied
    Maybe the 980 Ti is the one that is "misbehaving", not all the others. It is better of course, but not that much better, especially compared to the 980. (We can see almost 200% performance.) I suggest running image quality tests too, if PTS can do it. This is all too fishy. Seeing Windows Direct3D results right next to these one could also be interesting.

    Leave a comment:

Working...
X