Announcement

Collapse
No announcement yet.

RADV Adds Knobs To Force Shader Re-Compilation - Helping Games On The Steam Deck

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • #11
    Originally posted by skeevy420 View Post
    blackiwid

    By even having non-free repositories, hosting "for educational purposes" software, or even allowing discussions that go in-depth with links and how-to manuals, the hosters, like Michael here at Phoronix, put themselves at risk to prosecution and punishment for what assholes like you and me say and do. DMCA-like laws have since trickled down into other allied countries so there are few safe havens to host questionable content. ...


    Piracy isn't an advantage of a Steam machine, it's an advantage of any personal computer in existence.
    Well it's literally legal for me to download over a file hoster a game or something, be it for the switch or for pc. Yes it's legal, what is not legal is torrent, because I am seen as a commercial Provider and seller some big kingpin, if I download my stuff over that, they pretend because 1000 people dl 500kb from me that I provided everybody with the full copy of let's say 100gb. But that is just a side note, the point is it's legal, that said I pay for that with every USBstick that costs probably 4 times as much as in the US and Printer and other stupid stuff, there is basically some sort of piracy tax (but that does not allow piracy in general but just in some special way as I described, the more typical use would be the physical sharing with friends.

    So if I am allowed literally to break the copyright laws, it's ok but if we talk about it, not? I mean I don't want to share dl links here if you fear that

    Why do I bring that topic up under a news about steam, because steam literally makes piracy harder, and not only that even using mods in legal distributions like GOG, try to install the graphical mods for skyrim in the GOG version? some can't be installed at all.

    But just simpler install another game with mods like don't starve from another legal source, try to install mods that 99% now are exclusively behind the steam paywall... it's possible at least, but it's a pain in the ass, using some strange steam console client, which basically for the average gamer means it's impossible.

    Yes thanks to valve wine/proton is better, but the controller mapping is exclusive to steam sells which is not really fair, the linux community gave a lot to valve a OS they did not have to implement, drivers a display server etc... the funniest thing is that most of linux VR is dependent on constantly starting steamvr, so Steam became a driver for some monitor like device... commit to real open technology and not have this proprietary garbage everywhere, it's not all valves fault, some devs are just to lazy or want to appeal to more users, but some certainly is.

    Well it will eventually happen that you have better installers for piracy but meanwhile it would be nice if linux is even start-able without steam as preloader and it doesn't become process 0 or something... (obviously polemic, but who knows google did something similar than process 0 for chrome in chromeos, at least they learned there lessons and made chromeos more open and kicked chrome as preloader for everything out.

    Comment


    • #12
      Don't call something a "knob" which is not a knob.

      Comment


      • #13
        Originally posted by kiffmet View Post
        loganj Yes, the shaders are compiled using the CPU.

        Steam has a mechanism, where it can download all of the games' "raw" shaders and "replay" them before launching a title.

        What happens every now and then though, is that changes/fixes to Mesa's shader compiler get backported.
        Such changes can break the shader ABI and should always invalidate the cache, which doesn't happen.

        This can manifest as a variety of issues: I.e. subpar performance, graphical glitches in games.

        Thus, there's a need for an override, to force repopulating the cache with freshly compiled shaders.
        An alternative would be deleting ~/.cache/mesa_shader_cache and ~/.cache/radv_builtin_shaders{32,64}, but that's not on a per-game basis and forces the shaders for all applications to be recompiled, which can take a huge amount of time, esp. with large game libraries.


        Exactly, Oddly Mesa Developers don't want to Acknowledge Mesa Shader Cache only being a Problematic Function.

        #1 Its an Unnecessary Extension of Mesa that doesn't Respect the Individual Functioning of Drivers within Mesa that prefer to have their own cache handling or not.
        The only supposed Benefit of Mesa Shader Cache is being a waste of raw performance that could've gone to games directly.

        #2 This is Exactly how to Develop against a Flawless Console Experience in Mind.
        By Continuing to Opt for Throwing Tedious Complexities on the User End instead of Fixing the Problems in Mesa.

        #3 Proton Cache is what stores the game specific shaders for every windows game individually & determines the frame drops.
        It use to be the only Cache, its the only Cache that made sense as it actually helped game performance & should've remained as the Only.

        #4 Then Valve added a Shader Pre Caching option to Steam to Download Shaders then Mesa Shader Cache was Implemented into Mesa later on.

        An Alternative to Dealing with Caches, export MESA_SHADER_CACHE_DISABLE=true & export RADV_DEBUG=nocache​ for Radv​

        Comment


        • #14
          ATFx
          Its an Unnecessary Extension of Mesa that doesn't Respect the Individual Functioning of Drivers within Mesa that prefer to have their own cache handling or not.
          RADV had its own caching logic and switched to the common Mesa one. The result was 60% space savings for users and faster cache hits due to less data needing to be traversed. RADV devs weren't forced to do that - the common implementation just turned out to be better.​


          So this is how shader caching works with Steam:

          VKD3D/DXVK cache stores DXIL/DXBC -> SPIR-V translations, whereas Steam's fossilize tool stores a serialization of Vulkan pipeline objects (which may be vendor or device specific). Those two are downloaded during pre-caching.

          The SPIR-V and pipeline objects still need to be put together and compiled by the graphics driver in order to get the specific binaries that ultimately run on your GPU - that part happens during "replay" and its only purpose is to populate the actual driver (Mesa) shader cache.

          Fossilize is smart enough to point Mesa towards SteamLibrary/steamapps/shadercache/<appid>/mesa_shader_cache_* so that you don't have two separate caches holding the same data (with the other, hypothetical one, residing in the standard ~/.cache location). Steam uses set conditions, as well as heuristics in order to determine when to invalidate that last cache, but seemingly, there are some rare corner cases left (i.e. updating Mesa while Steam is running) - restarting Steam does the trick 99% of the time though.

          So what you're essentially doing, is deferring that last compilation step into the game's runtime. If you were using AMDVLK or RADV/LLVM, you'd very likely notice this as stuttering. RADV/ACO tends to be fast enough to avoid that in most of the time. Hitting a hot cache is still faster though.

          When running a title outside of Steam or Lutris (which can also do precaching for select titles), your settings essentially only lead to a DXIL -> SPIR-V translation cache slowly being created/populated at runtime and nothing more. Mesa still keeps a RAM cache, but that one is discarded after closing the application.

          Your rant is utterly unfounded and your "suggested fix" is as counterproductive as can be.

          Comment


          • #15
            Originally posted by kiffmet View Post
            ATFx


            Your rant is utterly unfounded and your "suggested fix" is as counterproductive as can be.
            your rant is utterly unfounded & your suggested no fix is as counterproductive as can be.

            There is Nothing Smart about Mesa Shader Cache, its Systemic Level Caching that Generates & Conflicts with itself
            waste cpu resources causes Stuttering/Glitches/Crashes whatnot & Never Clears it's Own Cache so it takes up Storage space.


            This is the only benefit of it if you actually knew how the Caching Worked,
            what was actually a Driver from what is Mesa & what is Steam from what is Mesa.
            Because there is No Connection between Mesa Shader Cache & Steam.....I take that Back, wow.
            Whenever mesa-db cache hits size limit, a half of cache is evicted. It works fine in general, but some use-cases are suffering from the large cache evictions. In...


            image.png
            Its essentially only a mechanism to Data Mine Steam users Hardware. Definitely not a Real Pro having.
            Last edited by ATFx; 15 November 2023, 11:39 AM.

            Comment


            • #16
              kiffmet i still don't understand. u said shaders are compiled using CPU but then you said that the GPU are compiling them? so which one is it? as far as i found on the internet all those compiling times have nothing to do with the GPU. only CPU affects the time it takes to compile those shaders.

              so what i don't understand?

              Comment


              • #17
                Originally posted by loganj View Post
                kiffmet i still don't understand. u said shaders are compiled using CPU but then you said that the GPU are compiling them? so which one is it? as far as i found on the internet all those compiling times have nothing to do with the GPU. only CPU affects the time it takes to compile those shaders.
                I believe kiffmet said that the GPU driver is compiling the shaders, not that the compilation is running on GPU hardware.
                Test signature

                Comment


                • #18
                  bridgman so the CPU use gpu drivers only to compile the shaders? isn't faster to use the gpu hardware directly?

                  Comment


                  • #19
                    loganj compilation is done on the CPU by a compiler that's shipped with the graphics driver. The task of compiling code requires lots and lots of "if-else" and "switch" statements, which makes general purpose CPUs much better suited for this job, than a massively parallel array of number crunching units (GPUs).

                    Comment


                    • #20
                      kiffmet i see. finally i understand why CPU is the one that is in use and why graphic driver matter.

                      Comment

                      Working...
                      X