Microsoft Makes The DirectStorage API Officially Available

[phoronix]

Phoronix: Microsoft Makes The DirectStorage API Officially Available

Microsoft in late 2020 announced DirectStorage as a new API in the DirectX family focused on delivering faster I/O performance for games to yield quicker game load times and more expansive virtual worlds. After being in a limited developer preview since last year, today Microsoft is making the DirectStorage API broadly available...

https://www.phoronix.com/scan.php?page=news_item&px=Microsoft-DirectStorage-API

[ATFx]

make their own microsofted standards take advantage of, themselves against. no no big no no. nothing they do can be good or for any good reason.

[RahulSundaram]

make their own microsofted standards take advantage of, themselves against. no no big no no. nothing they do can be good or for any good reason.

Wearing blinders like this is not helpful. Like any large company, they occasional do useful and broad standardization that is very useful (a recent example is https://en.wikipedia.org/wiki/Language_Server_Protocol) and occasionally do proprietary standards which aren't broadly helpful outside of their specific platform. So it's more helpful to evaluate efforts on a case by case basis. Documenting an API is always useful even in the case of a proprietary API, so it should be welcome.

[kpedersen]

Documenting an API is always useful even in the case of a proprietary API, so it should be welcome.

Exactly. Especially since the game development community is still a bit naive when it comes to Microsoft and instantly overconsume any feature they (temporarily?) support like it is digital crack.

Obviously an openly documented API helps for when the Wine developers need to pick up the pieces and maintain the long forgotten API decades later.

[ATFx]

Wearing blinders like this is not helpful.

Of course not, Games must explicitly make use of the DirectStorage API.
Documenting an API is always useful. Its just more special Microsoft tie ins to make up for a mess of internal displacements directly in games Microsoft can exploit, sigh is all.

[kiffmet]

I am very sceptical about DirectStorage. The devs have to fetch the compressed data from disk and upload it to the GPU, decompress it asynchronously into a large enough area of free VRAM and manage video memory fragmentation, which besides being a major headache also costs valuable GPU time, in addition to temporarily wasting space.

Modern systems do always have a few CPU cores/threads to spare, so just decompressing the data on-the-fly on the CPU and pushing it directly into video memory with rBAR seems to be a much simpler and more elegant solution. When this is properly scheduled (i.e. gets time-slots), so that it doesn't mess up the frametimes, it should be cheaper aswell.

For me, DirectStorage is just another "unnecessary" development that threatens Linux game compatibility once again, just when it started to get competitive.

[shmerl]

Why does it need a new API? I.e. can't developers already access system I/O interfaces as they need? GPU decompression bit sounds interesting, but I'm sure MS will provide it in some NIH fashion that will require reimplementation.

[chocolate]

Meanwhile, in terms of loading times, Linux platforms are faster for traditional games even on top of Proton right now – and have been for who knows how long – as shown by Linus Tech Tips in a recent video of theirs, and of course, as many individual players have surely noticed as well.

[tildearrow]

I am very sceptical about DirectStorage. The devs have to fetch the compressed data from disk and upload it to the GPU, decompress it asynchronously into a large enough area of free VRAM and manage video memory fragmentation, which besides being a major headache also costs valuable GPU time, in addition to temporarily wasting space.

Modern systems do always have a few CPU cores/threads to spare, so just decompressing the data on-the-fly on the CPU and pushing it directly into video memory with rBAR seems to be a much simpler and more elegant solution. When this is properly scheduled (i.e. gets time-slots), so that it doesn't mess up the frametimes, it should be cheaper aswell.

For me, DirectStorage is just another "unnecessary" development that threatens Linux game compatibility once again, just when it started to get competitive.

Parallel decompression? Say, for example:
- 1MB blocks compressed to "chunks" and decoded in parallel
- For a 1GB data file, this means running 1024 decompression jobs

[mdedetrich]

I am very sceptical about DirectStorage. The devs have to fetch the compressed data from disk and upload it to the GPU, decompress it asynchronously into a large enough area of free VRAM and manage video memory fragmentation, which besides being a major headache also costs valuable GPU time, in addition to temporarily wasting space.

Modern systems do always have a few CPU cores/threads to spare, so just decompressing the data on-the-fly on the CPU and pushing it directly into video memory with rBAR seems to be a much simpler and more elegant solution. When this is properly scheduled (i.e. gets time-slots), so that it doesn't mess up the frametimes, it should be cheaper aswell.

For me, DirectStorage is just another "unnecessary" development that threatens Linux game compatibility once again, just when it started to get competitive.

Yeah, try doing that for gigabyte sized textures that are streamed in real time...

DirectStorage is basically Microsoft's solution for what is on the PS5 and otherwise there isn't anything really comparable for PC systems. Standard file API's (including IO_uring) add too much overhead especially with SSD based NVMe drives and they are also not the right API for the job.

For linux to solve this problem they would also have to come up with their own API, there might also be issues with the NVidia blob in doing this.