Khronos Ratifies KTX 2.0

[CochainComplex]

To be more technically, DX has its own formats, and an engine using those would not need to use KTX, as those formats are supported on most recent GPUs. (As in the last 10 yrs.) They aren't as size efficient as the latest and greatest, but they are faster than converting, which is what is being targeted here.

There are several ways to tackle memory pressure on the CPU to GPU link, and the first is raw throughput. PNGs have to be converted on the CPU, then sent to the GPU in an uncompressed format, because the compression takes too long in the CPU. (This is not so true in a small indie 2D game, but is true in any sizable 3D game.) So this handles that case by doing the conversion upfront, and while they are at it, using the best compression tech freely available at this time to reduce the requirements, which increases throughput, or technically the number of textures that can be loaded per second.

Where do I hope to see this used? The Dolphin emulator has support to external textures. If they adopted support for this, it would allow for better performance at 4k.

Also, keep in mind, shaders are designed to use the exact textures the engine specifies. Not ot get too technical, but the DX formats actually throw away info in the format to get smaller sizes, and the shaders are made while looking at that reduced quality. These formats make different tradeoffs. It can cause artifacting and other issues. Have you ever saved a JPEG, edited it and saved it as a JPEG again? Or converted a movie from one encoding to another and notice there is a difference? You are basically taking all the worst qualities of the first format and adding the worst qualities of the second format for a new image. Now you have lost quality, that you can't magically get back. (There is some interesting work with AI, check out the FF9 up-scaling project Moguri. ) This can manifest in onscreen glitches, blurry textures, off color textures, particle effects that don't look correct, and who knows what else.

Not all games will benefit from this, because engines do things like group all the textures together into one big texture, and now you may get worse compression than you originally thought. (MegaTextures, think id's Rage game, were a big thing in the industries for a bit, and is now in Unreal Engine by default, IIRC. You have to select the feature to use it, though.) These tend to be highly optimized already, so it would be difficult to see any major speed gains from this.

Thank you for explaining it and giving your valuation of this vulkan extension. No speedkick for dxvk or vkd3d but it will be usefull for the mentioned scenarios and new developments.

besides also thx for the FF9 upscale mod link. As it is one of my favorite FF's, this was a real delightful surprise and well done with the AI based upscaling. Havent expected that this kind of mod is possible without having a set of the original textures (likely higheres then the result of the game).
I still hope for the Blade Runner Point and Click adventure 4k remake (1997 Westwood studios). Unfortunately the original Textures and Game Dev data is not existing anymore. They are also relying on AI based upscaling - but don't know the current status. According to the schedule proposed years ago it should be ready soon...well pandemic ..who knows.

[CochainComplex]

IF I am understanding right, (I have not read khornos article yet) this will allow compressed images to be stored on vram AND decoded within the gpu? this is a massive preformance gain for any texture bound game. not only will the save vram usage which is a massive sink now, but also increase performance on games that need to rapidly load and unload textures. this would effectively kill the need for texture streaming (mip map streaming) to a large degree. This is very exciting indeed. especially considering that any vulkan game could benefit, and it is not engine specific.

my thought was not so sophisticated as yours but that was on my mind - but seems to be difficult to implement it this way for dxvk/vkd3d...but vulkan engines of the future

[dragorth]

Why would this kill the need for mipmaps? They are an optimization to achieve the same or similar results, and can be used together. KTX2 also supports mipmaps implicitly.

From the Spec Doc.

Abstract


KTX^™️ (Khronos TeXture) is an efficient lightweight container file format for reliably distributing GPU textures to diverse platforms and applications. It is distinguished by the simplicity of the loader required to instantiate texture objects from the file contents. The contents of a KTX file can range from a simple base-level 2D texture to a cubemap array texture with mipmaps. KTX files hold all the parameters needed for efficient texture loading into 3D APIs such as OpenGL® and Vulkan®

Version 2 extends the functionality of version 1 with easier loading of Vulkan textures, easier use by non-OpenGL and non-Vulkan applications, the possibility of streaming, through sending small mip levels first, universal textures using Basis Universal texchnology and supercompression. Providing this new functionality requires a significantly different file structure from version 1.

[Quackdoc]

my thought was not so sophisticated as yours but that was on my mind - but seems to be difficult to implement it this way for dxvk/vkd3d...but vulkan engines of the future

IT wouldn't have any noticeable benefit, but would rather be detrimental, rightnow the process is,
Traditional (wastes Vram)
storage -> ram(staging) -> cpu(Decode) -> waste space on vram(Sit until used) -> used

Texture streaming (Wastes CPU resources)
storage -> ram(staging) -> (Wait until game calls the new level of mipmap) -> cpu(decode) -> vram->(Used)

and with KTX;
KTX
Storage -> vram (wait until used) -> (used)

this is pretty much the best solution, as it cuts down latency and cpu usage from decoding it from cpu when compared to texture streaming, saves space in vram compared to traditional. The downside is obvious, decoding need be done on your gpu, so if you are extremely gpu bottlenecked, this may not be ideal... but I doubt the overhead would be negative in most cases.

(Generally textures can be decoded when the cpu is not busy, or before gameplay starts. Texture, or mip map streaming decodes the needed mipmap level real time which can lead to more cpu time, but greatly can decrease vram so better for "slower" games like rpgs, not good for fps)

OKAY necessary background junk out of the way. how would this be implemented in dxva.

you would need to convert it, for games that don't have it, game that do would just be treated normally when support rolls out if needed, so it would look like this.

storage -> ram -> cpu(decode) -> ram(raw) ->cpu(encode) -> vram...

Needless to say, not efficient

[Azrael5]

Each software package needs to adopt support for such IIRC. So Blender, Three.JS (web js lib), Unity and Unreal, and so forth. Expect them to arrive at different times, and iirc with earlier glTF/KTX stuff support also varied.

Blender for example I'm pretty sure has stuck to one type of PBR workflow and chosen to neglect the other kind, import/export also didn't always have equivalent mapping of features, so you didn't always get what you'd expect, especially with import/export (load/save) of glTF files were not always retaining data even if Blender was the only tool involved I think..

thanks for explanation

[CochainComplex]

storage -> ram -> cpu(decode) -> ram(raw) ->cpu(encode) -> vram...

Needless to say, not efficient

Yes for the first run ...

first run:
storage(classic) -> ram -> cpu(decode) -> ram(raw) -> cpu(encode2ktx) -> vram +storage(ktx) -> used

subsequent runs:
storage(ktx) -> vram -> used

to mitigate step one you could preprocess them and distribute them as it is done with shader caches. But if I have understood dragorth correctly this implementation has to be done for each dxvk mapped engine individualy.
So no universal path for all dx games possible and therefore not able to be build in dxvk or vkd3d.

needles to say that before it has to be checked if a ktx copy is already processed and reprocessing can be skipped.

[Quackdoc]

Yes for the first run ...

first run:
storage(classic) -> ram -> cpu(decode) -> ram(raw) -> cpu(encode2ktx) -> vram +storage(ktx) -> used

subsequent runs:
storage(ktx) -> vram -> used

to mitigate step one you could preprocess them and distribute them as it is done with shader caches. But if I have understood dragorth correctly this implementation has to be done for each dxvk mapped engine individualy.
So no universal path for all dx games possible and therefore not able to be build in dxvk or vkd3d.

needles to say that before it has to be checked if a ktx copy is already processed and reprocessing can be skipped.

the biggest issue with that, is you need a layer that tells the application to use KTX instead of traditional textures, so you are no longer just simply transferring calls, this introduces chances for bugginess pair that with the need to cache (in a sense Duplicate) textures.

Really the only way I see this where benefits outweigh detriments, is when the game itself supports KTX.