AMD Releases Brotli-G For GPU-Accelerated Brotli Compression

[Anux]

"good" but lossy. I am unaware of gpu image formats that are lossless and actually used

Because that would be useless, you gain nothing visually from lossless, the files are bigger, the computation is more expensive.

[Quackdoc]

For images there are already good compression technologies but the submission of millions of objects might profit from such a thing.
Smaller chunks are not only good for parallelization but also for latency.

"good" but lossy. I am unaware of gpu image formats that are lossless and actually used

[Anux]

To me it looks like this is targeted at Windows' DirectStorage feature meant for faster decompression of assets on GPUs, hence the HLSL usage.

Sounds plausible, that's how they can get away with being incompatible. Even a specialized dictionary could be used to better target the data stream for GPUs.
For images there are already good compression technologies but the submission of millions of objects might profit from such a thing.
Smaller chunks are not only good for parallelization but also for latency.

[Quackdoc]

Nvidia has it it's own library for GPU accelerated compression for deflate, LZ4, zstd, etc.. The latest versions are not open source thought.
https://github.com/NVIDIA/nvcomp

while gpu accelerated lz4 encode and decode could be highly interesting, unless it's in a cross platform ecosystems entirely useless to me sadly

[abu_shawarib]

Nvidia has it it's own library for GPU accelerated compression for deflate, LZ4, zstd, etc.. The latest versions are not open source thought.

https://github.com/NVIDIA/nvcomp

[Alliancemd]

Michael "just not AMD Radeon graphics products" you probably meant "not just AMD Radeon graphics products". Has a different meaning

[brucethemoose]

What would you keep in VRAM that would benefit from brotli? Brotli is good with small files, it sucks with something as large as a texture. Maybe this is aimed at some sort of compute scenario?

Game assets. Textures, and anything else you are sending to vram would still benefit.

Maybe there are gpgpu scenarios where the dataset is highly crompressible (where saving pcie bandwidth savings are enough to offset the additional compute overhead).

[Artim]

Nah, they just store the file compressed in the first place, so they don´t have to compress on-the-fly

B.t.w. this is a completly "new" format from a practical point of view as:


This essentially means, that one has to introduce a new format in the HTTP header like Content-Encoding: br-g
as you can´t pass a brotli compresse bitstream to the Brotli-G decoder.. So essentially the current ecosystem is almost 0.

The question merely is if you need to indicate the format or if the decompressors are able to switch on demand between brotli and brotli-g when the latter fails.

[Quackdoc]

I thought brotli was designed to be easy on hardware (being able to run on phones and such). What am I missing here? Why does it need to run on GPUs now?

brotli is just a compression algo, it can be used on whatever you really want it to. there is a brotli tool that can be used to make brotli compressed files. if you have run across .tar.br files, those are brotli files (pretty rare though)

this is designed for various assets, but this should work for anything that needs to be utilized on the gpu, we need to compress resouces going to gpus, because the stuff we punt onto the gpus is far out pacing the amount of vram that increases as time goes on.

compression is necessary for this, but since you are under the assumption that decompression needs to happen real time, and that this will also that this will be used in gpu intense scenarios, so you want the performance impact to be as little as possible, brotli is easily the best choice for this.

this could be used for direct storage, but its not necessarily only useful for that, this could be useful for compute, this could be useful to trim that 8.5gb of vram usage to 8. but more importantly, we don't really have a good way to crossplatform supported, losslessly compress assets, as far as I know, I'm not sure if we even have any good lossless texture formats for gpu.

[Linuxxx]

To me it looks like this is targeted at Windows' DirectStorage feature meant for faster decompression of assets on GPUs, hence the HLSL usage.

Right now, that uses "GDeflate" developed by nVidia.
This is AMD's answer to that.

Care to chime in, bridgman?