Tweet
Originally posted by bug77
View Post
-
Like I explained, afaik all data that goes through the web is split into chunks so if something gets lost, you only have to resend those chunks instead of the whole file. So brotli combined with http won't need to compress big files, only lots of chunks of bigger files. And as the article states, it's not GPU only. But if a GPU could free up many CPUs, cloud and CDN providers might switch to just using GPU acceleration. And maybe AMD is even working on accelerating more stuff through GPUs so it might make more sense in the future. After all, they probably wouldn't mind if they could sell their GPUs in the server market too. -
Ok, I'm at an even bigger loss now. After reading their description on GitHub and reading their article https://gpuopen.com/brotli-g-sdk-announce/ it seems the GPU implementation isn't meant for compressing (at least not mainly), but for decompression. They want Windows users to use a DirectX Shader for parallelized decompression. Sure, when loading why web page using brotli will result in many brotli compressed packages that need to be decompressed in order to display the page, but why build it for Windows then? Those systems usually have enough horse power to do it the old fashion way. I would rather see an implementation for Android and iOS be more helpful. After all, they are on a much stricter power budget. So this makes just no sense at all. And who in the Windows world is supposed to implement that? That would need to be browsers, but I don't see any implementing it. Not when it doesn't work in all platforms.Originally posted by bug77 View PostComment
-
Nah, they just store the file compressed in the first place, so they don´t have to compress on-the-flyOriginally posted by Artim View Post
B.t.w. this is a completly "new" format from a practical point of view as:
This essentially means, that one has to introduce a new format in the HTTP header like Content-Encoding: br-gOne thing for developers to note is that assets that have already been compressed with Brotli cannot be decompressed with Brotli-G decompressor implementations. To take advantage of Brotli-G, assets must be recompressed using a Brotli-G compatible compressor (also supplied in the SDK).
as you can´t pass a brotli compresse bitstream to the Brotli-G decoder.. So essentially the current ecosystem is almost 0.Last edited by Spacefish; 22 November 2022, 06:36 PM.Comment
-
To me it looks like this is targeted at Windows' DirectStorage feature meant for faster decompression of assets on GPUs, hence the HLSL usage.
Right now, that uses "GDeflate" developed by nVidia.
This is AMD's answer to that.
Care to chime in, bridgman?Comment
-
brotli is just a compression algo, it can be used on whatever you really want it to. there is a brotli tool that can be used to make brotli compressed files. if you have run across .tar.br files, those are brotli files (pretty rare though)Originally posted by bug77 View Post
this is designed for various assets, but this should work for anything that needs to be utilized on the gpu, we need to compress resouces going to gpus, because the stuff we punt onto the gpus is far out pacing the amount of vram that increases as time goes on.
compression is necessary for this, but since you are under the assumption that decompression needs to happen real time, and that this will also that this will be used in gpu intense scenarios, so you want the performance impact to be as little as possible, brotli is easily the best choice for this.
this could be used for direct storage, but its not necessarily only useful for that, this could be useful for compute, this could be useful to trim that 8.5gb of vram usage to 8. but more importantly, we don't really have a good way to crossplatform supported, losslessly compress assets, as far as I know, I'm not sure if we even have any good lossless texture formats for gpu.Comment
-
The question merely is if you need to indicate the format or if the decompressors are able to switch on demand between brotli and brotli-g when the latter fails.Originally posted by Spacefish View PostNah, they just store the file compressed in the first place, so they don´t have to compress on-the-fly
B.t.w. this is a completly "new" format from a practical point of view as:
This essentially means, that one has to introduce a new format in the HTTP header like Content-Encoding: br-g
as you can´t pass a brotli compresse bitstream to the Brotli-G decoder.. So essentially the current ecosystem is almost 0.Comment
-
Game assets. Textures, and anything else you are sending to vram would still benefit.Originally posted by bug77 View Post
Maybe there are gpgpu scenarios where the dataset is highly crompressible (where saving pcie bandwidth savings are enough to offset the additional compute overhead).Last edited by brucethemoose; 23 November 2022, 01:37 PM.Comment
-
Michael "just not AMD Radeon graphics products" you probably meant "not just AMD Radeon graphics products". Has a different meaning
Comment
-
Nvidia has it it's own library for GPU accelerated compression for deflate, LZ4, zstd, etc.. The latest versions are not open source thought.
Comment
-
while gpu accelerated lz4 encode and decode could be highly interesting, unless it's in a cross platform ecosystems entirely useless to me sadlyOriginally posted by abu_shawarib View PostComment
Comment