Vulkan Video Arrives For New Industry-Standard Video Encode/Decode

[TheLexMachine]

If the Flowmaster Fuck-Knuckle Four Thousand GPU wasn't good at decoding video before, this really won't change things any. This just gives you a one-stop path of entry for software applications to simply access encode/decode functions and it's up to the GPU manufacturer to handle everything in their drivers, from there, using their own hardware/software.

[Quackdoc]

3 pages of comments and no one has posted the XKCD Standards comic? you nerds are slipping!
https://xkcd.com/927/

because it's not applicable here, this is just an easy dirty way to get hardware accelerated video decode/encode in an already existing Vulkan app, or by implementing vulkan, I don't very many people will go out of their way to implement Vulkan just for this.

however Vulkan is already the pretty much go to for cross-platform GL and compute. considering it works on just about everything. Linux, Windows, help even Android and BSD.

This is such a godsend for making cross-platform video accelerated apps. as you will no longer need to implement DXVA, VAAPI, and OpenMax. that also means you do not need to debug them. in the future it should be a simple few lines of code to get consistent cross-platform video acceleration.

it's not just video applications that benefit from this either. games that need pre-rendered cutscenes. This is especially useful on low end devices, web browsers especially those with Vulkan rendering. and more.

I doubt this will replace VAAPI. dxva, or openmax. it probably won't do a good job competing for apps that need them, But if you just need a simple video acceleration, Vulkan will be it... again if it works

[Quackdoc]

How to know which GPus are involved by this innovation?

Im not familiar with how GPU's are wired up, so I cannot comment on the specifics of that side, but any gpu that;
A) Has a vulkan driver
B) supports VAAPI h264 (and in the future Hevc, VP9, and Av1, and likely more going forward)

Should have hardware support for it available to my understanding. now software support I have no idea.

[oiaohm]

Do not blame poor coding on VAAPI and openmax it is not VAAPI's job to implement software decoding. it is the application developers. Vulkan-Video already had a potential user base. I highly doubt it will end up as a lemon.

Its not dxgi but close "DirectX Video Acceleration (DXVA)" you were after this. Here is the big thing DXVA does have software decoding fallback so makes applications developers life simpler.

The weakness of VAAPI and openmax is lack of fall back. This leads to firefox and other thing having to maintain more code paths.

Next VAAPI and openmax have another weakness. DXVA is not codec limited. DXVA exposes what functionally the video accelerator has so if it support different bits of functionality newer software codecs that use DXVA can pick up accelerated functions.

https://xkcd.com/927/

is a good example of what is going on here. I see the vulkan standard video encode/decode as another lemon if it done the same again. Currently its not exposing the proper functionality of the video decoder/encoder. Majority of PC video decoders/encoders are programmable because that what Microsoft DXVA expects.

The reality is the new standard with current design is not going to cover everyone use case properly so we are going to have another competing useless standard.

Ideal world would be vulkan video enccode/decode done properly with a vaapi and openmax overwrapper to it so the old standards can be deprecated.

Yes vaapi not being a software solution means programs like firefox were needing to maintain two code paths then they decide to maintain the 1 code path that cover all use cases. Guess what the one code path that covers all use cases is the software decode/encode. Hardware encode/decode without a software fallback will end up under used so the Hardware encode and decode end up for a lot of uses being tits on bull doing nothing for them.

What is the point of writing a new standard for hardware decoding of videos without a software fall back this going to be the same result where large number applications just keep on using software decode because that is the solution that works every time. We might as well stick with VAAPI and other broken crap in that case because its not going to get used.

[Quackdoc]

Its not dxgi but close "DirectX Video Acceleration (DXVA)" you were after this. Here is the big thing DXVA does have software decoding fallback so makes applications developers life simpler.

DXVA is part of a complicated beast, DXVA doesn't itself provide software fallback. DXVA is part of the MFT process which decides whether or not it can use DXVA, or fallback to a software decode

The weakness of VAAPI and openmax is lack of fall back. This leads to firefox and other thing having to maintain more code paths.

Next VAAPI and openmax have another weakness. DXVA is not codec limited. DXVA exposes what functionally the video accelerator has so if it support different bits of functionality newer software codecs that use DXVA can pick up accelerated functions.

It sounds like what you want is an abstraction layer of sorts. Im not sure if something similar exists on linux, maybe a simplified gstreamer? But still not in the realm of VAAPI's or vulkans responsibility.

https://xkcd.com/927/

is a good example of what is going on here. I see the vulkan standard video encode/decode as another lemon if it done the same again. Currently its not exposing the proper functionality of the video decoder/encoder. Majority of PC video decoders/encoders are programmable because that what Microsoft DXVA expects.

Its clearly not what is happening, more verbose below, but vulkan is a true cross-platform api for talking with GPUs, and that is all this is, a cross-platform API for talking with GPU video engines. that alone is worth merit. considering Openmax the only real alternative, is simply put, a massive mess...

The reality is the new standard with current design is not going to cover everyone use case properly so we are going to have another competing useless standard.

It doesn't need to cover everyones use case, it needs to fill a very much existent gap. and if it preforms well, it will do it well.

Ideal world would be vulkan video enccode/decode done properly with a vaapi and openmax overwrapper to it so the old standards can be deprecated.

Yes vaapi not being a software solution means programs like firefox were needing to maintain two code paths then they decide to maintain the 1 code path that cover all use cases. Guess what the one code path that covers all use cases is the software decode/encode. Hardware encode/decode without a software fallback will end up under used so the Hardware encode and decode end up for a lot of uses being tits on bull doing nothing for them.

IMO it's better that way anyway, as that allows them to use what software decoders they want to. and what versions of them they want to. but I understand the want of one simple code path, but again, that is not the responsibility of a hardware decoder. a simple layer that will automatically choose the best decoder/encoder whether it be software or hardware would indeed be nice to have on linux.

What is the point of writing a new standard for hardware decoding of videos without a software fall back this going to be the same result where large number applications just keep on using software decode because that is the solution that works every time. We might as well stick with VAAPI and other broken crap in that case because its not going to get used.

I doubt vulkan video will be as powerful, maybe someday in the future, but I doubt vulkan video will implement older codecs like vp8, Vulkan video is truly a cross-platform
singular implementation for video accel, on apps that ALREADY or will be migrating to vulkan. or for an abstraction layer. Android, Linux, and Windows apps that need to be crossplatform that need video-accel. Things like video games that have pre-rendered cutscenes, Webbrowsers that have vulkan renderers, Vulkan compute applications etc.

I do agree, some sort of layer to sit in-between would be nice though. but I doubt it would be cross platform. Which in the end is the goal here, a singular crossplatform solution for hardware encoding.

[oiaohm]

I do agree, some sort of layer to sit in-between would be nice though. but I doubt it would be cross platform. Which in the end is the goal here, a singular crossplatform solution for hardware encoding.

There is a problem here,

https://renderdoc.org/vulkan-layer-guide.html

Vulkan by base design has layers. Software version of Vulkan video with vulkan layers would not have to be cross platform binary. But could be interfaced in by the Vulkan layers system. Yes the way it interfaces in could be perfectly cross platform. You missed being vulkan there is already a layer in the middle its just exploit it.

Hardware encoding the practical reality here is it does not always work right you do get broken hardware. Also you have the problem that some software developer will be needing to test application against the API when they don't have a GPU that supports it yet. Another usage is continuous integration testing servers.

There are a lot of reasons why you want a software implementation. We have software implementation of opengl and vulkan they are not the fastest but there still a lot of test that can be done without a GPU because we have them.

How do you test applications using VAAPI or a openmax when you don't have access to gpu. You can test your software rendering path without a GPU.

[M@yeulC]

On my 5600xt, vaapi encode quality for h264 really sucks, compared to amf.
I manly use quality based encoding and vaapi produces 25% bigger files to achieve the same quality of amf.
Luckilly one can keep installed radv and amf togheter and switch on the fly.

That's possible, but I wouldn't make it a case of AMF vs VAAPI. It's probably more dependent on the implementation than the API that exposes it.

Come to think of it, I'm a bit surprised VAAPI doesn't have an AMF/nvenc/QuickSync/whatever backend, to present a unified video decode API.

Anyway, the big thing here is that this API is supposed to be cross-platform (except on Apple devices I guess), and proposed by a neutral, platform-independent third party.

[oiaohm]

Anyway, the big thing here is that this API is supposed to be cross-platform (except on Apple devices I guess), and proposed by a neutral, platform-independent third party.

If it does come with a software fall back option Vulkan on metal could fairly quickly support the extension on Apple. Idea case of cross platform is having a software fallback if you don't have the accelerated hardware hooked up on a platform yet it can somewhat work.

Please note that would not have to be a pure software fall back it could be fall back to GPU compute units so some GPU acceleration just not as optimal as a video decoder/encoder circuit..

[Azrael5]

Im not familiar with how GPU's are wired up, so I cannot comment on the specifics of that side, but any gpu that;
A) Has a vulkan driver
B) supports VAAPI h264 (and in the future Hevc, VP9, and Av1, and likely more going forward)

Should have hardware support for it available to my understanding. now software support I have no idea.

Any GPus supports a specific Vulkan release highilights by its own number version... but often the GPus support different values in confront of what is declared.