Vulkan Video Arrives For New Industry-Standard Video Encode/Decode

For years we have been eager to learn more about the long mentioned Vulkan Video API, which was supposed to come in H1-2020, but now has finally arrived with today's v1.2.175 update in provisional form. The new Vulkan Video extensions allow for GPU-accelerated video encode/decode. The initial public work is treated as a provisional specification and with limited codec coverage but will be expanded upon in time.

Vulkan Video allows for GPU-accelerated encode/decode and integration with the Vulkan API over scheduling, synchronization, and other Vulkan capabilities. The main new extensions for Vulkan Video are VK_KHR_video_queue, VK_KHR_video_decode_queue, and VK_KHR_video_encode_queue. Vulkan Video is designed to be extensible in terms of codec coverage while initially the new extensions there are VK_EXT_video_encode_h264, VK_EXT_video_decode_h264, and VK_EXT_video_decode_h265).

Notably absent right now is VP9 and AV1 open-source / royalty free video codecs... Fortunately, The Khronos Group says they will be coming in a future Vulkan update. Their initial focus was on ensuring the core architecture components are ready and once that is all settled will be broadening their codec support. Too bad they didn't make it for today's debut but hopefully won't be too long before seeing the expanded coverage. A Vulkan H.265 encode extension is also still in development.

The Khronos Group has published a blog post this morning on Khronos.org outlining the Vulkan Video functionality in much greater detail. Check that out to learn about Vulkan for video encode/decode/transcoding.

First out of the gate with Vulkan driver coverage for the video extensions are expected to be NVIDIA. NVIDIA should be releasing a new beta Vulkan driver this morning that supports these Vulkan video extensions. I haven't yet heard from AMD or Intel on when they plan to introduce Vulkan Video support with their (open-source) Linux Vulkan drivers. At least in the case of Intel hopefully it will come soon for ANV given their resources and already having their advanced open-source multimedia stack. On the AMD side for open-source they have been relying on Gallium3D-based video acceleration so far and will be interesting to see if/when they adapt AMDVLK to Vulkan Video and whether the independent RADV driver manages to tack on support for these video extensions too.

NVIDIA has published a Vulkan video decoder open-source example for those interested. The Khronos Group and their partners are also working on conformance test suite coverage, more code samples, and other additions around Vulkan Video.

Hopefully once Vulkan Video 1.0 specifications are finalized we will see more multimedia software make use of this video encode/decode API compared to the fragmentation seen now by different video interfaces especially with different supported APIs between Windows and Linux. With Vulkan Video 1.0 and when the expanded codec support is here we stand good chances of this hopefully becoming the dominant interface for GPU-accelerated video encode/decode... Well, hopefully it takes off better than OpenMAX.

The provisional Vulkan Video extensions are the most significant work found in today's spec update but there are also other new extensions in Vulkan 1.2.175.