Microsoft Lands VA-API To Direct3D 12 H.264 Video Encode/Decode In Mesa

Microsoft has made a lot of interesting developments and maneuvers over the past number of months for leveraging open-source Mesa for use by Windows Subsystem for Linux (WSL) and supporting various Khronos APIs atop Direct3D 12 for use when native drivers are lacking on Windows. This work so far has been focused on OpenGL, OpenCL, and Vulkan but Microsoft has now even implemented Direct3D 12 video API support within Mesa and leverages the VA-API state tracker support within Mesa.

Simply put, this latest effort allows WSL GUI applications like media players to enjoy hardware video acceleration. Within the code merged to mainline for Mesa 22.2, D3D12 video APIs have been implemented and relevant VA-API entry points for H.264 encode/decode. Thus Linux video players / multimedia software supporting the Video Acceleration API (VA-API) will "magically" be able to enjoy Direct3D 12 video acceleration when running within WSL(2).

This is quite a win for those wanting more robust GUI software support within the Windows Subsystem for Linux confines and further cements VA-API as the dominant Linux video API. While there is an experimental, independent VA-API implementation over NVIDIA's NVDEC, VA-API doesn't have support across the board for all GPUs/drivers, so this support atop Direct3D 12 video technically makes it more robust on Windows than Linux itself. Granted, at least for the moment it's limited to the H.264 codec but this current code is designed to be extensible to support more video codecs moving forward.

Microsoft engineers worked on this D3D12 video support and Gallium3D video acceleration front-end changes themselves and successfully tested VA-API usage with FFmpeg encode/decode/transcode and with the MPlayer-forked MPV and vaah264 under WSL with X11 windowing support enabled. It's also possible they may support NVIDIA Video Decode and Presentation API for Unix (VDPAU) as another supported implementation atop D3D12 video with some Gallium3D VDPAU changes also staged as part of this merge, but the initial focus appears to be on the VA-API interface.

This effort has been ongoing for a number of months with back in November I wrote about Microsoft eyeing Direct3D 12 video acceleration for Mesa. Now the initial working code is finally ready and mainlined.

More technical details for those interested via this merge request that a short time ago landed in Mesa 22.2. This is beneficial for WSL as outlined but for native, bare metal Linux users this 11.3k lines of new code in Mesa will likely be of little use.