Experimental VA-API Implemented Over NVIDIA's NVDEC - Allows Firefox Video Acceleration

For software like Mozilla Firefox that relies on the cross-vendor Video Acceleration API (VA-API) for hardware GPU-based video decoding and doesn't support NVIDIA's proprietary NVDEC interface for video decoding, there is an in-development VA-API implementation that works atop NVIDIA NVDEC.

An independent developer has been creating the open-source "nvidia-vaapi-driver" that is a VA-API implementation making use of NVDEC for enjoying NVIDIA GPU video acceleration with software only targeting VA-API. VA-API started out years ago as Intel's Linux video acceleration API but with time there has been the Mesa Gallium3D state tracker implementing the API and it's been wired up for enabling AMD Radeon GPU acceleration support and is also in use by other drivers. At least until if/when Vulkan Video takes hold, VA-API is pretty much the dominant Linux GPU video acceleration API while NVIDIA has been focused on their own NVENC/NVDEC interface and previously their VDPAU (Video Decode and Presentation API for Unix) that was fairly well received and adopted until this NVENV/NVDEC transition. Years ago there was the libva-vdpau-driver project for mapping VA-API to VDPAU but this new open-source effort dubbed "nvidia-vaapi-driver" is about going through the modern NVDEC as part of the NVIDIA Video Codec SDK.

This nvidia-vaapi-driver has been tested successfully across H.264, HEVC, VP8, VP9, MPEG-2, and VC-1. Known to not be working is JPEG support as the APIs are too different, MPEG-4 as there is also not a clean means of mapping MPEG-4 VA-API onto NVDEC, and no AV1 support yet for the developer involved not having the necessary GPU hardware.

This NVIDIA VA-API implementation has been successfully tested with Firefox 96 Beta when reducing the security restrictions of the browser while Firefox 97 should allow for better support. This VA-API implementation also has been tested with the MPV video player albeit not the most practical since that open-source multimedia player has native NVDEC support.

More details on this promising NVIDIA VA-API to NVDEC implementation via the NVIDIA Forums and the code for this cross-API translation effort is hosted on GitHub.