WebM, Theora Acceleration Talked About For Gallium3D

Earlier this week I mentioned a student developer looking to partake in Google's Summer of Code was interested in creating an H.264 state tracker for Gallium3D whereby any graphics card with a Gallium3D driver could have H.264 video decoding support using VA-API / VDPAU and accelerating the operations in shaders on the GPU, where in theory at least it would be universally supported across all drivers on this architecture. It's still looking hopeful that this will be hacked on this summer, but a few interesting points have been expressed.

Mesa developers haven't shot down this GSoC proposal like they effectively did with the OpenGL 4.1 state tracker proposal two weeks back over that being too ambitious for a single student to do in one summer. In fact, the Mesa developers seem fairly interested in such a video acceleration state tracker happening. As mentioned in previous articles, the only work done in this area on Gallium3D already has been for XvMC and MPEG with mixed success.

Christian König, the developer that has been working on XvMC for the R600g driver is excited about the possibility of another developer now working on "pipe-video", the branch of Mesa where this video acceleration for Gallium3D is being developed.

Other points that have been talked about for this work in recent days include:

- Shader-based video acceleration is still not ideal when modern graphics cards have dedicated units for encoding/decoding video. Unfortunately, these implementations like AMD UVD2 and NVIDIA PureVideo are not publicly documented and would require hardware-specific work.
- One opportunity instead of -- or to complement -- the H.264 video acceleration would be to target the WebM / VP8 or Theora formats. The video decoding hardware that's out there now doesn't support these open formats so it's all CPU bound for now, so working on WebM video acceleration for Gallium3D would be an original creation. Much of the work needed to target H.264, WebM / VP8, and Theora would be code that could be shared across formats.
- In terms of whether to target the VA-API or VDPAU API for such a state tracker, the consensus seems to be on using NVIDIA's Video Decode and Presentation API for Unix. It also appears that it would be easier to extend VDPAU than VA-API for handling new formats like Google's WebM.
- It was also mentioned there is someone working on WebM support over OpenCL (see the GitHub), but that's currently slower than a CPU-based solution and the open-source drivers currently don't have OpenCL support.

Let's hope this Google Summer of Code proposal is accepted and that it manages to materialize this summer into something useful.