Generic Buffer Formats, YUV For Wayland

A few interesting patch-sets were published today concerning Wayland.

Gwenole Beauchesne, the developer who formerly worked at Splitted Desktop Systems to create various VA-API drivers including the NVIDIA VDPAU back-end to VA-API and VA-API over AMD XvBA, has been working at Intel since last year. He hasn't had many user-facing patches hit the public lists in the past few months, but that changed this morning.

It appears that Gwenole Beauchesne may be in the midst of some interesting developments today. He kicked off the morning by publishing the patches to add generic buffer format support to Wayland, in particular introduced YUV format support. He then published separate patches that implement this format support for Weston, then he went on to publish some Mesa patches.

Here's his set of five patches concerning generic buffer format support for Wayland.

This series adds generic buffer formats (at the wl_buffer level) and YUV formats in particular. I believe this is generally useful because even for SHM buffers, we could also represent YUV contents in there. e.g. gstreamer sink.

Patch 1 adds the generic buffer types. Those are the most commonly used formats only. Besides, I also dropped YUV/YVU variants as YVU is YUV with different layout.

Talking of which, Patch 2 adds some layout infrastructure. If you already have an allocated SHM buffer around, it's theoritically possible to attach a particular layout to it. However, I didn't want to expose that because if the user allocates a YUV buffer, it will tend to have the same layout for the lifetime of the object. And, if the layout needs to be changed, allocating a new buffer should not be that expensive.

Since I added the layout right into the wl_buffer base object, Patch 3 bumps the version of its interface.

Patch 4 fills in layout information for SHM buffers.

Patch 5 adds some helpers to extract a sensible layout even if the buffer was implemented for a former interface.

This support can be useful for video handling with Wayland.

To Weston were then a set of four patches (on the mailing list) to add in support for YUV buffers with Weston. Gwenole's already tested this YUV buffer support in Wayland's Weston with VA-API on Intel hardware. This isn't the first time that VA-API on Wayland has come to light, but back in February I reported on VA-API for Wayland was being worked on by Intel.

Here is a first round of patches for supporting YUV buffers in Weston. It theoritically possible to support planar YUV from SHM buffers but I have only tested with VA-API on GenX (NV12, YUV 4:2:0 and 4:2:2).

Some changes are needed in Mesa, but I wanted to get your opinion on this model first. Rationale: a Wayland buffer is actually several individual sub buffers (one for each plane) that can be passed to eglCreateImageKHR(). Some people would argue to use OES_EGL_image_external, but I wanted to make the usage of shaders explicit. Besides, for other projects, there was also a need to map individual planes as separate buffers so that to perform some postprocessing with further shader code (noise reduction, sharpening, etc.).

So, I extended EGL_WAYLAND_BUFFER_WL spec to use a new attribute that specifies that actual plane to map: EGL_WAYLAND_BUFFER_PLANE_WL. The whole magic to calculate stride, offsets et al. is performed in Mesa with the help of Wayland buffer layouts.

For rendering, you just need to create an EGLImage and the corresponding texture for each plane of the YUV buffer. Then, you provide the shader with the current restriction that EXT_texture_rg is required. i.e. single components (e.g. Y) are mapped to GL_R8 [.r] and two components (e.g. UV for NV12) are mapped to GL_RG8 [.r/.g]. I could have sticked to standard GL_LUMINANCE [.x] and GL_LUMINANCE_ALPHA [.x/.w] but those tended to be deprecated in desktop GL IIRC and I wanted a means that could let the user use single glsl source code for rendering (depending on his winsys).

The other idea to hide stride, offsets into Mesa was that wl_drm_create_buffer_with_layout() could also take a future "flags" argument that would denote interleaved frames
(top/bottom fields).

Issues:
- EXT_texture_rg requirement, could be fixed with an extra EGL attribute.
- Use shaders to render from YUV even if some HW could sample from YUV textures directly.

Next on Gwenole's agenda for the day was to then publish the Mesa support for handling the GL_EXT_texture_rg extension within Mesa for OpenGL ES 2.0. As far as hardware drivers go for GL_EXT_texture_rg, he implemented the support inside the Mesa DRI driver. The mailing list message can be found here.

Last but not least, the latest Gwenole Beauchesne patch-set coming out today to cache regions plus an offset within the Mesa DRI driver. The patches under an "RFC" (Request For Comments) flag hit the mesa-dev list. This too is also part of the YUV surface work going on for Wayland.