The Wayland Situation: Facts About X vs. Wayland

[farnz]

For the ones requesting link to prove that Wayland uses Delta-frames, it's in here somewhere, I can't be bothered finding the specific time right now (I'm 99% sure this was the vid, either that, or this one with Daniel Stone, which in any case is rather relevant to the topic).

I can't remember where, but I'm rather certain that someone mentioned binary drivers not being able to use KMS due to licensing, I don't recall anyone answering this issue. Among the bits n' bobs I've read these past days, I read somewhere that Nvidia had a workaround for this issue merged with the 3.9 kernel, so it shouldn't be that much of an issue, though I bet it'd all work better with a proper open source driver.

Now I've read quite a bit and watched vids n' stuff, and just when I think I understand everything, I read or watch something that messes with everything I think I know. So, to get the basics right:
- In the video with Daniel, it's stated that nothing in Wayland requires GL. Is EGL/GLES not considered GL-related?
- Again, the vid mentions GLES for rendering, the wayland FAQ mentions GLES/EGL untill a potential WaylandGL library vomes into existance. Considering reported Nvidias Unified EGL driver, will that have any effect on the protocol being used?
- From the info I have found so far, it seems that OpenGL communicates directly with the hardware with both X and Wayland (I'm not at all sure, everything I read tends to give a different impression from what I think I've learned). Does this mean that fullscreen OpenGL applications will still suffer from screentearing?

I apologize if any of this has been answered in either of the vids, or in this thread, but my memory is currently way past its limit.

Just covering your OpenGL points:

• Nothing in Wayland needs OpenGL, EGL, or OpenGL ES. The only reason they keep coming up is that the Wayland devs don't intend to have a native Wayland Accelerated Rendering Protocol, and suggest that you use OpenGL for hardware accelerated rendering. Nothing stops you using Wayland as a dumb framebuffer system, though, with whatever rendering protocol you want implemented in software, and XWayland uses the existing DDXes to accelerate the X rendering protocols.
• Screentearing in X11 is a consequence of X11's buffer model (one buffer per window, composited onto the screen by hardware automatically) interacting badly with X compositing. Wayland doesn't have the same model, and therefore has multiple options to avoid this; I believe Weston implements scanout from application buffer (let a full screen application have direct control of the hardware, and don't bother compositing it).

[mannerov]

Amd released last month a catalyst driver with the GLX_EXT_buffer_age extension.

Does that mean that wayland will run with it ? (I've read somewhere it supports Opengl ES 2 and EGL 1.4)

[Serafean]

Amd released last month a catalyst driver with the GLX_EXT_buffer_age extension.

Does that mean that wayland will run with it ? (I've read somewhere it supports Opengl ES 2 and EGL 1.4)

Steps in the right direction, but AFAIK KMS (or some similar mechanism) needs to be supported (modesetting is deferred to the kernel driver now) too.

[rabcor]

Excellent, i look forward to be able to use wayland.

I've had some pretty rough experiences with X where i've had to battle it just to make certain things work properly, i've had some mean crashes, hangs and whatnot and i just quite haven't got a good impression of X11 at all. I don't fancy the idea of Mir either since i've never been a big fan of ubuntu, unity and canonical. I don't like ubuntu as an OS in almost any way, i hate where it's headed and canonical strikes me as a wannabe apple company. I already hate apple more than any other company out there and all of their recent products.

Personal opinions...

But i think Wayland looks quite interesting, but as the newbie i am when it comes to linux i'm not exactly about to go help in its development nor be a worthwhile tester. So i've largely just taken a break from linux completely after i finally manged to set it up just like i wanted it to be. Even if i got that far it just wasn't enough for me, and it seems theres a lot going on in the graphics server area of linux so i intend to wait it out until people are saying wayland, mir or something else is reliable and good window managers have been ported to it, i intend to return to linux once i can get rid of X and hopefully by then linux will be a bigger OS with better driver support from companies (creative, logitech, nvidia, amd, etc) these mainstream companies revolve around the gaming world, valve (creator of steam) is making an ambitious attempt to expand the gaming world onto linux to make linux a viable gaming platform, if that will be successful (which is where it seems to be headed, many great games have already been ported) linux is bound to get better attention and care from the hardware giants that don't even bother making proper drivers for it atm.

I sortof liked where this guy was going, and i sortof hope that the linux community will learn the things he's talking about, since its been a while since i saw it i don't remember what he talked about, but one of the biggest deals that he mentioned imo is how X is old and how packaging sucks (so many god damn stupid extensions, why can't we just use tar.gz or fucking .zip or .rar like any other good children. Why .deb or .rpm!!! it's just stupid.).

[nachokb]

it totally makes sense, but all the drawing is totally toolkit-specific, hence our suggestion to do this kind of thing at the toolkit level instead.

does wayland / weston require that the surface buffer is specifically an image? what if it's something like a "document" (or command dump) instead... say, a buffer has an "image/svg" MIME content-type; the toolkit updates that svg, and weston delegates the drawing itself to some SvgRenderer (TM) (off-process).

The idea being, there could be many different renderers (so if a renderer gets old, it's only loaded if an app requires it). Thus, GTK+ apps could use a CairoRenderer (it would need some kind of document format, or at the very least marshalling cairo calls, perhaps the Broadway work could be useful here), Qt apps could use a QPainterRenderer, something else could use a PDFRenderer (wine using a win32 renderer :O).

That way, the buffer would contain enough information for the remote system to render the image while minimizing bandwidth requirements...

This could be implemented as some kind of RenderManager which does not need to be integrated into the compositor. But wayland/weston would need to support adding hooks for the renderer.

Also, how does wayland manage the case in which different backends could needs different kind of buffers (e.g. EGL-backend needs a GL buffer, but Pixman backend requires something else?).

Lastly, GTK+ already has some remoting work done with its Broadway backend...

tl;dr add support for *external*, deprecatable, registerable implementations of drawing APIs, and each buffer having a MIME content-type

-- nachokb

[farnz]

does wayland / weston require that the surface buffer is specifically an image? what if it's something like a "document" (or command dump) instead... say, a buffer has an "image/svg" MIME content-type; the toolkit updates that svg, and weston delegates the drawing itself to some SvgRenderer (TM) (off-process).

The idea being, there could be many different renderers (so if a renderer gets old, it's only loaded if an app requires it). Thus, GTK+ apps could use a CairoRenderer (it would need some kind of document format, or at the very least marshalling cairo calls, perhaps the Broadway work could be useful here), Qt apps could use a QPainterRenderer, something else could use a PDFRenderer (wine using a win32 renderer :O).

That way, the buffer would contain enough information for the remote system to render the image while minimizing bandwidth requirements...

This could be implemented as some kind of RenderManager which does not need to be integrated into the compositor. But wayland/weston would need to support adding hooks for the renderer.

Also, how does wayland manage the case in which different backends could needs different kind of buffers (e.g. EGL-backend needs a GL buffer, but Pixman backend requires something else?).

Lastly, GTK+ already has some remoting work done with its Broadway backend...

tl;dr add support for *external*, deprecatable, registerable implementations of drawing APIs, and each buffer having a MIME content-type

-- nachokb

Core Wayland requires that a wl_buffer (the buffers that clients hand to the compositor) are pixmaps in a format the compositor advertises that it can support.

However, Wayland is (at heart) an efficient IPC protocol. There is nothing preventing you defining (say) an svg_render_buffer, and providing a library that permits a client to always use svg_render_buffers, even when the compositor does not support said interface (when the compositor supports it, pass the client ops through to the compositor; when it doesn't, draw into wl_buffers and pass those to the compositor). If there are genuine efficiency gains from doing that, go for it!

[nachokb]

the compositor advertises that it can support.

that's interesting... so can one running instance of a compositor support windows with different buffers of different types? can a compositor advertise multiple types?

so right now a client (e.g. a toolkit) renders to an EGL buffer or a Pixman buffer depending on what is advertised by the compositor right?

There is nothing preventing you defining (say) an svg_render_buffer, and providing a library that permits a client to always use svg_render_buffers, even when the compositor does not support said interface (when the compositor supports it, pass the client ops through to the compositor; when it doesn't, draw into wl_buffers and pass those to the compositor). If there are genuine efficiency gains from doing that, go for it!

that sounds good... I can imagine this capability will be used in the future...

[nachokb]

There is nothing preventing you defining (say) an svg_render_buffer, and providing a library that permits a client to always use svg_render_buffers, even when the compositor does not support said interface (when the compositor supports it, pass the client ops through to the compositor; when it doesn't, draw into wl_buffers and pass those to the compositor). If there are genuine efficiency gains from doing that, go for it!

So if a compositor advertises, say, "svg_render_buffers", it's the compositor who gets the responsibility to do the actual rendering right? This could get extracted (that's my idea of a RenderManager) to be used by many compositors... but the important part is that the payload data between the client and the compositor is lightweight enough (and semantically rich enough*) to enable efficient remoting.

* what curuga said "sending text as text"... it's not just about bandwidth, but compressing text with a lossy DCT codec should be a sin :P... although I'm completely opposed to adding a drawing API to the wayland protocol

-- nachokb

[farnz]

that's interesting... so can one running instance of a compositor support windows with different buffers of different types? can a compositor advertise multiple types?

so right now a client (e.g. a toolkit) renders to an EGL buffer or a Pixman buffer depending on what is advertised by the compositor right?



that sounds good... I can imagine this capability will be used in the future...

A compositor can indeed advertise different buffer types; note that for pixel buffers, as used in core Wayland, there is no difference between EGL and Pixman - both provide you with a lump of pixels laid out in memory. However, a compositor could (for example) advertise support for 32-bit ARGB, RGB, AYUV and YUV buffers in various pixel layouts, permitting a movie player to use a hardware decoder to decode into a buffer the compositor can use directly, as well as permitting Pixman and EGL based renderers to draw directly to buffers the compositor can handle. You can also easily imagine a compositor advertising support for high bit depth buffers for professional screens (e.g. 16-bits per channel ARGB).

[farnz]

So if a compositor advertises, say, "svg_render_buffers", it's the compositor who gets the responsibility to do the actual rendering right? This could get extracted (that's my idea of a RenderManager) to be used by many compositors... but the important part is that the payload data between the client and the compositor is lightweight enough (and semantically rich enough*) to enable efficient remoting.

* what curuga said "sending text as text"... it's not just about bandwidth, but compressing text with a lossy DCT codec should be a sin :P... although I'm completely opposed to adding a drawing API to the wayland protocol

-- nachokb

Exactly that. If you do it right, you can implement a proxy Wayland server that just passes things through unmodified if your upstream compositor supports the "RenderManager" extension internally, and that renders to a wl_buffer (using Cairo, or EGL/OpenVG, or EGL/OpenGL ES 3, or magic fairies, or whatever) if it doesn't.

Then, I only need to implement core Wayland in my compositor to work with apps that need your RenderManager; the app uses your proxy to convert svg_render_buffers to wl_buffers that I can use. If I see advantages to rendering svg_render_buffers myself, I can do that, too).

Note that one of many reasons to not provide a rendering protocol in Wayland is to avoid the pain X11 can get into where a client asks it to spend lots of time rendering something - X appears to take lots of CPU, while the client takes very little, because X is doing all the rendering work on behalf of the client. A separate RenderManager proxy helps with this, especially if you layer it as client->per-client RenderManager->compositor, as it's a specific client's RenderManager that starts to eat CPU, and you can thus identify the culprit.

It's also true here that (because of the way Wayland IPC is designed), you can experiment with extensions like a RenderManager without having to make the compositor aware that such an extension even exists- just write a proxy that translates your extension back into core Wayland protocol (e.g. by doing the rendering work needed to transform a svg_render_buffer to a standard wl_buffer). If the extension turns out to be incredible, many compositors will implement it internally, with no need for the RenderManager proxy. If it's useful, people will run it where it's needed, and eventually, it'll be an extension that people just assume is available.