Broadcom Open-Sources VideoCore IV 3D Graphics Stack

[brad0]

well, I'm going to ignore mir because (a) single-distro solution for a distro that probably doesn't care about supporting non-linux kernels, and (b) I could really care less about mir.

As far as wayland, there is nothing about it that is tied to any particular linux[1] feature, as far as I can tell. What *weston* is more or less tied to (unless you want to create a custom backend) is not doing user mode setting. You need to have a kms driver, or at least an fbdev driver that sets up the mode for userspace.

All of this is completely unrelated to r-pi, which already has it's own custom weston backend. (And, yes, to answer ssvb's point, it is pretty much equivalent to a custom window manager with some private extensions to coordinate between window manager and xserver. But really, that sounds like a pretty horrid option compared to weston + xwayland.)

[1] hmm, ok, according to https://wiki.freebsd.org/Graphics there is also a udev dependency.. although there are only a handful more references to udev in weston src code compared to xserver codebase, so I suspect if xserver can be made to work both with and without udev, then so can weston.

That still doesn't change the fact that for the time being and for the foreseeable future Wayland is still Linux only; udev is a Linux feature. The FreeBSD page you even pointed to makes it pretty clear Wayland will not happen anytime soon.

Sounds like you are reading things into the OP's statement. Or at least I did not draw the same conclusion as you.

Yes, you did not come to the same conclusion.

(btw, you *did* know that linux is a UNIX-like operating system, right?)

Of course, when *I* say UNIX-like I include OS's powered by the Linux kernel but the whole point of the comment was to reference other OS's that are not powered by Linux. That should be common sense in the context it was used.

[pq__]

Can you elaborate on this? What exactly is accelerated? I mean, does it already offer something other than just fancy windows composition (which is optional and better to be disabled altogether on low-end hardware anyway)? Fully accelerated is a rather strong claim.

On Wayland, there is practically nothing else for the server to do on graphics than to composite and present, and you cannot live without compositing if you want to have windows. On that point of view, "fully accelerated" is accurate. Server side rendering simply does not exist.

Accelerating Wayland clients is a whole different issue, and you already found the github pull request to enable EGL/GLESv2 clients.

Enabling accelerated clients is mostly about allowing the server to understand hardware-based graphics buffers. Once you can pass all needed kinds of buffers to the server, and the server can put them on screen or composite with them, you're practically done from the Wayland point of view. As hardware buffers are platform specific, the platform specific Wayland protocol bits are usually hidden in libraries like EGL. But sometimes we just cannot live without platform specific code in the server.

One thing that has been hindering the EGL/GLESv2 enablement work is that the standard EGL API hides too much, and that was practically the only API we had towards the firmware to manage hardware buffers.

[pq__]

For the record, it is well possible to have a Wayland server working without udev, and clients do not need udev at all anyway. Weston's Android backend worked without udev, Android didn't have udev. So if you want a server on a platform that doesn't have udev, it is perfectly possible, since you have some other way to find the devices and information about them.

Wayland itself does not care at all about udev. Udev is only a server implementation detail, and there is no "The Server" on Wayland like the Xorg Xserver is for X11. And even the Xorg Xserver is not the only X11 server implementation in existence.

On any platform we of course want to use the existing infrastructure when it's there. On Linux, udev, Mesa-EGL, and DRM are such. In Weston's case specifically, we just have not had contributions from developers making Weston work on other particular kind of platforms. I believe they would be welcome if they did and actively maintained their work. Btw. Raspberry Pi is not Mesa-EGL nor DRM, yet the support was well received into Weston.

[robclark]

That still doesn't change the fact that for the time being and for the foreseeable future Wayland is still Linux only; udev is a Linux feature. The FreeBSD page you even pointed to makes it pretty clear Wayland will not happen anytime soon.

Well, the fact still is, if I have some relatively low powered hw with a beefy compositor engine, the best way to utilize that hw block and not just leave it sitting idle, is with a display architecture that looks more like wayland than x11. This is why surfaceflinger and mir share the same architecture (at a high level) as wayland. As a graphics driver developer, I can tell you, it's what you have to do to take advantage of what the hw has to offer in order to get a smooth fluid UI.

The udev dependency seems like a pretty minor detail compared inventing a new display architecture. If freebsd folks have a bit of porting effort to do, then so be it. They should get to work, rather than complaining about linux leaving them behind.

Of course, when *I* say UNIX-like I include OS's powered by the Linux kernel but the whole point of the comment was to reference other OS's that are not powered by Linux. That should be common sense in the context it was used.

given that it took a few posts for me to even figure out what your concern was, I'd say that you are providing less context than you think. People can't read your mind via forum.

[robclark]

On Wayland, there is practically nothing else for the server to do on graphics than to composite and present, and you cannot live without compositing if you want to have windows. On that point of view, "fully accelerated" is accurate. Server side rendering simply does not exist.

Accelerating Wayland clients is a whole different issue, and you already found the github pull request to enable EGL/GLESv2 clients.

Enabling accelerated clients is mostly about allowing the server to understand hardware-based graphics buffers. Once you can pass all needed kinds of buffers to the server, and the server can put them on screen or composite with them, you're practically done from the Wayland point of view. As hardware buffers are platform specific, the platform specific Wayland protocol bits are usually hidden in libraries like EGL. But sometimes we just cannot live without platform specific code in the server.

One thing that has been hindering the EGL/GLESv2 enablement work is that the standard EGL API hides too much, and that was practically the only API we had towards the firmware to manage hardware buffers.

interesting.. ok, so if EGL was the main constraint for accelerating x11 (in that it doesn't really provide a sane way for sw fallback rendering), then maybe their is hope. The proposed solution for that same issue for glamor that came up at last XDC was simply "well, we own the gl stack too.. let's just implement whatever extensions we need to make glamor perform well". I suppose if the same extensions were added to r-pi's egl/gl then it could use glamor as well for passable x11 accel?

Still doesn't change the fact that the dispmanx block will be sitting idle. I'm *assuming* (given the constraints of dispmanx) that it really is a parallel hw block to the gpu, and not simply an alternative interface provided by the fw bit implemented using the gpu.

[caligula]

Makes direct use to do what? Windows composition is just an extra overhead (even if it is the hardware accelerated overhead).

Can any of the Wayland supporters in this thread answer the following questions:
1. Do you actually have Raspberry Pi hardware?
2. Have you actually tried running Wayland/Weston on it?
3. What is your Wayland based software setup, which can readily replace the LXDE desktop used in Raspbian by default?

Took a bit long to get back here but
1) yes, two rev B boards and one rev A
2) yes, there are prebuilt binaries
3) I just run wayland, weston and do simple 3d development with Python. Still waiting for things to stabilize. There are no desktops availalble yet.

[pq__]

interesting.. ok, so if EGL was the main constraint for accelerating x11 (in that it doesn't really provide a sane way for sw fallback rendering), then maybe their is hope. The proposed solution for that same issue for glamor that came up at last XDC was simply "well, we own the gl stack too.. let's just implement whatever extensions we need to make glamor perform well". I suppose if the same extensions were added to r-pi's egl/gl then it could use glamor as well for passable x11 accel?

Still doesn't change the fact that the dispmanx block will be sitting idle. I'm *assuming* (given the constraints of dispmanx) that it really is a parallel hw block to the gpu, and not simply an alternative interface provided by the fw bit implemented using the gpu.

Yeah, I guess the X11 accel could be conceivable. I have no idea how practical it would be. I haven't even looked at the code drop yet, either.

To my understanding, the DispmanX compositing block really is a separate thing... but I have no idea how much hw is involved and what happens on sw (fw). It does compositing on the fly during scanout if it can, and if you have a too complex scene it can fall back to compositing into a big framebuffer, which is then scanned out. If you disable that fallback (or do not enable it in /boot/config.txt to begin with), falling off the real-time-compositing cliff causes the display signal to break. IOW, too many elements can cause your screen go blank or crazy.

To recap for everyone: Weston on RPi uses the DispmanX block to composite, and any GLESv2 machinery is left completely for clients to (ab)use.

[ssvb]

On Wayland, there is practically nothing else for the server to do on graphics than to composite and present

Yes and no. I'm not taking individual parts (just the server or compositor alone), but looking at the graphics stack as a whole. The purpose of a window system is to provide multiple windows for multiple applications in a multitasking operating system, so that they all work nicely together (can be resized, minimized, moved around, partially obscure one by another, etc.). The window systems (X11, Wayland or Mir) are all evaluated based on how well they can serve this purpose.

If you don't really need windows or multitasking, then something more simple is a better fit for the job. Such as the existing DispmanX EGL on the Raspberry Pi or the framebuffer based flavours of the EGL/GLES drivers on the other ARM systems.

and you cannot live without compositing if you want to have windows.

True. But the Xorg server has a pretty efficient built-in basic compositing already, which can only handle rectangular opaque windows. That's usually enough for a simple and usable desktop.

If we are talking about more fancy compositing such as translucency effects, shadows, bending or tilting windows, wobbliness and other stuff, then in the X11 desktop this is handled by a compositing window manager. And this kind of compositing is an extra stage in the pipeline with extra overhead.

Weston on the Raspbery Pi does something that could be called "partially fancy compositing". By supporting a subset of effects allowed by DispmanX hardware. It surely has more features than just the plain opaque rectangular windows. But it still needs to be proven/confirmed whether it offers any performance advantages over X11 desktop running without a compositing window manager.

On that point of view, "fully accelerated" is accurate. Server side rendering simply does not exist.

Looking only at a single component of the graphics stack makes no sense. We really need to look at the bigger picture.

Accelerating Wayland clients is a whole different issue, and you already found the github pull request to enable EGL/GLESv2 clients.

It's not a different issue. The whole point is that not being able to hardware accelerate client applications would be a failure for the Wayland based Raspberry Pi graphics stack. You simply can't call Wayland adaptation fully hardware accelerated without this feature. And of course nobody had any doubts that it was solvable because this work had been supervised by the Raspberry Pi foundation, who can introduce changes to the firmware blob if needed The only question is: when is it going to be really ready for the end users?

Once the Wayland based graphics stack is deployed to the end users, it would make a lot of sense to compare its performance/footprint with the traditional X11 desktop in the following areas:
1. Client side software rendering (XShmPutImage/DRI2/DRI3 vs. native Wayland methods)
2. OpenGL ES integration (DRI2/DRI3 vs. native Wayland methods)
3. Legacy X11 software support (using native Xorg vs. using XWayland compatibility layer)

I'm keeping an eye on the Raspberry Pi Wayland news.

[robclark]

Yes and no. I'm not taking individual parts (just the server or compositor alone), but looking at the graphics stack as a whole. The purpose of a window system is to provide multiple windows for multiple applications in a multitasking operating system, so that they all work nicely together (can be resized, minimized, moved around, partially obscure one by another, etc.). The window systems (X11, Wayland or Mir) are all evaluated based on how well they can serve this purpose.

Sure.. with something like wayland you are at the mercy of what the client does as to whether it is "fully" accelerated or not. Although if there is going to be software involved (fallbacks or otherwise), I much prefer it to happen one time per frame / window update, upstream of the acceleration, never having to block waiting for the gpu. Versus x11 where you can have scenarios (depending on what your hw can do and whatever drawing operation your DDX driver can accel (ie. with EXA there are some operations that are always sw fallback). With x11, you can hit scenarios where you are alternating between cpu and gpu access to pixmaps, killing performance.

True. But the Xorg server has a pretty efficient built-in basic compositing already, which can only handle rectangular opaque windows. That's usually enough for a simple and usable desktop.

not entirely sure if you are talking about bare x11 (which is front buffer rendering, *not* compositing), or the composite extension (as used by gnome-shell/compiz/etc) which *can* handle transparency?

If we are talking about more fancy compositing such as translucency effects, shadows, bending or tilting windows, wobbliness and other stuff, then in the X11 desktop this is handled by a compositing window manager. And this kind of compositing is an extra stage in the pipeline with extra overhead.

well, it really depends on your hardware. A lot of the SoC's can composite a certain finite number of layers in scanout (and at least on some hardware, for opaque layers there is no extra cost.. the hw is clever enough to not read data from portions of obscured layers).

Once the Wayland based graphics stack is deployed to the end users, it would make a lot of sense to compare its performance/footprint with the traditional X11 desktop in the following areas:
1. Client side software rendering (XShmPutImage/DRI2/DRI3 vs. native Wayland methods)
2. OpenGL ES integration (DRI2/DRI3 vs. native Wayland methods)
3. Legacy X11 software support (using native Xorg vs. using XWayland compatibility layer)

I'm keeping an eye on the Raspberry Pi Wayland news.

just speaking generically, not specifically about r-pi:

at least conceptually wl_shm and XShmPutImage should be equivalent. And passing buffers via flink name (DRI2) vs dmabuf handle (DRI3) should be same (regardless of whether that is in contents of wayland or x11).

gl/gles integration performance differences should amount to client side decorations vs not. With hardware rendered clients (ie. opengl(es)), with either wayland or x11, there should be exactly the same number of copies (ie. one).. and that is regardless of of compositing vs non-compositing window manager in x11 case. The problem is server side (or really window manager side) decorations screw things up and you end up require an extra copy. But *technically* x11 can do client side decorations too.. so in theory it is a moot point. Although in practice client side decorations seem predominant in wayland and visa versa in x11.

wayland (or rather gbm, which weston uses) does have the egl-buffer-age stuff so it can do some clever optimizations that a compositing x11 window manager cannot do today (although the recent "present" extension should make this a possibility).

[daniels]

Makes direct use to do what? Windows composition is just an extra overhead (even if it is the hardware accelerated overhead).

in x11, yes, composition adds some extra overhead. in wayland, it doesn't, because the composition is done inside the display server. x11 actually does composition too without a compositor, it's just not very good at it. using the videocore shifts the composition load from the cpu (where you do memcpy forever) to dedicated hardware. so it is a pure performance benefit in this case.

Can any of the Wayland supporters in this thread answer the following questions:
1. Do you actually have Raspberry Pi hardware?
2. Have you actually tried running Wayland/Weston on it?
3. What is your Wayland based software setup, which can readily replace the LXDE desktop used in Raspbian by default?

1) yes
2) yes
3) none as yet.