Mesa "Vallium" - Software/CPU-Based Vulkan Based On LLVMpipe

[mppix]

Why would you want a slow software rendered Vulkan for the compositor rather than fast hardware accelerated OpenGL?

Or are you saying that because *you* have the hardware accelerated Vulkan everyone else can go to hell and put up with software rendering? Just so long as "Vulkan is being used!".

Sounds like not a solution to me. However I am sure Gnome will adopt it tomorrow and be even slower for non-gamers!

Oh boy.. so many things to unpack
- Who talks about Gnome? Do you have to bring your anti-Gnome lyrics to every topic?
- OpenGL has a software rendering for times where hardware rendering is not available; Vulkan can use one too; Vulkan will become as available as opengl in due time
- Even if a DE starts to investigating this; we are talking years for a usable implementation; I find this an interesting direction

[pal666]

i wonder whether talos with opengl renderer on zink via vallium works

[kpedersen]

Oh boy.. so many things to unpack
- Who talks about Gnome? Do you have to bring your anti-Gnome lyrics to every topic?

You were responding to a quote about "Vulkan-powered Wayland compositor". I referred to Gnome as an example because honestly due to the nature of Wayland as it is now, there is never going to be many different compositors.
Any software rendered compositor would be painful compared to hardware accelerated. As it stands, using a compositor like Gnome's over i.e VNC is pretty horrible.

I guess what I am saying is until Vulkan on hardware is possible on 99.9% of workstations, a software rendered version is *not* a solution. The only sane choice is that developers should stick to OpenGL until then.

But ~10 years ago I also suggested that a desktop environment requiring hardware accelerated OpenGL is absurd but that also happened. That is why Linux is getting useless with remote desktop systems. FOSS desktop developers are not always sane.

- OpenGL has a software rendering for times where hardware rendering is not available

Yes and having to use the software rendering slowly just to render a desktop UI is daft too. Luckily unlike Vulkan, most consumer machines do not need this.

[mppix]

...

The part that I find interesting is that a Wayland Vulkan compositor could obsolete the need for the Nvidia-specific implementation (EGLstreams) and possibly result in long-term benefits also for other platforms. Of course, this a very high level opinion. It is abundantly clear that this could not replace today's Wayland implementations anytime soon for several classes of devices (embedded, etc).

I hope you understand that your post came off as heavily opinionated.
Unrelated, I'd invite you to read up on today's state of X11 forwarding, VNC, etc. The problem is not the DE's use of graphics primitives but the compression (if it exists) of X11 forwarding, VNC, etc.

[kpedersen]

Unrelated, I'd invite you to read up on today's state of X11 forwarding, VNC, etc. The problem is not the DE's use of graphics primitives but the compression (if it exists) of X11 forwarding, VNC, etc.

I have done some measurements in the past and it seems the largest impact is extracting the pixels into main memory from the GPU (think glReadPixels) and the fact that we are dealing with a primitive raster. This may be less "chatty" than X11 (less back and forwards); but the overall amount of data is pretty extreme for large resolutions. In the past I have written (an admittedly brief) paper comparing rasterised graphics vs an "intelligent" protocol based on OpenGL (https://tinyurl.com/gld222) and intelligent does always seem to win. Even more so when dealing with large resolutions.

So compression is one of the few angles here to improve transmission of a raster image but there is only so much that can be done for a stream (you are right in that it is better than X11 but it also needs to send more in total) including a tradeoff for the processing power required for (de)compression.

Is what I am asking possible? A fast remote desktop solution? Ultimately yes; Windows NT 4.0 Terminal Services did it well. Unfortunately it looks less than appealing to modern users. Perhaps there is room for an "enterprise desktop" and a "home user desktop". It kinda worries me that Linux communities have no interest in enterprise remote desktops. Perhaps this simply is because ssh is seen as "better" and for everything else there is Windows :/

[mppix]

I have done some measurements in the past and it seems the largest impact is extracting the pixels into main memory from the GPU (think glReadPixels) and the fact that we are dealing with a primitive raster. This may be less "chatty" than X11 (less back and forwards); but the overall amount of data is pretty extreme for large resolutions. In the past I have written (an admittedly brief) paper comparing rasterised graphics vs an "intelligent" protocol based on OpenGL (https://tinyurl.com/gld222) and intelligent does always seem to win. Even more so when dealing with large resolutions.

So compression is one of the few angles here to improve transmission of a raster image but there is only so much that can be done for a stream (you are right in that it is better than X11 but it also needs to send more in total) including a tradeoff for the processing power required for (de)compression.

Is what I am asking possible? A fast remote desktop solution? Ultimately yes; Windows NT 4.0 Terminal Services did it well. Unfortunately it looks less than appealing to modern users. Perhaps there is room for an "enterprise desktop" and a "home user desktop". It kinda worries me that Linux communities have no interest in enterprise remote desktops. Perhaps this simply is because ssh is seen as "better" and for everything else there is Windows :/

I don't pretend to have a full answer. However, I think your proposal of sending OpenGL data gets complicated quickly if you want to support say CAD applications or gaming. In fact, I tend to look at the game streaming services (Nvidia, Sony, Google stadia) when looking at what is possible (high resolution/color + low round-trip latency). They all render sever-side and client-side decoding does not seem to be a major issue. First video(s) that I found on youtube (not that I recommend the channel in general)

https://www.youtube.com/watch?v=d3dNoCRzbAs

Also there are some "gaming" remote packages

https://www.youtube.com/watch?v=B821HqH-dWI

So I think it is definitely possible to get a good general purpose remote desktop environment. However, the open-source graphic developers are quite busy with Wayland and Vulkan (and fixing Wine) so it could take a while.

[stiiixy]

Gallium + Vulkan as Vallium is both a portmanteau and a double entendre, although a little obvious rather than hidden.

Good show, ol chap.

As for CPU rendered tech, I was under the impression it allowed for last resort, reliable, testing of the modules in techs like OpenGL and Vulkan you may not be fully using (especially when you can set a profile to a version), without having to resort to annoying things like constant driver version changes, video card swapping.

[kpedersen]

I don't pretend to have a full answer. However, I think your proposal of sending OpenGL data gets complicated quickly if you want to support say CAD applications or gaming.

You are right in that CAD gets difficult because the sending of many commands (rather than using buffers) and (modern) games get difficult because of having such a highly programmable (and yet stupid state driven) pipeline. OpenGL|D started life as simply a way to preserve older software (<=OpenGL 2.0 era). It is quite cool being able to run a form of OpenGL on Plan 9 for example.
The ideas behind OpenGL|D works best when writing software specifically for it rather than porting existing GL programs. Perhaps something like this with a GUI system built on top. Wayland compositors would need to be written in a disciplined manner and unfortunately I feel the Linux desktop community is lacking discipline when it comes to features over functionality.

In fact, I tend to look at the game streaming services (Nvidia, Sony, Google stadia) when looking at what is possible (high resolution/color + low round-trip latency)..

Yes but you need to be careful to compare against these guys because they use bespoke hardware that is not available to your typical corporation. It certainly isn't available to the casual user. Some of them even have the equivalent of a GPU with a network card inbuilt. This means that no matter how long consumers wait, this kind of hardware will never appear in the shops.

There is a middle ground for simple remote desktop usage (i.e office documents) but unlike other operating systems, Linux actually seems to be peddling the wrong way!

[mppix]

Yes but you need to be careful to compare against these guys because they use bespoke hardware that is not available to your typical corporation. It certainly isn't available to the casual user. Some of them even have the equivalent of a GPU with a network card inbuilt. This means that no matter how long consumers wait, this kind of hardware will never appear in the shops.

I did not know that there were OEM GPU's with integrated networking. The GPU "networking" that I know of are NVlink marices and things GPU's with HDbaseT out, where the latter is not really relevant here.
Either way, I always assume that the highest latency is introduced by the home router and WiFi (when used on today's laptops without Ethernet).

By any chance, did you ever evaluate the compression of Spice?

https://www.youtube.com/watch?v=VUQYs72hrbM

Not sure if it is still a thing..