Wayland 1.16 & Weston 5.0 Release Candidates For Testing

[starshipeleven]

Pointless ad hominem ignored. Colourful descriptions also work as a quick and rough way to filter readership and exclude certain unwanted elements.

This hurts my feelings to no end. I posted more than 8000 times from 2015, and you never saw me once? I think I'll go cry under my bed.

/just kidding

D-Bus, Pipewire and Wayland solve 90-95% of the same tasks in similar but not exactly compatible ways (to be fancy "object-relational impedance mismatch"). The tasks in question are connection discovery, client authentication, feature negotiation and timing sensitive data- and metadata- transfers. Instead of finding a common denominator and build on that, maximizing code reuse, all three will get duct taped together with some difficult synchronization bugs waiting in a lot of code.

Sorry what? All the "connection discovery, client authentication, feature negotiation and timing sensitive data- and metadata- transfers" is stuff the application doing the remote desktop is dealing already on its own even on Xorg. Why I still have the same sensation I got with the other post?

In their plan, Wayland would require an API to offload the rendered screen to Pipewire, which is supposed to be a media processing backend/server/thing, and then any application doing remote desktop would communicate with Pipewire to get its audio/video stream. https://blogs.gnome.org/uraeus/2017/...hing-pipewire/
D-Bus is an inter-application messaging system and will be used to.... (wait for it).... do inter-application messaging.

I'm not seeing a Rube Goldberg machine here. I mean, OK, this is moving the media processing from the remote desktop applications into Pipewire, but the remote desktop applications usually rely on multimedia frameworks or libraries already to do the leg work on media processing anyway.

So far the only thing that happened is that GNOME created its own little API to do that, without contributing it upstream to Wayland. https://www.omgubuntu.co.uk/2018/06/...ecorder-ubuntu and some applications are already using it https://github.com/foss-project/green-recorder

It could be seen as a test run for something they plan to upstream, or not (see below).

The reason I bring it up as an example is that the features they require are on the 'security reasons - no' list. Synergy requires information about relative monitor positions, passive global cursor control, clipboard monitoring and injection, keyboard grabs and mouse/keyboard input injection.

Even if a way to provide this stuff gets a NAK from Wayland upstream it does not mean it is game over as it would be if Wayland was not just a protocol.

As noted with GNOME above, the DE project controls what features their compositor supports, so they can easily add additional stuff that isn't a Wayland standard, and it would not be a massive pain in the ass as doing a customized Xorg hack, as they are the upstream of the compositor already so there is no upkeep increase due to custom patches, Wayland is just a protocol their compositor can support.

So I think it's entirely possible that DEs could implement such APIs on their own and eventually converge on a de-facto standard.

The situation seems similar for Wine, which is almost certainly the most difficult target to write a wayland backend for. It was also on their roadmap, yet there is not a single line of code to that effect and not mentioned at all on last month's wineconf (judging by the videos).

Afaik Wine does not really require to work on Wayland natively anyway, it can be used with XWayland compatibility layer and it will be fine.

[oiaohm]

The lack of Desktop Recording support is a symptom, though it's the one where the pending solutions are hilariously bad. If you want a case to think about and see how well the 'protocol' would be able to support, take 'Synergy' (the virtual KVM thing).

First question is should lots of Synergy be wayland/X11 level in the first place.

D-Bus, Pipewire and Wayland solve 90-95% of the same tasks in similar but not exactly compatible ways (to be fancy "object-relational impedance mismatch").

https://github.com/PipeWire/pipewire/blob/master/src/modules/module-protocol-dbus.c

There is not three here. Pipewire is extension to dbus. dbus has too much overhead for your general compositing. So it dbus with extensions vs wayland extention vs low level for what is the right way to-do stuff.

Also you have to consider the fact 4.19 Linux kernel we got virtual kernel mode setting. Pipewire can be made go down to VKMS(set VKMS as clone of current screen output) in the kernel so allowing it to capture screen output of Linux text based console and x11 and wayland by the one interface provide by kernel.

From the looks of it, it is very much empty promises. The reason I bring it up as an example is that the features they require are on the 'security reasons - no' list. Synergy requires information about relative monitor positions, passive global cursor control, clipboard monitoring and injection, keyboard grabs and mouse/keyboard input injection.

Most of what you listed is a duplicate path provided by X11.
https://www.freedesktop.org/wiki/Accessibility/ AT-SPI2 would not be crossing your mind right. This is another thing on dbus. Wayland desktops really should support AT-SPI2 for the people needing it. AT-SPI2 covers you clipboard monitoring and injection, mouse/keyboard input injection. Passive global cursor control by wayland and X11 compositors/wm supporting AT-SPi2. Please note AT-SPI2 include authentication protection of dbus.

X11 features that Synergy is using have mostly being duplicated by AT-SPI2 but AT-SPI2 has security where X11 implementations do not.

Of course this does leave the question if lots of Synergy should be done at the wayland/x11 server in the first place.

Lets think could large section of Synergy be done as a fuse evdev item like https://gitlab.com/xboxdrv/xboxdrv so be a evdev device for keyboard and mouse. To be correct don't be a mouse be a touch screen pointer with absolute address. This route you keyboard and screen pointer would be X11/Wayland/Linux console neutral.

Doing a fuse based evdev would fix the problem of Synergy not working before logged in also fixes the issue of Synergy not working if X11 does not start.

The situation seems similar for Wine, which is almost certainly the most difficult target to write a wayland backend for. It was also on their roadmap, yet there is not a single line of code to that effect and not mentioned at all on last month's wineconf (judging by the videos).

Really wine is not a hard problem just Android support promises to pay money sooner. Android and Wayland provide the same set of basic limitations. Please note coding under android and wayland means dealing with Open GL ES not your X11 GLX. Yes global cursor and restricted keyboard injections and restricted mouse/keyboard input injection are all part of Android as well.

A lot of the complaints people have about wayland is tunnelled visioned. Attempting to get wayland to-do everything X11 use to is wrong thinking that x11 was in fact duplicating lots of system functionality.

VKMS + fuse evdev could give us remote desktop on Linux from as soon as networking comes up. Of course how can this be optimised and improved still in the future.

[cj.wijtmans]

Excuse me good sir, have you heard about our Lord and Savior, Rust?

Back serious, I think I've seen people complaining about the lack of desktop recording support in Wayland.

But overall I agree with you. Wayland per-se is mostly OK, what is still unstable are the DEs that are adding support for it. And freezing Wayland's limited development and bugfixing won't change a thing, it's not the culprit.

As long as popular software for recording/streaming like OBS Studio doesnt work on wayland and compositors keep implementing their own shitty recording apps wayland is broken. The whole mir/wayland thing has the linux desktop broken, not fixed.

[crazyloglad]

Sorry what? All the "connection discovery, client authentication, feature negotiation and timing sensitive data- and metadata- transfers" is stuff the application doing the remote desktop is dealing already on its own even on Xorg.

Yes? that’s the argument? A client here would need to solve the same problem (each of the mentioned categories) once for D-bus, once for Pipewire and once for Wayland. Instead of.. once. Local “simplicity” (compositor side) traded for global (everyone else) “complexity”. Solving the same problem (Desktop IPC) three times even though they could’ve shared a solution - I've demoed such a solution twice the last ~5 years, at a third of the code cost of wayland-server alone. 3 Desktop IPC systems each half-assing the same task and somehow expected to cooperate via yet to be completed mechanisms is, in my world, a system design facepalm to such a degree that even Android engineers can collectively point fingers and laugh; not a good place to be in.

Why I still have the same sensation I got with the other post?

It was quite clear from your previous reply that you already made up your mind on the basis of some superficial bias, and I don’t care enough to try and change it. Why not do some basic doxxing?

In their plan, Wayland would require an API to offload the rendered screen to Pipewire, which is supposed to be a media processing backend/server/thing, and then any application doing remote desktop would communicate with Pipewire to get its audio/video stream. https://blogs.gnome.org/uraeus/2017/...hing-pipewire/
D-Bus is an inter-application messaging system and will be used to.... (wait for it).... do inter-application messaging.

I'm not seeing a Rube Goldberg machine here. I mean, OK, this is moving the media processing from the remote desktop applications into Pipewire, but the remote desktop applications usually rely on multimedia frameworks or libraries already to do the leg work on media processing anyway.

So far the only thing that happened is that GNOME created its own little API to do that, without contributing it upstream to Wayland. https://www.omgubuntu.co.uk/2018/06/...ecorder-ubuntu and some applications are already using it https://github.com/foss-project/green-recorder

It could be seen as a test run for something they plan to upstream, or not (see below).

Even if a way to provide this stuff gets a NAK from Wayland upstream it does not mean it is game over as it would be if Wayland was not just a protocol.

As noted with GNOME above, the DE project controls what features their compositor supports, so they can easily add additional stuff that isn't a Wayland standard, and it would not be a massive pain in the ass as doing a customized Xorg hack, as they are the upstream of the compositor already so there is no upkeep increase due to custom patches, Wayland is just a protocol their compositor can support.

So I think it's entirely possible that DEs could implement such APIs on their own and eventually converge on a de-facto standard.

If the “de facto standard” you assume would emerge trumps any attempt at defining a protocol in the domain of said protocol, what’s the point of having one? The protocol is, by its very definition, a formal attempt at coming to an agreement as a way of avoid de facto “standards” to popup because they suck from an engineering standpoint and is a big contributor to how X11 turned into the hot mess it is.

Wayland (the ecosystem) literally burned half a decade on “sort of” agreeing (current solution being incomplete, partially marked unstable, not uniformly accepted or implemented) on the details for allocating a buffer, populating it with pixels, and sending it from point a to point b. What makes more sense, adding a direction attribute for doing the same thing from point b to point a - or having n different actors repeating the same dance again, each coming up with a solution, let them battle it out for “de facto” status and then merge if there is anyone still alive that.

Afaik Wine does not really require to work on Wayland natively anyway, it can be used with XWayland compatibility layer and it will be fine.

If the goto answer to developers trying to write something for wayland is first “don’t do it, use a toolkit”, (which it is) ok - that's a huge flashing warning sign that someone fucked up simplicity royally, but lets continue. If developers then run into the problem that "writing a toolkit backend for wayland is intractable or impossible in its current state" (what the wine case indicates), your suggested answer is to "don't fix the protocol, use X"? Then there is a very easy optimization to be done here. One that won't end well for Wayland.

[crazyloglad]

First question is should lots of Synergy be wayland/X11 level in the first place.

https://github.com/PipeWire/pipewire...rotocol-dbus.c
There is not three here. Pipewire is extension to dbus. dbus has too much overhead for your general compositing. So it dbus with extensions vs wayland extention vs low level for what is the right way to-do stuff.

Pipewire overlays its IPC (src/pipewire/*.c) over D-Bus if that method is found, or there is the module-protocol-native.c (domain socket, …) but it only treats that as a carrier wave. Binder proved that D-bus like message passing worked just fine as a basis for system graphics and even though Wayland is very verbose, it pales compared to the corresponding set of Android "managers" and "services" enough to a degree that would offset most of all the extra ‘What the F…’ in D-Bus.

Also you have to consider the fact 4.19 Linux kernel we got virtual kernel mode setting. Pipewire can be made go down to VKMS(set VKMS as clone of current screen output) in the kernel so allowing it to capture screen output of Linux text based console and x11 and wayland by the one interface provide by kernel.

Ooh, technically arguments. Lets dance :-) I normally prefer “implement it on the lowest level of abstraction possible” (along with mechanism, not policy) solution to everything, hence my previous comments - but these problems cut across the stack, and that is what makes them interesting. The problem with both your suggestions (vkms + evdev) are very similar. The metadata that is needed to implement these features at the desired precision aren’t available on the chosen level of abstraction; You need context from the window system. See below.

Most of what you listed is a duplicate path provided by X11.
https://www.freedesktop.org/wiki/Accessibility/ AT-SPI2 would not be crossing your mind right. This is another thing on dbus. Wayland desktops really should support AT-SPI2 for the people needing it. AT-SPI2 covers you clipboard monitoring and injection, mouse/keyboard input injection. Passive global cursor control by wayland and X11 compositors/wm supporting AT-SPi2. Please note AT-SPI2 include authentication p§rotection of dbus.

X11 features that Synergy is using have mostly being duplicated by AT-SPI2 but AT-SPI2 has security where X11 implementations do not.

Of course this does leave the question if lots of Synergy should be done at the wayland/x11 server in the first place.

Note the suggestions on how to deal with AT-SPI2 on Wayland, that the universal answer seems to be "leave it to the compositor" and then a lot of empty space.

Lets think could large section of Synergy be done as a fuse evdev item like https://gitlab.com/xboxdrv/xboxdrv so be a evdev device for keyboard and mouse. To be correct don't be a mouse be a touch screen pointer with absolute address. This route you keyboard and screen pointer would be X11/Wayland/Linux console neutral.

Doing a fuse based evdev would fix the problem of Synergy not working before logged in also fixes the issue of Synergy not working if X11 does not start.

A lot of the complaints people have about wayland is tunnelled visioned. Attempting to get wayland to-do everything X11 use to is wrong thinking that x11 was in fact duplicating lots of system functionality.

VKMS + fuse evdev could give us remote desktop on Linux from as soon as networking comes up. Of course how can this be optimised and improved still in the future.

First - Evdev isn’t FUSE, it’s uinput. Uinput is “root only” because if you give someone access to it, all the input isolation work touted as “security” in Wayland would be undermined; you can imitate any other device and libinput wouldn’t be able to tell the difference. Second, it couldn’t work at that level, as its grab/release/inject behavior is dependent on input translated to both the coordinate space of the wayland “outputs” and of the state of the client and shell grabs, you wouldn’t know when the cursor hits the edge of the screen to trigger warp/grab/redirect without knowing the state of the shell WM.

Recall all the input acceleration that libinput does? that is also contextual to the compositor coordinate system(s) - so your “absolute address” wouldn’t match the “address” where the user sees the pointer. Keyboard is even worse, as the state machine for the keyboard layout virtual machine (latched, pressed, depressed, held + modifiers and repeat- state) has one instance in every wayland client and in the compositor itself. Then comes clipboard/drag’n’drop as icing on the cake.

Really wine is not a hard problem just Android support promises to pay money sooner. Android and Wayland provide the same set of basic limitations. Please note coding under android and wayland means dealing with Open GL ES not your X11 GLX. Yes global cursor and restricted keyboard injections and restricted mouse/keyboard input injection are all part of Android as well.

Android has both a permission negotiation mechanism, authentication mechanism and UI for these very purposes - it has been part of the system design for ages. Let me quote the *entirety* of formal wayland documentation on the problem; not what people are imagining is there, but what is actually documented (the wayland pdf, section 4.7).

“• mostly about access to underlying buffers, need new drm auth mechanism (the grant-to ioctl idea), need to check the cmd stream? • getting the server socket depends on the compositor type, could be a system wide name, through fd passing on the session dbus. or the client is forked by the compositor and the fd is already opened.”

Didn’t even bother to provide complete sentences, only vague bullet points. That's the entirety of the documented security section. Does that inspire confidence to you?

Wine-x11drv is about 21kloc (+3kloc audio, 2+kloc XGL), the android version is ~6kloc (+2k audio). Android gets away with being much simpler because the client is “fullscreen”, it doesn’t need to integrate and mix with another windowing system, and that’s where the cost and the complexity comes from. You’d either need fully special support paths like XWatyland has, which overlays X like WM semantics inside the compositor through a side channel or serious protocol work. The way win32 etc. does window management is very much in conflict with how xdg-wm-base sees popups, toplevels, positioning and dirty updates. The size of this task is larger than that of the Qt and GDK wayland backends - both of which are still very much “Work in Progress” status. The ‘android version’ approach would be comparable to Chrome-Ozone or Firefox-Gecko backends in size and requirements, which are still even more “Work in Progress” after 3+ years.

[oiaohm]

Note the suggestions on how to deal with AT-SPI2 on Wayland, that the universal answer seems to be "leave it to the compositor" and then a lot of empty space

One of the super fun parts here is if you Wayland compositor is based around GTK/QT toolkits you automatically got AT-SPI2 support from the toolkits. So this is not leave it to the compositor or wayland protocol but leave it to the graphical toolkit you use and hopefully it be one that support AT-SPI2 and for those with physical issues all toolkits really should support it.

AT-SPI2 is one of the bits that there in most cases yet its not part of the Wayland protocol.

First - Evdev isn’t FUSE, it’s uinput. Uinput is “root only” because if you give someone access to it, all the input isolation work touted as “security” in Wayland would be undermined; you can imitate any other device and libinput wouldn’t be able to tell the difference.

Injecting keystrokes not able to clearly tell the difference under X11 already happens. uinput does not have to be root only. Seats under logind/consolekit2 control what devices are displayed to user. Synergy could appears as it own device type with its own dbus. So providing a fuse device on uinput does not mean allowing applications todo what ever with uinput. Also this does not have to undermine input isolation.

Yes uinput can be used to pretend to be other devices but it also can be used to create a unique device/s that libinput could identify so avoiding acceleration problems and getting extra information required for function out of libinput. uinput for a keyboard/mouse works when you are in Linux text mode no X11 or wayland. So wayland compositor/x11 server crashed has crashed on a machine if Synergy was using uinput it would be possible to switch to a text based terminal on that machine and restart the graphical using Synergy currently you cannot do that because Synergy dies as soon as the graphical does.

Reality here providing a virtual keyboard/mouse/pointer you are undermining security anyhow. Look for the path that does not result in duplication so you have less code to audit for security issues.

Second, it couldn’t work at that level, as its grab/release/inject behavior is dependent on input translated to both the coordinate space of the wayland “outputs” and of the state of the client and shell grabs, you wouldn’t know when the cursor hits the edge of the screen to trigger warp/grab/redirect without knowing the state of the shell WM.

That is if you are emulating a mouse. That is not your only option.

Recall all the input acceleration that libinput does? that is also contextual to the compositor coordinate system(s) - so your “absolute address” wouldn’t match the “address” where the user sees the pointer.

Ever wonder why Synergy mouse fails with particular games so badly. Touch screens and light pens have absolute address values on screen. Reporting as a absolute device will avoid games and libinput attempting to do input acceleration on that pointer and not trap that pointer. So a uinput device created for each screen connected to the computer.

Keyboard is even worse, as the state machine for the keyboard layout virtual machine (latched, pressed, depressed, held + modifiers and repeat- state) has one instance in every wayland client and in the compositor itself.

This one is always a pain. uinput can used to provide raw keyboard information but its not just limited to that. uinput is used by some software testing systems. Question do you have to obey the state machine. Ever noticed how when you plug in 2 keyboards+ it works. Going to evdev/uinput you only need to worry if key is down or up. You will get informed when they locks led change.

Now where in wayland system is (latched, pressed, depressed, held + modifiers and repeat- state) being calculated and set. The answer is libinput. Slow but surely libinput is coming the only input driver on x.org as well.

The reality here is evdev system/uinput is extendable. So you Synergy uinput keyboard could have extra features that libinput knows about so it reports the state machine out to the device. So there are three possible paths to the keyboard state machine. Having unique code for X11/Wayland and using those protocols to access the statement or coming from the evdev/uinput side and working with libinput that would be x.org and wayland compositor neutral. The good part is the uinput device could drop back to being a standard raw device when X11/wayland are not running so text based mode of Linux works.

Remember to get access to the keyboard state machine under current x.org/wayland you are basically working your way to libinput.

Then comes clipboard/drag’n’drop as icing on the cake.

Clipboard/drag'n'drop is quite straight forwards as these you just need to use a proxy application aligned with your input device. You already have clipboard/drag'n'drop proxies working under wayland.

Wine-x11drv is about 21kloc (+3kloc audio, 2+kloc XGL), the android version is ~6kloc (+2k audio). Android gets away with being much simpler because the client is “fullscreen”, it doesn’t need to integrate and mix with another windowing system, and that’s where the cost and the complexity comes from. You’d either need fully special support paths like XWatyland has, which overlays X like WM semantics inside the compositor through a side channel or serious protocol work. The way win32 etc. does window management is very much in conflict with how xdg-wm-base sees popups, toplevels, positioning and dirty updates. The size of this task is larger than that of the Qt and GDK wayland backends - both of which are still very much “Work in Progress” status. The ‘android version’ approach would be comparable to Chrome-Ozone or Firefox-Gecko backends in size and requirements, which are still even more “Work in Progress” after 3+ years.

Wine android version driver is so small because wine just does it virtual desktop. Wine virtual desktop mode means it does not need to give a rats as long as it can get a window to draw into wine can function.

Yes win32 windows management conflicts with way X11 does thing why wine has options to basically tell the wm please don't do anything let wine do all the drawing including boarders at times. Life for wine is a lot simpler without server side decorations.

Desktop integration is a feature that is technically option and is not required for wine to function. So wine project can just throw desktop integration out for a while and run in pure virtual desktop mode. Of course this means you only need like 6k of code instead of 21k.

The work required to get wine working under wayland is way less. Because virtual desktop just need a single output buffer and wine will work.

Its a lot harder problem when you have to care about integration. Wine is one of the programs where desktop integration is optional.