X.Org Could Use More Help Improving & Addressing Its Security

[birdie]

The most annoying thing missing from either X11 or Wayland right now is ability to detach your graphics session from one output and connect it to another, i.e. detach you session from local display/inputs and attach it to remote display server for remote access and than go back to local.
It actually should be possible for Wayland I think I've read news here about redirecting output here. But right now even screen share with GNOME is a no go unless you're logged in and unlocked the session.

We have it for X11: https://xpra.org

[blacknova]

We have it for X11: https://xpra.org

I'll try it out.

[billyswong]

...

Next part would be true multi head support. That a while out but the KDE developers are currently working on it. This is where you can run a wayland compositor per monitor connected and also transfer applications between those compositors. This would provide higher crash resistance.

Wayland is head in a direction way different to X11. X11 you start one X11 server that controls all your monitors as one big unified item. Wayland the client application is not to know about the monitors in the early design this is no mistake. This is part of the long term to check point and restore and multi head support. Think about it if the client can ask the server about all connected monitors how do you do proper isolation in multi head support and transfer applications between compositors.

...

This is the part I disagree with Wayland blueprint. The client application "not to know about the monitors" is not a solution to multi-head support.

Case 1: All monitors are in the same model or similar enough.
In this case X11 way of doing things is okay because all monitors are going to apply the same color profile and same dpi setting. There is nothing wrong with treating them as "one big unified item".

Case 2: Mixed DPI / Refresh Rate / Colour Profile / WCG / HDR / Orientation
In this case the "correct" bitmap is different for each monitor.
Outputting a HDR or WCG image / video to capable monitors is different from legacy monitors. Those new monitors are also all different in their actual capability in colour gamut / dynamic range. There are multiple ways to map wider gamut and higher range to narrower gamut and lower range. Different applications may want to choose different way of mapping.
Subpixel AA and font hinting for text can only be done when DPI and subpixel orientation is known. Even for the same monitor model, a different bitmap is required when the thing is rotated by 90 degrees. Cutting all these information away from client application? The output result is guaranteed to be worse than X11. At least X11 can give correct result for one of the monitors, not none of them.
Real time animation want to know the monitor refresh rate to vsync and prevent tearing. Video streaming want to know the same to provide the best high frame rate video but no higher than what the monitor(s) is/are capable. Without the refresh rate information, both will degrade in either energy efficiency or output quality.

Maybe Wayland team think they can abstract away all these and do the transformation themselves after client applications output their bitmap. Or they may be doing what happened in GTK4 - axe subpixel AA and font hinting in text, then refuse to admit they are doing a disservice to users.

A proper multi-head support shall let client applications know the monitor profiles individually and adapt accordingly.

[sinepgib]

1) each framework draws a different window decoration; but this is only an aesthetic problem

I'd say considering one of the selling points for Wayland is no lost frames and no tearing, it's part of the premise that we want to avoid aesthetic problems.

Maybe Wayland team think they can abstract away all these and do the transformation themselves after client applications output their bitmap. Or they may be doing what happened in GTK4 - axe subpixel AA and font hinting in text, then refuse to admit they are doing a disservice to users.

Most likely like you say but with Wayland saying nothing, just leaving that up for the compositor to decide.

[oiaohm]

Case 1: All monitors are in the same model or similar enough.
In this case X11 way of doing things is okay because all monitors are going to apply the same color profile and same dpi setting. There is nothing wrong with treating them as "one big unified item".

This case 1 in fact falls flat on face. As soon as you start using minor calibration tools you find out something same model monitors don't have the same color profile. Let alone the idea of similar enough. So the one big unified item is in fact broken and always has been.

Case 2: Mixed DPI / Refresh Rate / Colour Profile / WCG / HDR / Orientation
In this case the "correct" bitmap is different for each monitor.
Outputting a HDR or WCG image / video to capable monitors is different from legacy monitors. Those new monitors are also all different in their actual capability in colour gamut / dynamic range. There are multiple ways to map wider gamut and higher range to narrower gamut and lower range. Different applications may want to choose different way of mapping.

https://www.collabora.com/news-and-blog/blog/2020/11/19/developing-wayland-color-management-and-high-dynamic-range/

HDR basically forces the compositor to have color space conversion. Majority of applications are like to be SDR. So a SDR application moving between monitors will not want to know about HDR. This is the reality here applications going forwards need to be disconnected from the monitors colour depth. Yes a HDR only application a user may want it output on a SDR screen. So lot of cases going forwards applications only need to know what the wayland compositor tells them about the screen. Of course this could be a total fib by the wayland compositor.

Subpixel AA and font hinting for text can only be done when DPI and subpixel orientation is known. Even for the same monitor model, a different bitmap is required when the thing is rotated by 90 degrees. Cutting all these information away from client application? The output result is guaranteed to be worse than X11. At least X11 can give correct result for one of the monitors, not none of them.

https://www.freetype.org/freetype2/d...l-hinting.html Interesting enough subpixel orientation for most common form of AA font and font hinting used on Linux is not effected by sub-pixel orientation. DPI yes orientation no. Why this is simple the type of font AA is the non flat colour that depends on the subpixel orientation freetype does not use the non flat colour AA font or font hinting. Yes there is a down side to non flat colour AA font or font hinting the fact you can get rainbow effects around text when the monitor DPI is not as consistent as it should be or if you don't have the right rotation it really does not look any better than flat color subpixel-hinting and aa for fonts. So downsides with no benefit..

Sorry you don't in fact require a different bitmap when rotated 90 degrees on Linux when using distribution provided freetype. There are different ways to solve the subpixel problem. Microsoft and others patented the using the different color pixels in non flat colour for subpixel hinting that causes freetype to have to look into other methods that did not break patent and the result was finding flat color subpixel hinting and AA gave a lot of advantage without most of the downsides.

Yes this is on of the arguments that people bring up that just happen not to really apply to OS X, OSi, Andriod and Desktop Linux as all use flat color subpixel hinting and AA for fonts. Yes when you rotate 90 degrees on windows you have to. Yes X11 applications basically always use the flat color system. So this is area where X11 and Wayland will not be different so the idea that output result is guaranteed to be worse than X11 is bogus because the system that causes the 90 degree rotation problem is just not used on the X11 platform or if it is used its insanely rare because it was patented that means people would have had to pay money to use it.

Yes apple decide with OS X and OSi that it was not worth paying for the patents because there was no quality gains really for fonts. Flat color system for font hinting and font AA just happens to be the better one for least complexity and the best quality output.

Real time animation want to know the monitor refresh rate to vsync and prevent tearing. Video streaming want to know the same to provide the best high frame rate video but no higher than what the monitor(s) is/are capable. Without the refresh rate information, both will degrade in either energy efficiency or output quality.

vsync that one gets complex particular with gsync and freesync.

We also see example where the wayland compositor lies to applications.

https://github.com/Plagman/gamescope

Yes a real time animation want to know the monitor refresh rate is this in fact true??? Valve has found many games give them 200hz+ monitor refresh rate that is the real monitor refresh rate and the application just dies because they were not designed in a time frame where 200hz monitors existed. Yes you can have the reverse problem.

So do you really need to know the monitor refresh rate. Or do you just need to know when your output was displayed. Video streaming can be done quite well without knowing the exact monitor refresh rate.

Something else to be aware of just because you have a 200hz monitor does not mean you have a GPU able to process to produce 200hz either. So telling application the truth on monitor refresh rate can be the cause of the application running itself out of GPU and CPU resources.

A proper multi-head support shall let client applications know the monitor profiles individually and adapt accordingly.

Multi head for crash resistance the application can only have information from the compositors/compositors it is connected to. The reality here the compositor is to let the application know as much information as the application really need to know about the monitor. Let the application know information it does not need to know can make application miss behave. So tell application that its has 200hz monitor when the application is perfectly happy with 50hz can result in the application not performing right.

The split from application and monitor by the compositor is no mistake with wayland. Letting the compositor be the gate keeper on what information applications know cure a lot of problems. Big mistake in your logic you never allowed for the problem where client application cannot adapt to the monitor profile information. Yes modern gaming and workstation monitors with legacy applications this is quite a common problem where the application cannot adapt to the new monitor.

[mppix]

Not exactly - title bar in X11/Windows is WM managed area if you click on some buttons in window title your application might receive appropriate event or not if WM handle event by itself, while in Wayland whole window is application managed area and your application will get input event, not some DE/WM system event.

Are you saying that a gui toolkit cannot (or should not be able to) create system events?
... this is trivial to pass through.

I guess I'm just generally suprised why some still argue for SSD, when modern toolkits start leverage the usually 'dead space' of the topbar in more creative ways. See every modern browser, or hamburger menus (KDE and Gnome).

[blacknova]

Are you saying that a gui toolkit cannot (or should not be able to) create system events?
... this is trivial to pass through.

I guess I'm just generally suprised why some still argue for SSD, when modern toolkits start leverage the usually 'dead space' of the topbar in more creative ways. See every modern browser, or hamburger menus (KDE and Gnome).

Actually the point is that in some cases you don't want application to receive any events at all, i.e. some WMs support option roll up window so only header is visible, or tiling WM would hide any window controls.
If application is pure CSD it would give exactly zero f$cks of what you or your WM want and would behave entirely on its own, meaning mixing apps from different DEs would be coming even more of the headache.

[mppix]

RDP has always been a vector protocol ( [MS-RDPEGDI]: Remote Desktop Protocol: Graphics Device Interface (GDI) Acceleration Extensions ) and I remember I could use it just fine over a 56Kbit/sec modem connection, something `ssh -X -C` struggled with.

In fact not only it's vector based, it applies the same to video/audio/3D graphics, so those are a lot more efficient as they are decoded and played on the client side and consume significantly less bandwidth.

It is almost a cheap shot on this forum to show how wrong birdie is. Anyhow...
ssh -X passes bitmaps nowadays because no UI toolkit that I am aware of uses X drawing primitives today. UI toolkits change a lot faster than display servers/compositors or remote desktop protocols so separate the two is a good thing.
Separately, 3D graphics anything needs massive bandwidths (pcie) that you don't want to pass anywhere.
Last, pretty much every system has HW acceleration to encode/decode video streams that can be leveraged by remote desktop protocols.

Again, this feature is not and will never be available under Wayland because it knows nothing but rasterized image buffers. VNC under Wayland will be easier to implement and likely work faster than for X11 and that's it.

Standard VNC lacks a functional governing body and is stuck in the 90s. Its main problem is the utter lack of support of modern streaming technologies as well as the lack of extensions, e.g. audio channel return, usb redirect, etc.
Standard VNC is dead except for legacy stuff. Its derivatives are only mildly better (and largely incompatible...).
Gnome is introducing RDP for this reason.
Alternatively, there is SPICE as a proper modern and high performance open-source remote desktop solution.

[oiaohm]

We have it for X11: https://xpra.org

I'll try it out.

https://xpra.org/trac/wiki/Usage/OpenGL

birdie except when you come to opengl applications xpra turns into a mess. Xpra can reconnect some X11 applications simply others you are kind of screwed. Yes this is due to X11 being too diverse in what it can do over the protocol. Yes blacknova pays to read the docs.

This is just one of those on going X11 half fixes that need to be fixed correctly. Yes some of the reasons why you cannot checkpoint and restore desktop X11 applications are also the reasons why xpra can be problematic. Please be aware majority of our current generation graphical toolkits for Linux use opengl poor support for opengl is truly problemmatic.

[billyswong]

This case 1 in fact falls flat on face. As soon as you start using minor calibration tools you find out something same model monitors don't have the same color profile. Let alone the idea of similar enough. So the one big unified item is in fact broken and always has been.

Well, we can still "lies to application" (your words in later paragraph here) and do the gamma correction in the compositor side. Most software aren't colour-managed and they are fine with this kind of post-processing. Wayland is not telling applications individual colour profile of each monitor so such post-processing still need to happen. It is not taking advantage of the compositing buffer being separate for each monitor. Therefore any supporter of Wayland design lost their ground in complaining X11 presenting one big unified buffer for this particular case.

https://www.collabora.com/news-and-blog/blog/2020/11/19/developing-wayland-color-management-and-high-dynamic-range/

HDR basically forces the compositor to have color space conversion. Majority of applications are like to be SDR. So a SDR application moving between monitors will not want to know about HDR. This is the reality here applications going forwards need to be disconnected from the monitors colour depth. Yes a HDR only application a user may want it output on a SDR screen. So lot of cases going forwards applications only need to know what the wayland compositor tells them about the screen. Of course this could be a total fib by the wayland compositor.

It doesn't change my standpoint that "the "correct" bitmap is different for each monitor". Doing the conversion in compositor side in the case of colour management doesn't change such fact.

https://www.freetype.org/freetype2/d...l-hinting.html Interesting enough subpixel orientation for most common form of AA font and font hinting used on Linux is not effected by sub-pixel orientation. DPI yes orientation no. Why this is simple the type of font AA is the non flat colour that depends on the subpixel orientation freetype does not use the non flat colour AA font or font hinting. Yes there is a down side to non flat colour AA font or font hinting the fact you can get rainbow effects around text when the monitor DPI is not as consistent as it should be or if you don't have the right rotation it really does not look any better than flat color subpixel-hinting and aa for fonts. So downsides with no benefit..

Sorry you don't in fact require a different bitmap when rotated 90 degrees on Linux when using distribution provided freetype. There are different ways to solve the subpixel problem. Microsoft and others patented the using the different color pixels in non flat colour for subpixel hinting that causes freetype to have to look into other methods that did not break patent and the result was finding flat color subpixel hinting and AA gave a lot of advantage without most of the downsides.

Yes this is on of the arguments that people bring up that just happen not to really apply to OS X, OSi, Andriod and Desktop Linux as all use flat color subpixel hinting and AA for fonts. Yes when you rotate 90 degrees on windows you have to. Yes X11 applications basically always use the flat color system. So this is area where X11 and Wayland will not be different so the idea that output result is guaranteed to be worse than X11 is bogus because the system that causes the 90 degree rotation problem is just not used on the X11 platform or if it is used its insanely rare because it was patented that means people would have had to pay money to use it.

Yes apple decide with OS X and OSi that it was not worth paying for the patents because there was no quality gains really for fonts. Flat color system for font hinting and font AA just happens to be the better one for least complexity and the best quality output.

The article you linked to tells a different story to me, unlike your interpretation. What FreeType v40 bypassing is patented subpixel hinting (grid-fitting). FreeType still need to know the monitor orientation to apply subpixel AA and hinting to their corresponding direction. In standard subpixel order, left edge pixel is blueish and right edge pixel is redish. When a monitor is flipped upside down, the left edge pixel now shall be redish and right edge pixel shall be blueish. In standard orientation hinting is only done vertically while a 90 degree rotated screen will do hinting horizontally.

vsync that one gets complex particular with gsync and freesync.

We also see example where the wayland compositor lies to applications.

https://github.com/Plagman/gamescope

Yes a real time animation want to know the monitor refresh rate is this in fact true??? Valve has found many games give them 200hz+ monitor refresh rate that is the real monitor refresh rate and the application just dies because they were not designed in a time frame where 200hz monitors existed. Yes you can have the reverse problem.

So do you really need to know the monitor refresh rate. Or do you just need to know when your output was displayed. Video streaming can be done quite well without knowing the exact monitor refresh rate.

Something else to be aware of just because you have a 200hz monitor does not mean you have a GPU able to process to produce 200hz either. So telling application the truth on monitor refresh rate can be the cause of the application running itself out of GPU and CPU resources.

No objection to the need of lying for legacy software. We are going to convert legacy software from SDR bitmap to HDR bitmap in compositor side for HDR monitors already.

Multi head for crash resistance the application can only have information from the compositors/compositors it is connected to. The reality here the compositor is to let the application know as much information as the application really need to know about the monitor. Let the application know information it does not need to know can make application miss behave. So tell application that its has 200hz monitor when the application is perfectly happy with 50hz can result in the application not performing right.

The split from application and monitor by the compositor is no mistake with wayland. Letting the compositor be the gate keeper on what information applications know cure a lot of problems. Big mistake in your logic you never allowed for the problem where client application cannot adapt to the monitor profile information. Yes modern gaming and workstation monitors with legacy applications this is quite a common problem where the application cannot adapt to the new monitor.

These doesn't change the need of compositor telling client applications (A) subpixel color order / orientation and (B) the "DPI" or more accurately the intended scaling factor. Legacy software that can't handle non-standard values of them can fallback to compatibility mode, but they are essential information for proper text rendering and shall not be gate-kept by default. In contrast, software that are mixed monitor ready shall be provided these information for each monitors, and let them paint multiple bitmaps for each individual output.

A software would be able to declare itself mixed monitor ready / hiDPI ready / high framerate ready / HDR ready etc. The capacity can be a mix-and-match. HiDPI capacity can be separated into integer scaling / stepwise fractional scaling / arbitrary fractional scaling. HDR / WCG capacity can be separated into fully colour managed / dumping HDR frames for compositor to convert.

I am not against hiding information from legacy incompatible software. Sorry if that was not clear.