When Time Comes For GTK5, It Might Be Wayland-Only On Linux

[rogerx]

From what I last witnessed, XWayland still lacks many of X11/Xorg features. And more so for forcing users into buying newer video cards.

Gnome developers seem to be only irritating longtime users of GTK, and are sounding more and more like Microsoft employees.

[binarybanana]

From what I last witnessed, XWayland still lacks many of X11/Xorg features. And more so for forcing users into buying newer video cards.

Gnome developers seem to be only irritating longtime users of GTK, and are sounding more and more like Microsoft employees.

It's called "rootful(l) Xwayland". It'll let you run a full X11 desktop/WM on Wayland. Honestly, seems a bit like admitting defeat. We've come full circle now.

[oiaohm]

The discussion thread is originally about GTK5 so I thought we are talking about what can a GUI toolkit such as GTK do if the problem of fractional scaling can be pushed back to toolkit level such as GTK. When I suggested 1/120 or maybe 1/128, 1/256 etc as a higher resolution logical pixel unit, I am thinking in terms of letting toolkit such as GTK convert coordinates of UI components only once per scaling factor, then reuse those coordinates in the application ever since. And I expect such extra set of coordinates to be calculated via integer math.

This kind of comes part 120 divides by 1,2,3 and 4 then does not divide by 5.. Yes this is why it pays to read what has already been tried. 1/128 and 1/256 and 1/300 ..... all have been tried to try to get the conversion in toolkit to be able to be done by integer maths. Bad news is this has failed because at times you will run into a scaling fact that does not exactly right leading to rounding issue. This is the same problem that use to happen with font rendering and html5 both of these solve this by basically image cache fonts they generate a gypth image and use that over and over again and web browsers image cache the render. This remove doing the maths multi times. Maths multi times gives you risk of doing rounding errors different each time.

When I wrote "the blurry stage", I were referring to what happened when an Windows application that aren't monitor-aware be dragged from screen DPI A to screen DPI B region. Or what happened when an Linux or Mac application resized by compositor-based fractional scaling. In those cases, we just resize the image buffer as-is and don't care the content within. GPUs succeed in doing that consistently unless you tell me they are done by CPU in current Linux and Mac.

The reality here the GPU does not succeed at doing a image buffer scaling at fractional consistently they have some blur to hide the error. If you reduce the contrast where error happens human does not end up on floor from the error. Of course then some people complain the game is not as sharp and they are right. The art here is having enough blur run to run not to have human delectable flicker problem(person having Seizure because of it)while not having too much blur the user hates. Or the other option is image caching/math caching so you limit your maths passes so limit you rounding error issues so not having the flicker problem from maths error.

Windows example with DPI is not that useful when you think steamdeck or docked phone next to a monitor and to the human you want them to see readable text.

But honestly, I really didn't think (and still don't think) alpha blending window surfaces is that hard for GPU. The Aero visual effect in Windows Vista and Windows 7 is already over a decade ago. All GPUs now can do alpha blending right? Right?

There is a problem here the laws we have to obey today for a desktop OS are not the same as when windows Vista and Windows 7 released. Please note Windows 11 does transparency different to Vista and 7.

https://mspoweruser.com/microsoft-says-windows-11s-mica-transparency-effect-and-rounded-corners-will-not-affect-performance/

Mica, the semi-transparent effect which lets you see the background of your app through the app, “it only blurs the (background) image once” rather than with every video frame, to offer better performance and experience than Acrylic.

Microsoft does not write here and is legal because we don't have math errors different run to run in the blur any more causing flicker. Yes you use to get away with having flicker causing features as long as you had a off switch for that feature. That does not stand up on court.

Yes the Aero transparency effect is code name Acrylic and by current day laws if it was being released new today its illegal in many countries. The design requirements from law are getting stricter. To be correct Windows 8 and 8.1 and 10 Microsoft claimed Acrylic under grandfathering rules that it existed the law. Of course when they wanted to alter/improve their transparency that protection from the law went away.

The laws around flicker to make sure you don't effect that 1 in 4000 person has progressively coming into force over the years. This is also why you have to be very careful saying odd software did this so we can do the same lot of cases this will be wrong. Problem is old software can either be grandfathering around the law or was prior to the law and if you do that today in new software you have just made something illegal. Remember this is something that can cause a person physical harm that in theory could lead to a criminal charges for negligence. Yes the legal minefield of making graphical software.

This is why doing this is not easy. Yes Wayland starts after the anti-flicker laws had been passed.

[billyswong]

Hmm, 120 is divisible by 5. And we are doing the coordinate math only once per scaling factor here. The UI toolkit don't need to recalculate them per frame. Only once per change of scaling factor. So I am unsure why you stress the issue of rounding being unrepeatable per frame, even after I stressed I am suggesting using integer / fixed point math for UI coordinate scaling calculation. Fixed point multiplications / divisions truncate the same every time. I have also showed the coordinates in 1/256 or 1/120 px unit can fit in 23-bit nicely, leading to efficient GPU implementation. If we use the slightly less efficient 32-bit integer in GPU, the screen space is logically over a mile long. Throughout 21st century, there should be zero demand to compute screen coordinates in 64-bit integers.

So, why is here the never-ending worry of inconsistent rounding error? It is only calculated once! It is cached! It is fixed point math! There should be zero flicker in sizing and positioning! The existing refusal I've seen in GTK / Wayland are all about they don't like the rounding result, never about they can't get a consistent rounding result. Or if there are talk of inconsistency, it is inconsistency of tile size if they want tiles without gap, not inconsistency between frames. And the worry of tile gap lead to my suggestion of alpha-blending.

About Win7 Aero being illegal by today's law... maybe we shall try some temporal anti-aliasing here. If an application let the compositor cache its frame buffer, TAA alpha-blending. If a particular surface is "high-security" (e.g. paranoid to pirates thus DRM'd), blurry alpha-blending or no alpha-blending. l consider this a good enough tradeoff. Let paranoid applications get gaps between tiles.

[oiaohm]

Hmm, 120 is divisible by 5. And we are doing the coordinate math only once per scaling factor here.

I typed 0 instead of 8 as in 128/256 is not divisible by 5 and 120 is not divisible by 7. There is a reason why a party is looking at being able to say to application you know that window you gave me that 800x600 I need it 960x720 with everything look close to the same.

Yes the important thing we have to get to coordinate maths only once per scaling factor. Problem is some applications like to dynamically change interfaces. Also some of programmers don't like caching values. They are trying to make there program as light on memory as possible.

About Win7 Aero being illegal by today's law... maybe we shall try some temporal anti-aliasing here. If an application let the compositor cache its frame buffer, TAA alpha-blending. If a particular surface is "high-security" (e.g. paranoid to pirates thus DRM'd), blurry alpha-blending or no alpha-blending. l consider this a good enough tradeoff. Let paranoid applications get gaps between tiles.

Win7 Aero was grandfathered from Windows Vista. So for anyone todo it that way when windows 7 released as a new project it was basically already illegal.

Temporal anti-aliasing does not work CPU side at this stage too much processing. FSR has only recently been released with generic GPU Temporal anti-aliasing.

Please note Temporal anti-aliasing is not no blur it less blur and this is because you are able to look back frames and pick up when you are going to cause flicker and avoid it. No flicker without temporal feature leaded to higher amount of blur required but with a lower processing cost. Temporal requires more processing power to have less blur. No free lunch here this is why user choice may be required here.

Some people have noticed with Windows 11 that when using transparency items that should have animated movement in background are frozen solid. Like the clock on the wall paper you are seeing though the application to see no longer ticks where it would under prior versions of windows.

I do see wayland compositors could use Temporal anti-aliasing to provide bit sharper output while still being legal of course with the cost you have to have enough GPU power for it(CPU power not going to cut it at this stage).

Fractional scaling with wayland end up in the too hard basket of problems to get right for a decade and bit makes sense once you know the laws you have to play by and that what was being suggested was not working. This year we finally got access to some features that may allow this problem to be finally solved properly.

Please note too hard basket in this case the problem is put in there people look at it attempt a solution does not work and put it back repeatedly. But to the anyone not watching the basket/issue list it looks like no one is attempting to fix the problem even that many people have been attempting and failing.

Yes as you said for security for Wayland compositor to be able to perform Temporal anti-aliasing is going to need to be able to ask the application is this window safe to be cached. Caching password dialogs or other things might be bad. This also means for legacy applications users will need to be asked is it fine to scale this application because the application will not have the feature to answer the compositor if it can be scaled this way or not.

Integer scaling is simple to perform in CPU that another thing to consider. Processing cost of the different scaling methods. Amount of blur in fractional scaling to avoid flicker does link to processing cost more processing cost lower blur less processing cost more blur. Lowest blur possible with fractional scaling with rounding errors causing flicker/shimmer. This is a trade off.

I do agree fractional scaling in wayland could be better. The reality is being legal even a few years ago the methods to do much better really did not exist. Remember X11 due to be a legacy thing gets away with using horrible methods that are not legally acceptable today in a new solution as well.

Yes this has caused hell the law changed after vista with grandfathering clause that if you were doing X prior to the law change the law did not apply to you. You want to create something new law applies. Of course those developing Wayland when it comes to scaling the law does apply. Yes do this like X11 does or do this like Windows 7/vista does are both not option for Wayland. Scaling one of those curse areas must redo new due to law change hitting the early Wayland developers then having the hell no one else had so no existing solutions to base off.

[billyswong]

I used "high-security" with a pair of quotation marks because we would want almost every window surface to be "cacheable", yes even for password dialog, for example accessibility tool magnifying the password dialog for users to see. The only images that can't be cached even with root privilege is those UHD Netflix stuff, protected from their server up till our monitors (HDCP). Existing mechanism is enough to distinguish copyright-paranoid content and standard root-account accessible content.

People would be a lot more forgiving if they said they are dealing with new anti-flicker legal requirements and be open to any future technology advances that can fix their current problem. Also, TAA or standard blur is only for applications / UI toolkits that refuse to cache dynamically floating-point math generated coordinates. On one hand, the major GUI toolkit that refused doing native fractional scaling so far is GTK, a retained mode GUI library that does cache coordinates. On the other hand, an immediate mode UI toolkit can generate flicker even without calculating coordinates dynamically via floating point math, as they may lack guarantee that every frames are drawn in time completely. Flickers are awaiting if they don't do double or triple buffering. And then there is also fringe stuff such as the stupid toy program I wrote at school.

A allow/denylist approach should be applied to applications and UI toolkits. TAA or standard blur will be applied to questionable programs, and the setting will be adjustable manually per application.

[oiaohm]

I used "high-security" with a pair of quotation marks because we would want almost every window surface to be "cacheable", yes even for password dialog, for example accessibility tool magnifying the password dialog for users to see. The only images that can't be cached even with root privilege is those UHD Netflix stuff, protected from their server up till our monitors (HDCP). Existing mechanism is enough to distinguish copyright-paranoid content and standard root-account accessible content.

People would be a lot more forgiving if they said they are dealing with new anti-flicker legal requirements and be open to any future technology advances that can fix their current problem. Also, TAA or standard blur is only for applications / UI toolkits that refuse to cache dynamically floating-point math generated coordinates. On one hand, the major GUI toolkit that refused doing native fractional scaling so far is GTK, a retained mode GUI library that does cache coordinates. On the other hand, an immediate mode UI toolkit can generate flicker even without calculating coordinates dynamically via floating point math, as they may lack guarantee that every frames are drawn in time completely. Flickers are awaiting if they don't do double or triple buffering. And then there is also fringe stuff such as the stupid toy program I wrote at school.

A allow/denylist approach should be applied to applications and UI toolkits. TAA or standard blur will be applied to questionable programs, and the setting will be adjustable manually per application.

https://zamundaaa.github.io/wayland/...n-wayland.html Wayland was designed without immediate mode. Turns out in performance freesync with complete buffers and immediate mode are fairly close to each other in performance..

Remember immediate mode comes from the time frame when did not have freesync or gsync monitors or the newer legal requirements.

billyswong this is the problem things that you use to be able do are not allowed in default any more. Immediate mode for new software is really only allowed for games with flicker warning if you are aiming to be legal.

I do agree user overrides and configuration will be required. There will always be that application out there that says it scale itself but then does not do it correctly so need to be blacklisted from being asked to do fractional scaling. There will will be the legacy applications that need scaling for hidpi monitors.

This has been very hard. Different developers in the wayland mailing list and in issues with scaling have stated the legal problems. But the legal problem has not made it to general users.

People have not notice that games and consoles have been needing to put a flicker warning on them since 1991 and that regulations for flicker appeared in start 2008 for general usage yes just after vista and just before Wayland. Yes Wayland is end of 2008. This is what you call bad timing. If wayland had started a 2007 might have been able to use the grandfather clauses.

More and more parties are pushing that games don't have exception.

Really immediate mode we really do need to take a serous look at this. Is the 1 in 4000 risk of problem to user vs slight performance gains of immediate mode over freesync worth it. If we be real if you are not a competitive class gamer the advantage of immediate mode is not greater than the higher risk you take from the tearing flicker that immediate mode can cause. The big problem with the 1 in 4000 risk here is you don't know you are in the 1 in 4000 until after you have been adversely effected in a lot of cases. Competitive gamer can do their immediate mode under medical supervision.

[billyswong]

oiaohm we are using different meaning of "immediate mode" here. Any programs that don't store and reuse rendered UI in buffer are immediate mode UI as they have to redraw itself per frame. Retained mode UI don't redraw itself per frame. It only do partial updates upon user interaction or scheduled animation. Retained mode GUI applications are by definition no risk of coordinate flicker due to fractional scaling. Meanwhile most 3D games are running immediate mode UI and X / Wayland have nothing to do with whether their UIs are immediate mode or retained mode.

A compositor has no legal obligation to babysit flickering applications so that they don't flicker and yet still stay performant. A compositor also has no obligation to babysit every applications to be not flickering as long as they declare themselves "safe" (and nobody report they are lying). Let's apply TAA to everything in doubt while allowing users to force-enable / disable the TAA in per program manner. I think this is a better policy than the current one - right now Wayland is assuming applications are always flicker-free before scaling and always flicker-prone after fractional scaling, which is far from reality.

[oiaohm]

oiaohm we are using different meaning of "immediate mode" here. Any programs that don't store and reuse rendered UI in buffer are immediate mode UI as they have to redraw itself per frame. Retained mode UI don't redraw itself per frame. It only do partial updates upon user interaction or scheduled animation. Retained mode GUI applications are by definition no risk of coordinate flicker due to fractional scaling. Meanwhile most 3D games are running immediate mode UI and X / Wayland have nothing to do with whether their UIs are immediate mode or retained mode..

I was thinking X11 define of "immediate mode". Yes the x11 define is a lot more savage version immediate mode. Like application has rendered half a frame and X11 server goes o well I am displaying that anyhow yes this is kind of the worst of the worst. So "X11 immediate mode" has tearing and flickering coming for the way the X11 server and drivers is handling the buffers provided by the application. You also find this form of horrible immediate mode back in Windows 9x(95...).

Yes under X11 using "immediate mode UI toolkits" with X11 server in "X11 immediate mode" equals mess. Retained mode UI can also run into hell with X11 server in "X11 immediate mode" for the same issue.

You do kind of want at least buffer fences that X11 "immediate mode" does not have. Buffer fence basically means when you are done putting what you want in the buffer you say I am done here compositor you can display this.

The X11 server immediate mode comes about from being stuck at 30 hz or lower time frame of monitors where the next time you can render a frame is ages away and the trade off was to tear/flicker to get a faster display of information to user. We mostly don't need this horrible solution any more.

Yes we were using different terms here. The "X11 immediate mode" type of immediate mode need to disappear for most users. Immediate mode UI still when it misses the vsync(and not using freesync) to avoid displaying a part buffer there need to be retained copy of prior buffer in video card/compositor/server.

Retained mode doing partial updates with "X11 immediate mode" can get badly flickerly as well. Think about it you start updating buffer for next frame and are only partly complete and the X11 server displays this one half way though updating then it does it on the next frame...

The "X11 immediate mode" is fast but its multi levels of cursed. You can think of "X11 immediate mode" as possible Wayland compositor/X11 server feature. Please note you don't have X11 immediate mode if you have a X11 compositor loaded into X11 server.

A compositor has no legal obligation to babysit flickering applications so that they don't flicker and yet still stay performant. A compositor also has no obligation to babysit every applications to be not flickering as long as they declare themselves "safe" (and nobody report they are lying). Let's apply TAA to everything in doubt while allowing users to force-enable / disable the TAA in per program manner. I think this is a better policy than the current one - right now Wayland is assuming applications are always flicker-free before scaling and always flicker-prone after fractional scaling, which is far from reality.

When it comes to workplace health and safety in different enterprise settings the compositor can be need to babysit applications. That the developers doing the base of Wayland are funded by enterprise this is also a problem. You have the legal requirements and there are many of them. This is a case I think general home user and enterprise are out of alignment on what is tolerable as well.

[Luke]

That means by the end of this decade, Mate and Xfce must have a Wayland session. I'm fairly sure these projects don't have developers capable of developing their own Wayland compositors from their existing window managers (even if they have, it will take many years until they have a mature Wayland session), so like Alexmitter said, I think the most practical approach is simply adopting an existing Wayland compositor for their Wayland session. I think Mutter is the best choice because it's the most mature one and will continue to be so in the near future. I heard Xfce still haven't even begun working on Wayland support, but most of Mate's apps already support Wayland natively, so I don't think creating a Wayland session on an existing compositor will take too long.

I am working now on creating a usable MATE-wayland session, we now have an almost feature-complete mate-panel and mate-applets on WLRoots-based wayland compositors. We have icons on the desktop in Caja plus navigations windows, all running under wayland rather than via Xwayland. Any wlroots-based compositor witrh foreign-toplevel support in use should be usable. You need foreign toplevel for the window list, and gtk-layer-shell to keep the desktop window on the bottom, menus and volume sliders etc on top, and position all those panel popup windows where they belong.

At a minimum we will get a solid shell for a variety of wayland compositors. Try mate-panel from git master on wayfire, plus caja from git master. It will look so much like your normal MATE session with compiz. Oh yeah-wayfire is much less likely to crash than compiz, though if you do find a way to crash it it will take out your whole session. Research is underway at KDE to keep the client sockets alive while a crashed compositor restarts.

We have no plans I've ever seen to port to GTK4 much less GTK5. If GTK3 ever goes away we must then choose between forking GTK3 or writing an extension library for GTK4 (or 5, or whatever). Note that most traditional GTK2/3 widgets can be remade from newer widgets, such as using GtkBox to construct a menubar that once created can be used over and over again. GNOME devs directly suggest doing exactly that when such widgets are needed for GTK4. If that day ever comes, a rational choice will have to be made based on amount of effort to get good, reliable code. Note that either a "libmate-gtk-extensions" library or a GTK3 fork would be a lot easier to get distros to handle in say, 2030 than updating a bitrotted X11 that won't run on bare metal anymore. Native wayland is all but certain to outperform running a DE in "rootful Xwayland" as well as make any wayland-only app (e.g. a future gtk5 GUI for MPV) much easier to work with.