Wayland Protocols 1.38 Brings System Bell, FIFO & Commit Timing Protocols

[MrCooper]

But, Wayland overall is still quite badly designed regarding timing and latency issues, and these additions won't be enough to provide full support for low-latency scenarios.

You'd have to be more specific, in general Wayland supports same latency as or better than Xorg though.

[...] any Gnome rep can't NACK protocols just because they don't like them,

That has always been the case, no such thing has happened.

If you look at the whole Gnome dynamic triple buffering MR.. then this dude named "Michel Dänzer" is always critizizing everything.

Mind the selection bias: Reviewing a proposal is mainly about raising issues and discussing possible solutions. It doesn't mean everything's bad, there's just no need to list everything good in the same way. Everything which isn't raised is assumed good or at least acceptable.

I mean shit always stays unresolved for so fucking long.
Almost like they purposely don't want shit to progress.

This is a common misconception: Raising issues of a proposal and discussing possible solutions is moving the proposal forward. If the proposal just sits there with no activity, it'll never get merged.

Anyway, I was pushing for merging the triple buffering MR for the 47 release. In the end it was Daniel himself who decided against it. Moving a proposal forward is first and foremost the responsibility of the proposer. I sincerely hope Daniel will be able to get it over the line for 48.

I know it's complicated and I'm in no shape or form educated on how to implement a certain feature into some Wayland compositor.

And yet you feel entitled to criticize those of us who are.

If the GBM spec doesn't say anything about synchronization you can be like:

The documentation for gbm_surface_lock_front_buffer doesn't say anything about synchronously waiting for the GPU to finish because it's not supposed to do that.

1. Okay NVIDIA does it differently, lets take it into account and implement it so it works for both Mesa and NVIDIA.

2. Fuck NVIDIA, if is doesn't behave like Mesa then just flush it down the toilet.

If I was a driver vendor entering an established ecosystem, I'd first of all try to make my driver work as well as possible without any changes to the ecosystem, allowing my users to use the whole ecosystem right away. Only then would I suggest (backwards compatible) changes to the ecosystem to address any pain points for my driver. It's a somewhat similar concept as getting invited to somebody else's place.

The current mutter triple buffering patch is a hack. The idea is to get weak Intel GPUs to ramp up their clock speed to maintain the max framerate. If performance is sub-refresh rate, when it switches to triple-buffering it tells the driver to draw two frames at once, briefly doubling the workload. This will raise the clock speed, but it's not sustainable. Once the swap queue is full, it returns to drawing one frame again and clocks drops off, creating a microstutter cycle. The more obvious thing to do would be to directly, not indirectly, instruct the GPU to raise the clock speed.

The rationale about clocks was always dubious, there are other unavoidable reasons why triple buffering is needed in some cases though. Similarly, while the implementation wasn't great early on, it has improved a lot.

The problem vanvugt is having with NVIDIA seems to be that when turning on explicit sync support, it absolutely refuses to queue extra frames, and instead blocks.

It's not about explicit sync but about the nvidia driver handling implicit synchronization in an awkward way (in gbm_surface_lock_front_buffer, making it behave like glFinish), which blocks the calling compositor thread (not only with triple buffering, in fact it limits the extent to which the GPU can run ahead of the CPU with triple buffering). Upstream drivers handle this asynchronously between the GL & kernel drivers. In https://gitlab.gnome.org/GNOME/mutte...1#note_2236184 I described a straightforward way to do the same in the nvidia driver.

[access]

2. The macOS concept of window management is overly simplistic; in my opinion, it is targeted to complete noobs who don't have a good understanding of classical window management. It doesn't even support control by keyboard (without mouse).
Then, less development effort is given to other desktops, because Gnome Shell absorbs most of it (in an unfair way, just because it is the default desktop).

A bit amusing to call an OS that can trace its roots back to one of the first GUIs to not have classical window management. And you're wrong about macOS not having keyboard shortcuts for window management.

[access]

GNOME has keyboard shortcuts for window management too.

[Daktyl198]

That has always been the case, no such thing has happened.

It has happened, which is the entire reason the Valve developer feels the need to specify the rules surrounding NACKs.

Yes, I wasn't very specific.
Xorg is not a good comparison point, because it is an 80's design that never concentrated on low latency.

To the latency specifics:

1. The compositor should ANNOUNCE future frames (i.e. the deadlines before frame buffer is flipped), to each client separately, so that clients can synchronize their output properly. Those deadlines can be no more than estimates, nevertheless they are crucial. At least two frame-flips must be announced in advance (but 10 would be better). The compositor is allowed to modify the timing estimates at any moment.

2. The compositor must provide to the clients the timing information about current and past frame flips. For example, did the previous frame arrive too late from the client? More precisely, the compositor communicates the time difference (how much excess time or a lack of time was there when compositing the last composited frame).

This sounds dumb, tbh. Compositing a desktop takes VERY little time, so little that I couldn't ever see Wayland's design ever having an issue with getting clients to send a correct reference before the frame cutoff. I run my KDE desktop (which arguably does FAR more compositing with way more bells and whistles than any other Linux desktop environment) at 240hz and have never once noticed input or frame latency on the desktop. Maybe at 480hz+ you might start running into issues... maybe. And even then, Wayland has things in place to handle it, it's not like it just panics.

The only applications I could see running into issues hitting frame times are 3D applications like games, almost all of which are run fullscreen and bypassing the compositor entirely.

Not saying it shouldn't be added... but I'm saying I doubt it would do much of any good unless you're on extremely ancient hardware trying to run a riced up desktop.

[Daktyl198]

If the client can determine that it cannot render the frame before the deadline, then it should skip the current frame and start working on the next frame immediately. It shouldn't waste time on frames that are not going to be displayed at the correct moment. Low latency is a simple as that.

Which is a fine idea in theory... but in reality, clients (applications) render frames so fast that it never comes up. I run KDE on an ancient AMD mobile CPU and there is zero compositing delay or stuttering. In the worst case scenario, a client will render a frame slightly too slow and Wayland will simply discard that frame and wait for the next, but I've never seen it happen in my couple years of running Wayland. And I do notice stuttering and input lag. I'm quite sensitive to it which is why I'm even running Wayland at all. X.org introduced extremely noticeable latency when running my display at 144hz (let alone 240hz) to the point where even though the "FPS" was 144 it was unusable.

I'm not sure exactly what's causing the stuttering on your display, but I don't think it'll be solved by adding in the feature you are talking about lol. Additionally, traditional applications do not bypass compositing in full-screen because "compositor fullscreen" and "actual fullscreen" are two different things so going fullscreen in a regular application wouldn't help either.

The one place I can see this being useful is nested compositors, but that usecase is already extremely niche. Still maybe worth doing, but not high on the priority list. As for "hard to augment the wayland protocol" it's really not. You can design a wayland protocol right now and submit it for review. Having the compositor send the clients timing signals would hardly be that crazy of an addition. Why not plead your case on the wayland protocol gitlab instead of on the Phoronix forums?

[MrCooper]

1. The compositor should ANNOUNCE future frames (i.e. the deadlines before frame buffer is flipped), to each client separately, so that clients can synchronize their output properly.

wayland-protocols!276 is a proposal for this.

At least two frame-flips must be announced in advance (but 10 would be better).

That makes little sense. With fixed refresh rate, the client can just extrapolate itself (the presentation time protocol provides the duration of a refresh cycle). With VRR, it's not really possible to predict anything but the earliest and latest possible time for the next refresh cycle to start.

2. The compositor must provide to the clients the timing information about current and past frame flips.

That's what the presentation time protocol is intended for.

It has happened, [...]

Where? (Looking for direct evidence, not hearsay)

This sounds dumb, tbh. Compositing a desktop takes VERY little time, so little that I couldn't ever see Wayland's design ever having an issue with getting clients to send a correct reference before the frame cutoff. I run my KDE desktop (which arguably does FAR more compositing with way more bells and whistles than any other Linux desktop environment) at 240hz and have never once noticed input or frame latency on the desktop. Maybe at 480hz+ you might start running into issues... maybe.

The higher the refresh rate, the less of an issue latency becomes, since a refresh cycle is so short that missing one isn't a big deal. What Uiop mentioned is rather important for lower refresh rates.

The only applications I could see running into issues hitting frame times are 3D applications like games, almost all of which are run fullscreen and bypassing the compositor entirely.

There's no such thing as bypassing the compositor, it's always actively involved in presenting client frames, so it incurs non-0 latency (though it can be lower with direct scanout of client frames).

I thought Wayland wants "perfect" frames. However it is fine if those frames are so perfectly perfect that they are displayed at the wrong time.

"Every frame is perfect" isn't about timing or never missing a display refresh cycle (which isn't possible to guarantee, not even by pessimizing latency). It's about allowing the client to ensure its window is always presented in a fully consistent state, never in an inconsistent mix of intermediate states.

[billyswong]

About copying macOS desktop: I personally don't like macOS desktop or Gnome Shell, but I guess there is about 20% of people who do (i.e. the users of macOS), so a similar desktop working atop of GNU/Linux is a good way to attract those users.
Other than that, I don't see how is GNOME copying macOS?

About HiDPI and the link you have posted: My display is set to "slight subpixel antialiasing", because I use something in-between HiDPI and normal. Given the current desktop resolution sizes, I would guess that most users are similar to me here, with a wide variation. So, it is impossible to have one default setting which fits everyone, and, naturally, there are going to be fights about the default setting for subpixel antialiasing.
Therefore, I can't take the linked discussion as an evidence of anything extra-ordinary.

The link provided by Weasel is about grayscale AA being the default. Unfortunately the GNOME developers have pushed forward their arrogance to the point of removing subpixel AA in GTK4. You can no longer choose subpixel AA even if you know what it is and prefer so.

I agree that with high enough DPI, subpixel AA can retire. But the market share is far from that in 2024. No, the market share won't be there yet even we fast forward to 2034. It takes pixel density of 250%+ (of the logical 96dpi) for subpixel AA stop being useful. Most computer monitors are between 100% and 200%. Monitors in classical DPI range are going to stay for perhaps another 10 to 20 years.

Because the decision is made in GTK, they are forcing their aesthetics beyond the GNOME DE users. Oh yeah, those arrogant fanboys will call for abolishment of "hobby" DEs. I can hear they coming.

[MrCooper]

I'm strongly against such assumptions about what clients should "guess".
In a good protocol, the client says what it wants to the compositor, and the compositor responds with information requested by the client.
Deviating from such principles eventually results in a bad protocol.

So, from my point of view, client requests estimations of future frame times, and then gets those estimates from the compositor.

The compositor doesn't have more information than the refresh rate / duration of a refresh cycle, all it can do is extrapolate, same as the client. Passing 10 extrapolated values in the protocol doesn't give the client any more information than the duration of a refresh cycle, which is already available. Passing redundant information like that in the protocol is wasteful.

You should try to de-couple the protocol from your wild guesses about what the user's hardware can do.

It's not a wild guess, it's how today's displays work.

Experience shows that trying to anticipate the future in a protocol (or API) tends to be a mistake, because things tend to evolve differently than we expect. A good protocol works well for the real world at present and is extensible for future needs.

Anyway, if that "presentation protocol" is done right, it should also be able to solve some of Wayland's problems with double buffering and triple buffering also.

There are no such problems I'm aware of. A Wayland client can use as many buffers as it likes.

[billyswong]

Uiop This GTK blog post should be authentic enough for your need

https://blog.gtk.org/2024/03/07/on-fractional-scales-fonts-and-hinting/

[billyswong]

Also see this closed ticket https://gitlab.gnome.org/GNOME/gtk/-/issues/3787