xf86-video-modesetting TearFree Gets Fixed Up For A/V De-Synchronization Issue

[phoronix]

Phoronix: xf86-video-modesetting TearFree Gets Fixed Up For A/V De-Synchronization Issue

A few months back the generic xf86-video-modesetting X.Org driver added TearFree page-flipping support. The option eliminates screen tearing without the use of a compositor and was seen as a win by many for this generic DDX driver that works atop the modern DRM/KMS kernel drivers. But a rather annoying issue was discovered that could lead to audio/video synchronization problems was uncovered and is now fixed in the latest driver code...

https://www.phoronix.com/news/xf86-video-modesetting-Fix-TF

[fagnerln]

So now this is more or less what Wayland do? I still need to try for myself, I want to buy some high frequency (180/240hz) monitor later this year. What's the point in using freesync on wayland in a monitor with so high response?

[acobar]

Nice! I'm still waiting for KDE to improve a bit more under Wayland, and improvements on X11 are welcome while I wait.

I guess, the *BSD will also benefit from this.

[kpedersen]

Wasn't this quite easy to implement?

I did similar for my small gameboy/drm modesetting port here.

vsync and atomic pageflipping are fairly solved issues. And wouldn't audio be unrelated to video? Both will keep to time individually or maintained by the i.e media player. Even if video was disabled, you wouldn't expect audio to drift any more or less.

[kerneltoast]

Wasn't this quite easy to implement?

I did similar for my small gameboy/drm modesetting port here.

vsync and atomic pageflipping are fairly solved issues. And wouldn't audio be unrelated to video? Both will keep to time individually or maintained by the i.e media player. Even if video was disabled, you wouldn't expect audio to drift any more or less.

Sadly no, it's not so easy because you don't get to control the code in applications from X.Org, and there are most often multiple applications using the display at once. In your example, achieving this sort of thing when your application gets the whole display to itself is easy.

Audio synchronization is related to video if your application doesn't know when the video appears on the display. Again, this is easy achieved when your application has the entire display to itself, but not so simple when there are other applications rendering pixels to the display through a centralized display server. What happens when two different applications want to render pixels at almost the same time, and you want to page flip those pixels onto the display? Well, once the page flip is sent to the kernel it can't be changed or stopped, so the application that loses the race is going to have to wait a bit until its pixels can be shown on the display. And in the case of a media player, it needs to know that such a delay occurred, otherwise you'll go out of sync with the audio because the media player will think the video appeared on the screen sooner than it really did.

[treba]

So now this is more or less what Wayland do? ...

No, not at all. It's basically a "put a compositor into the driver" with the respective overhead but without adding any features. You don't get transparency, you don't get synchronization (e.g. between SSD and client content) and you certainly don't get advanced features like hardware planes, YUV support etc. But for users of classic window managers it's probably a nice band-aid for the most obvious and visible issue.

I personally hope that this will make the remaining distros that still ship e.g. the Intel DDX by default stop doing so. Vendor specific DDX drivers have been terribly under-maintained for years, causing annoying bugs in applications. Having a single, vendor-neutral driver increases the chance that at least a certain level of quality will be ensured.

[Monsterovich]

No, not at all. It's basically a "put a compositor into the driver" with the respective overhead but without adding any features. You don't get transparency, you don't get synchronization (e.g. between SSD and client content) and you certainly don't get advanced features like hardware planes, YUV support etc. But for users of classic window managers it's probably a nice band-aid for the most obvious and visible issue.

It's okay to have V-Sync in the driver, because it's possible to configure a system without a compositor at all, or there may be cases that are not covered by the compositor.

[MuPuF]

But for users of classic window managers it's probably a nice band-aid for the most obvious and visible issue.

Yep...

I personally hope that this will make the remaining distros that still ship e.g. the Intel DDX by default stop doing so. Vendor specific DDX drivers have been terribly under-maintained for years, causing annoying bugs in applications. Having a single, vendor-neutral driver increases the chance that at least a certain level of quality will be ensured.

YES! That's the reason why I made the initial merge request so many years ago: https://mupuf.org/blog/2018/09/24/te...ng_new_tricks/

I unfortunately never got to finish it because my employer implemented another solution in i915 instead which was better for its users... Some users kept using this feature, and finally Sultan Alsawaf took over and rewrote the entire thing. At this point, I merely helped review the couple of MRs he landed.

[binarybanana]

No, not at all. It's basically a "put a compositor into the driver" with the respective overhead but without adding any features. You don't get transparency, you don't get synchronization (e.g. between SSD and client content) and you certainly don't get advanced features like hardware planes, YUV support etc. But for users of classic window managers it's probably a nice band-aid for the most obvious and visible issue.

A compositor is not full a replacement for TearFree. At least as implemented no X compositor will fix tearing on multiple screens with mixed refresh rates or when a monitor is rotated. In theory it should be possible but no one has put in the work (requires intelligent framebuffer management/submission for each monitor). Not sure about scaled monitors, maybe same problem. Hardware planes on X are for the most part exposed through Xv but it's the apps that have to opt-in to it. Back in the day it was used for video playback (hence the name). Yes, doing this dynamically at the compositor level for foreground windows is probably better but there's nothing stopping a compositor from doing this on Xorg, in theory.

[kerneltoast]

A compositor is not full a replacement for TearFree. At least as implemented no X compositor will fix tearing on multiple screens with mixed refresh rates or when a monitor is rotated. In theory it should be possible but no one has put in the work (requires intelligent framebuffer management/submission for each monitor). Not sure about scaled monitors, maybe same problem. Hardware planes on X are for the most part exposed through Xv but it's the apps that have to opt-in to it. Back in the day it was used for video playback (hence the name). Yes, doing this dynamically at the compositor level for foreground windows is probably better but there's nothing stopping a compositor from doing this on Xorg, in theory.

XPresent also doesn't have the ability to page-flip per-CRTC frame buffers right now with the modesetting DDX either, so X.Org would need to be updated to make this possible.

I looked at that code a little bit while working on TearFree but it seemed like there weren't any clients out there that actually care to submit per-CRTC buffers through XPresent, so even if X.Org were updated to make this possible, it seems like no applications or compositors could immediately make use of it.