KDE XWayland Video Bridge Aims To Improve Linux Desktop Screen Sharing

[Weasel]

Other than that is broken as hell for xdg desktop portals providing file and print dialogs and other things like it. xcomposite does not deal with multi process applications.

"Specific applications" does not mean "specific windows". You have 3 cases need to be address.
1) full screen.
2) specific windows(Xcomposite gets you this far but with audio sync issues) Video capture in the sound server has some serous advantages.
3) specific applications
Yes number 3 you have to deal with multi process applications and work out that these different windows generated by different processes with different PID numbers are in fact the same application and all those windows you want to capture.

The fix to this correctly is quite major changes. Notice KDE system manager tracks applications by the session manager using cgroups this handles multi process applications correctly.

How to go from a socket filehandle to what cgroup it is without doing anything race condition that you have to counter still need Linux kernel work.

Never had any issues with Xcomposite, other than a bug with swapped RGB→BGR which can be easily worked around on (OBS even has a flip for it). Yes, it's specific windows only, not "applications".

That's what you want in 99% of cases. Either that, or grabbing the entire screen of course, which again X11 can do, but it's better to just grab the framebuffer and avoid round-trips to X (this bypasses Wayland's "security" too btw).

How do you even record "specific applications" windows? What happens if they're not in a rectangular shape because they're moved in weird shapes on top of each other? What's visible "behind" them? Black? I legit never seen anyone ever "record an application" this way, it's either a window, or the entire screen. But fair enough.

I guess you could do that on X11 by just querying application windows and then grabbing the windows themselves, taking care to composite them yourself. I don't think anyone really does that though.

[oiaohm]

Never had any issues with Xcomposite, other than a bug with swapped RGB→BGR which can be easily worked around on (OBS even has a flip for it). Yes, it's specific windows only, not "applications".

There are particular issues that you are not thinking of yet.

That's what you want in 99% of cases. Either that, or grabbing the entire screen of course, which again X11 can do, but it's better to just grab the framebuffer and avoid round-trips to X (this bypasses Wayland's "security" too btw).

This is kind of where it starts coming questionable todo it in the display server/compositor. Think with HDMI/Displayport out the sound output matching the applications actions can also be going to the video card so capturing at the video card you can capture the synced audio with sync video output and use the GPU encoder at the same time. Display interfaces have evolved a lot from the time X11 protocol display capture was developed. This could evolve even more into the future.

[QUOTE=Weasel;n1379893THow do you even record "specific applications" windows? What happens if they're not in a rectangular shape because they're moved in weird shapes on top of each other? What's visible "behind" them? Black? I legit never seen anyone ever "record an application" this way, it's either a window, or the entire screen. But fair enough.[/QUOTE]

Think about when you do screen capture with Xwayland without this XWayland Video Bridge. You see all the X11 applications alone on a full screen canvas.

Specific application capture. is the same kind of thing. This is not specific windows.

I guess you could do that on X11 by just querying application windows and then grabbing the windows themselves, taking care to composite them yourself. I don't think anyone really does that though.

X11 has killed your means todo this in the protocol correctyl. You do not get filehandles of any form so you cannot be sure if the Window information about what application created them is still current. Yes storing raw PID values not a great idea if you don't want race conditions. xace did attempt to fix this problem but when we use X11 server we don't use Xace in most cases because it breaks too many applications and the extra information is normally hidden by the xace system from the software you would want doing the capture with xace anyhow.( yes snoop be terminated by the MAC system).

To make specific application capture work with X11 without race conditions in fact requires a X11 protocol modification.

Yes part of doing specific application capture is custom compositor for the capture. Other part is having the means to correctly identify what parts should be in the custom composite to capture a particular application output. Of course this is going to need to include the audio coming out the application.

specific application capture
1) something like cgroups around the application so you can do this.
2) race free way to id what windows/buffers need to be in the per application composite.
3) be able to run a capture composite.
Remember users will want to start specific application capture after they have started the application so you cannot get away with a proxy compositor/ xnest or x11 server todo this.

Specific application capture think you are doing a bug report with a issue with libreoffice you don't want to be capturing your email/socal networking ... as well to add to a bug report. Issue reporting to those processing the bug reports can see what went wrong specific application capture could be good. Doing a business meeting specific application capture could also be useful particularly if you have todo a usage demo....

Specific application capture is is own level of migraine headache above and beyond specific window capture. Specific application capture one of those features people ask for serous-ally and people commonly mix it up with specific windows capture. Weasel I don't fault you for this mistake but you do need to remember when a person asks for specific application capture that they in a lot of cases are not asking for Specific Window capture. Yes it a way harder beast to pull off.