A Call To Move Games Outside Of Linux Desktop Environments, Own Wayland/KMS Setup

[sarmad]

So, if this is implemented, how much of a performance gain do we get? It might not be worth it.

[Petteri]

SteamOS is still X11 based. It doesn't do anything special. AFAIK, its compositor doesn't even unredirect.

I have understood that it does some tricks to integrate games without flickering and modesets to Steam UI and Steam overlay. Unredirecting would be problematic because it could cause flickering when transitioning to games and when using the overlay.

[ssokolow]

I don't think this is a blog where he wants to bring PR really, it's a blog post for gods sake. It's to bring an argument/discussion to the table to actually start thinking about something. If this was a solid solution he would be in the table working on it already without saying a word since it is kind of useful technology to have for Linux users. Let's just try to be optimistic and reason if it's possible, I say optimistic because we would REALLY want Linux Gaming to become a thing... right?

My point in saying it lost him a lot of credibility is that I have a hard time trusting the judgement of someone who comes up with that idea and doesn't realize the weaknesses in it (at least enough to acknowledge them) between starting to type and clicking submit.

It's not a matter of PR, it's a matter of me losing respect in Martin's ability to quickly recognize glaring flaws in solutions. It calls into question his ability to recognize less obvious flaws at all.

[jrch2k8]

The only complications really are external programs running on the original desktop like OBS or Tox. If you switch VTs and halt Kwin and everything on top of it you freeze up the rest of your software. It would also mean in a multi-monitor setup either you run the game across all monitors or you are going to have dead screens.

not really but in wayland you don't need to halt anything just inform it to stop rendering, so win can stop rendering in screen3 where your games is but stay fine in screen 1 and 2, you just need to alt+tab for kwin re route input(may require some magic TM)

[profoundWHALE]

I posted this on that blog, but I'll say it again here:

If a PS4 with a FreeBSD based OS can do it, then I’m sure that Linux could do it.

[zanny]

Would this be too much burden on the devs? Would there be a framework for doing this? It's too bad that companies already spent time and resources porting games to linux... Before wayland.
Anyway, this seems like an awesome idea.

Most of them are using SDL, and a change like this would be at that level. The way something like this would work would be:

User starts a game -> game tells compositor its a "take over the computer" app like a game (or Kodi or Steam Picture Mode) -> compositor like Kwin says "alright" and spins up another wayland VT, reparents the game to that, switches to it, and suspends itself so its not burning system resources.

The only logic there is really notifying the compositor a game is running (which the game binary itself doesn't even need to do, something like Steam could do it for it) so it can generate a new VT and run the game on it rather than itself. SDL would need some updates to handle running EGL right on DRM, plus I'm not sure what setup libinput takes on a Wayland session that might need done, but it could probably include that functionality or spin it off into a minimalist "fullscreen program" Wayland server.

The game itself doesn't need to do anything, though, as long as its not directly calling X functionality. If changes were needed in SDL and the game were shipping against an old version (such as the Steam Runtime) it would need to be updated.

The only complications really are external programs running on the original desktop like OBS or Tox. If you switch VTs and halt Kwin and everything on top of it you freeze up the rest of your software. It would also mean in a multi-monitor setup either you run the game across all monitors or you are going to have dead screens.

[jrch2k8]

i haven't really checked the grimy details about this but i think is possible to still keep switching between games and desktop with this idea just informing the compositors and client the output is inactive(so they stop rendering, that for all purpose release the hardware resources) keeping the last render in a DMA_BUF fd, then just use the regular wayland screen handle to select which server will have the X+Y output time and simply exchanging the DMA_BUF handles for consistency purposes. Input should be very simply since i believe is easy to inform libinput which server will have the focus to just route the input events properly.

In fact since this assumes by default the game and desktop will share the same session(because is the same user of course) there is no need to handle multiple TTY switches, so both server can be trivially handled by the compositor itself and it should be trivial to exit fullscreen mode since wayland allow by design nesting that again should be trivial to handle by the compositor

The only thing here is propose a standardised way for compositor/SDL2 to handle this scenario and avoid each game doing things differently every time making this a salad mess

[philcostin]

I like this idea but agree that there should probably be a middle ground.

This might sound crazy – but what about a small freedesktop.org library (not to be X of course!) which each different compositor such as kwin can use as its base / root window / gaming context?


That component would be the one which listens to libinput on behalf of the compositor and captures some events (such as Alt+Tab) early and passes them as calls to the compositor. The compositor (such as kwin) could then pass buffers with the content of the alt-tab window (or whatever else) for the root window library to overlay onto the game.

Once the game has been tabbed-out-of, the compositor gains full control of that lower component.

[rabcor]

Hey I'm on board with this but there are just two major problems.

1: Alt-Tab support (and that right now we are used to running games in one workspace, having other stuff in other workspaces, this breaks that too, we can't switch in and out of the game by switching workspaces)
2: Windowed mode gaming (some people actually prefer that)

These are the major problems for your typical end user (especially noobs on ubuntu or steamos that know nothing about linux and just use it like they would any other OS, not saying they're bad, just that 2 or 3 fps increase wouldn't matter to them as much as the 2 above things; convenience is everything to the consumer, performance and quality is everything to the prosumer (like the hardcore gamers and enthusiasts, why do you think I don't just use VLC like everybody else? Because it's quality and performance suck compared to mpv and mpc(windows)))

For users like me, I always play fullscreen and I have no major problem with switching from the DE's tty to the game's tty and back, if it is just launched in the next tty (i.e. I have my DE running on tty1 and game would launch on tty2 and if I launch another game it goes to tty3 and so on) but how do you explain all that to a normal consumer?

And sure, you could enable windowed mode gaming by running the game in an independent wayland session within wayland (like how you can run xorg sessions with xwayland or wayland sessions within x at the moment) but this would probably require a game restart (an option that can't be changed in the game's runtime, always a bother). And could potentially be a pain in the ass to support by individual games.

There's also the possible solution to alt tab support would be that the VT the game is launched in interprets alt tab as "switch back to DE" but then there is nothing to tie the DE back to the game, so for users used to doing that, that could only be confusing.

If all of these can be worked out then this would all be great, but unless they can be. I don't see this working out as a "default option" kind of thing (i.e. I think it would be best if the user is required to execute the game in this way of his own volition by adding launch commands, like "vtlaunch gamename" and voila you have a game on another tty on it's own wayland server, same principle applies to x11 games, that's the only viable way I'm seeing right now, unless you can convince game devs to officially support this. (Good luck with that, but it's not gonna happen fast))

If games would start supporting wayland out of the box, this could already start making some slight improvements to game performance right now if they were to be launched like this (and within an x session wayland sesions can be run anyways, I don't think they even take a performance hit, do they?, so the game could be run normally within an x session) which brings me to my 3rd problem... doesn't Wayland only support OpenGL ES atm? Sure it may add vulkan support in the future, but OpenGL ES is not as feature complete as say 4.3 I heard somewhere it's only just on par with WebGL (maybe it would be more accurate to say it's the other way around), but wouldn't this severely limit game developers in supporting wayland at this moment?

[Ancurio]

I've been toying with the idea of writing a small sample game that only uses KMS/DRM to setup a GBM buffer to render to via OpenGL (ES), query input devices via libudev and open them via systemd's logind (so my game doesn't have to be root). Audio could still go normally through OpenAL as the PulseAudio server doesn't care about what my graphics are doing, but I've considered directly using the PA API as well.

Unfortunately logind's API is very poorly documented IMO, but after a bit of digging I figured out how all the minor/major device id stuff works. After you've got the udev device you want, you can just query two integers from it that uniquely identify it, and providing them, logind will hand you an open fd to read generic input events from. (The problem is identifying which udev device is the main keyboard and not some pseudo one with 3 volume keys).

A good code sample on how to setup the GL context on bare KMS/DRM: https://github.com/robclark/kmscube (run this on a separate VT)

Now I wish I would have had the motivation to continue writing it