VirtIO DRM Window Server Support: Letting Guest VMs Interface With Host's Compositor

[tajjada]

I also assume in the future would be possible to pass each windows from the VM compositor to the host compositor as a series of buffers, so like on Mac VM Fusion you can have applications from the VM running as a window in the host compositor seamlessly.

I thought that this is exactly the point of this project. I.e it is not "in the future", this is exactly what it does.

Yeah it would, but there would be a ton of magic involved.

This can map Linux applications running in the guest to the Linux compositor running in the host, and this is great and all, but Windows applications aren't designed to interface with Linux compositors, so they won't work.

Unless someone replaces the whole Windows GUI components with a dedicated compatibility layer.

MORE LAYERS OF REDIRECTION FOR THE ABSTRACTION GOD!!!!

Everyone here seems to be thinking about Windows. Guys, this was done by the ChromeOS folks.

What I see as being the purpose of this is to allow you to easily and seamlessly run normal Linux apps in ChromeOS on your Chromebook. You can use ChromeOS and run some other Linux distro in a virtual machine and have the various gui apps running in there show seamlessly like normal windows in ChromeOS.

Although I still fail to see why they prefer a virtual machine over a chroot / lightweight container. It should be easier to install the distro of your choice in a chroot (like what you can already do with crouton) and expose a connection to the wayland compositor into it, so that you can then launch wayland apps from within the chroot and they will just connect to ChromeOS's wayland compositor. I don't understand why they are instead going for the complexity of dealing with virtual machines and having to use special drivers to cross the boundary between two operating systems.

[ermo]

Everyone here seems to be thinking about Windows. Guys, this was done by the ChromeOS folks.

What I see as being the purpose of this is to allow you to easily and seamlessly run normal Linux apps in ChromeOS on your Chromebook. You can use ChromeOS and run some other Linux distro in a virtual machine and have the various gui apps running in there show seamlessly like normal windows in ChromeOS.

Although I still fail to see why they prefer a virtual machine over a chroot / lightweight container. It should be easier to install the distro of your choice in a chroot (like what you can already do with crouton) and expose a connection to the wayland compositor into it, so that you can then launch wayland apps from within the chroot and they will just connect to ChromeOS's wayland compositor. I don't understand why they are instead going for the complexity of dealing with virtual machines and having to use special drivers to cross the boundary between two operating systems.

(emphasis mine)

What if this is Google's way of teaching their own non-Linux version of the Google OS to interact seamlessly with 'legacy' Linux-based desktop OSes?

Google is building its own NextGen micro-kernel-based OS that isn't Linux (the kernel is apparently called Zircon and the OS is called Fuchsia), and in that scenario, it would make all the sense in the world to teach this OS to display Linux apps running on Wayland in a VM?

Say what you will about Linux, but it is hard to argue that it isn't becoming increasingly legacy in terms of its design, now that it has been demonstrated that you can create performant micro-kernel OSes.

[starshipeleven]

Everyone here seems to be thinking about Windows. Guys, this was done by the ChromeOS folks.

You are quoting the wrong person, I said this is clearly made for linux host + linux guest setups.

What I see as being the purpose of this is to allow you to easily and seamlessly run normal Linux apps in ChromeOS on your Chromebook. You can use ChromeOS and run some other Linux distro in a virtual machine and have the various gui apps running in there show seamlessly like normal windows in ChromeOS.

Completely unnecessary, more often than not you just need a chroot to do that.

It's also completely pointless for their usecase, they make no money on Linux applications, neither directly nor with ads services. Google isn't a cartoon evil character like Microsoft or Omnomnomnomnomnomracle, but they still need to monetize their efforts somehow.

Although I still fail to see why they prefer a virtual machine over a chroot / lightweight container.

The only thing that is a Linux system but is alien enough to require some sort of VM setup is Android.
And Google did say they were working on running Android apps in ChromeOS, and hey have some setup to do that but it's not good enough.

[starshipeleven]

What if this is Google's way of teaching their own non-Linux version of the Google OS to interact seamlessly with 'legacy' Linux-based desktop OSes?

Yeah, this is also another possible reason, but that's something very future for now.

Say what you will about Linux, but it is hard to argue that it isn't becoming increasingly legacy in terms of its design, now that it has been demonstrated that you can create performant micro-kernel OSes.

I don't see how Linux design is legacy, beyond your personal dislike for monolithic kernels.
Also micro-kernels have no way to ensure that drivers for them are written with any amount of quality, unlike with Linux. Open drivers do have to conform to kernel code quality standards.

[oiaohm]

You are quoting the wrong person, I said this is clearly made for linux host + linux guest setups.

Completely unnecessary, more often than not you just need a chroot to do that.

It's also completely pointless for their usecase, they make no money on Linux applications, neither directly nor with ads services. Google isn't a cartoon evil character like Microsoft or Omnomnomnomnomnomracle, but they still need to monetize their efforts somehow.

Really there is a simple reason for google to be doing exactly what they are doing. Think about the server farm at times it may be useful to be able to run graphical to be more correct use GPU acceleration.

Yes you do have Intel(GVT-g), AMD (MxGPU), and NVIDIA (vGPU). You do have Nvidia with vGPU limiting the cards that support it.

So this may have absolutely nothing to-do with supporting other operating systems but instead provide a path for host/Domain 0 to multiplex cheaper GPU units for acceleration in a generic vendor neutral way so help in price negations in future with GPU vendors.

It always simple to forget google runs a huge server farm and it also simple to forget https://cloud.google.com/gpu/ that server farm uses GPU for acceleration. The difference between desktop and high end server is not as large as it use to be. The introduction of server farms using GPU acceleration moved things in a big way. ChromeOS and Android have been good places for Google to try stuff out some makes it back into their server farms.

There is the problem with Fuchsia idea. Yes Fuchsia has it own kernel?? wait does it. Why are Google developers careful to mention both. When you read the Fuchsia Template Library(FTL) you find out something interesting there is no such thing as a Fuchsia only application as all Fuchsia applications have to be built against the FTL and that is platform neutral. At this point everything should start to make sense. So Fuchsia is a experimental space with no trust that it will be using the prototype kernel included with it in future and it designed particularly that you can scrap the kernel at any time. So Fuchsia can fact will run on a Linux kernel. Really those putting up Fuchsia as some form of serous threat have not done their homework. It will take many more years before we will be sure what path Fuchsia is taking and that may or may not include the micro kernel. The documentation and code about Fuchsia makes that clear.

Now lets think for a moment about the company Collabora who is working on bring virtio_wl to Linux kernel main how it a use to them. Collabora works on Libreoffice online. Does Libreoffice online depend on gpu acceleration at all for best performance? Scary answer yes like opencl for calc. It could also be useful for build farm costs for Collabora and allow broader gpu version testing for Linux desktop and Server versions of their CollaboraOffice line(that is based off libreoffice).

Lightweight container is not much usage when you need to be testing against multi kernel versions or exact as operating systems are deployed.

This is one of those things that you could have predicted someone would implement at some point mainline because it basically makes sense.

[starshipeleven]

So this may have absolutely nothing to-do with supporting other operating systems but instead provide a path for host/Domain 0 to multiplex cheaper GPU units for acceleration in a generic vendor neutral way so help in price negations in future with GPU vendors.

Wait.
1. it's done on the ChromeOS kernel, and Google isn't running that thing in cloud VMs unless I'm mistaken.
2. this works for integrating a Wayland application on the host's compositor, which makes it look native. Not providing acceleration to the application.

Last time I checked Wayland applications are responsible of their own rendering, so they make OGL or Vulkan calls on their own, render the frame of their window, then send this frame to the compositor that deals with it to create the full screen image. Sooo, this can't be accelerating. It can only be integrating applications from host and guest(s) running Wayland in the most native way possible. Because the application (in the VM) still needs to have access to a GPU (real or otherwise) through other means to render itself first, which is outside of the scope of this feature.

[Hasenpfote]

Yeah it would, but there would be a ton of magic involved.

Well, then call these guys Wizards or Warlocks:

https://looking-glass.hostfission.com/

https://forum.level1techs.com/t/a-little-teaser-of-what-is-to-come/121641/499

Code:

https://github.com/gnif/LookingGlass

Essentially they are copying the framebuffer to a shared memory on the Windows guest. On the host, this is mapped onto an OpenGL plane and then rendered.
Speed is like native! This has been implemented by one guy (+ support from several others) I think. Within two months!

Here is a video of an early alpha build

https://youtu.be/1MI1s4hZ_yE?t=285

This is THE feature people want for Linux!
Finally gaming in Linux without dual booting or having a second screen on the graphic card.
Yes, you still need a second graphic card.

I don't understand why VMware, Oracle and Microsoft did not came up with it.
Luckily they didn't and those guy release it under GPL.

[starshipeleven]

Well, then call these guys Wizards or Warlocks:

I said "magic OR someone wrote a windows component to redirect its GUI over" and yeah it seems they did.

And that Looking Glass is Linux-Host-with-Windows-guest only.

I don't understand why VMware, Oracle and Microsoft did not came up with it.

I guess there isn't much incentive to do that when the only way to get decent 3D performance in the VM is to passtrhough a GPU (which isn't even supported by VMWare and Virtualbox). That is clearly not what they envision most of their customers will want to do.

[oiaohm]

Wait.
1. it's done on the ChromeOS kernel, and Google isn't running that thing in cloud VMs unless I'm mistaken.
2. this works for integrating a Wayland application on the host's compositor, which makes it look native. Not providing acceleration to the application.

Last time I checked Wayland applications are responsible of their own rendering, so they make OGL or Vulkan calls on their own, render the frame of their window, then send this frame to the compositor that deals with it to create the full screen image. Sooo, this can't be accelerating. It can only be integrating applications from host and guest(s) running Wayland in the most native way possible. Because the application (in the VM) still needs to have access to a GPU (real or otherwise) through other means to render itself first, which is outside of the scope of this feature.

This is what happens when you don't read the patch.

https://lists.freedesktop.org/archives/dri-devel/2017-December/160309.html

Note this is a modification virtio-gpu.

https://www.youtube.com/watch?v=Ir_to-SuwXE

Yes this is last year where virtio-gpu was allowing virtual machine linux guests to access host gpu unit. So virtio_wl idea from google chromeos is now merged into virtio-gpu.

So this adds compositor support some gpu accelerated programs refuse to work if they don't find a X11 server or Wayland compositor.

Features of virtio_wl merged into virtio-gpu as Collabora developer had done makes a very interesting modification. Buffers in a virtio-gpu context are way more covering this is all your shaders, textures..... but it was not your compositor until now. Reality is one of the missing pieces from the virtio-gpu mix has just been added.

virtio-gpu + vir3d gets interesting.

http://www.studiopixl.com/blog/index.php/posts/3d-acceleration-using-virtio

Really that Looking glass stuff is just another group attempting to-do what virtio-gpu is up to using a qemu particular part.

[starshipeleven]

This is what happens when you don't read the patch.

https://lists.freedesktop.org/archives/dri-devel/2017-December/160309.html

Note this is a modification virtio-gpu.

You must read things I don't see, because I still see it makes a way to send guest-rendered frames to the host's compositor.

And rendered frames on the guest still need some kind of access to a GPU.

Yes this is last year where virtio-gpu was allowing virtual machine linux guests to access host gpu unit.

That is unrelated to this patch (as it is what virtio-gpu can do already), and still fits within what I stated.

virt-io GPU is a way to access a host GPU from the guest, which is required to have the guest render stuff on its own in the first place. Then this patch adds a way to move over rendered "windows" to the host.

Really that Looking glass stuff is just another group attempting to-do what virtio-gpu is up to using a qemu particular part.

Wat?
Looking Glass is still a way to pipe rendered frames to the host and integrate them seamlessly in it as if they were native programs, but per-se isn't exposing host GPU.

While virtio-gpu is much more than just this feature (as it also exposes the GPU in the guest, which is still required because the guest still renders stuff on its own before sending the buffer to the host's compositor).