VirtIO DRM Window Server Support: Letting Guest VMs Interface With Host's Compositor

[oiaohm]

AIUI, the L4 family demonstrated that high performance is achievable in a microkernel design.

In addition, seL4 has the interesting property of being formally verified against its specification, which is admittedly orthogonal to any performance claims, but still pretty cool.

Not really L4 microkernel made a good hyper-visor but most of the time it was using drivers in monolithic Linux kernel. So while the Linux kernel had the cpu time it was handling more than 1 hardware device. So there has not be a micro-kernel with micro-kernel drivers demonstrate high performance. Even QNX lot of it drivers went monolithic.

[ermo]

When has this been demonstrated? What project has demonstrated this? What are these performant micro-kernel OSes you are talking about?

AIUI, the L4 family demonstrated that high performance is achievable in a microkernel design.

In addition, seL4 has the interesting property of being formally verified against its specification, which is admittedly orthogonal to any performance claims, but still pretty cool.

[tajjada]

now that it has been demonstrated that you can create performant micro-kernel OSes.

When has this been demonstrated? What project has demonstrated this? What are these performant micro-kernel OSes you are talking about?

[oiaohm]

https://xpra.org/

Not as good as you would think. Having to cross network adds a lot of latency. Windows TCP stack has connection limits.

https://community.spiceworks.com/topic/980907-windows-10-maximum-connection-limits

Yes you desktop version of windows are between 10 to 20 incoming active tcp/udp connections. So using RDP or xpra you have consumed very valuable limited resource under Windows.

Virtio-gpu and the new project looking glass are both using virtual machine supported IPC these don't count in the network stack.

Also going into the network stack xpra introduces more latency so less felling like native than virtio-gpu and the new project looking glass.

So like it or not what you are suggesting bofh80 is really not as good as we can do.

With the connection issues of windows it would be great to have a virtio-filesystem that could be network file systems from host that are not consuming up the Windows limited network connections. So it could be possible for Windows to perform better under a virtual machine than native all due to inconvenient limitations designed into Windows.

[oiaohm]

Hm, this is very interesting, thanks.

I didn't consider that if you offload the rendering to host GPU with virtio-gpu (instead of using a KVM/Xen passthrough GPU the host has no control over) then the rendered frame is already on the host and can be sent to compositor directly.

It makes a hell of a lot of difference as well. Remember PCIe transfer limits are lower than host ram transfer limits. So using KVM/Xen passthrough GPU and scraping the graphics back off to be transferred to host costs the PCI-e bus all the render request and the scrape then uploading the scrape onto the right video card. Virtio-gpu has more cpu mmu memory juggling and the possibility of bundling multi applications on one screen output before performing scrape but what is transferred across the pci-e bus is kept as light as possible. Of course Virtio-gpu used with Prime and a KVM/Xen pass-through will wake up a lot of the pce-e buss transfer space consume problems.

So it will be how the applications choke the hardware that will decide if KVM/Xen pass-through gpu processing or virtio-gpu gpu processingis better and sometimes a mix of the two.

Also with host controlled gpu something generated on the GPU can stay on the GPU in the GPU ram even it if request of generation was requested by a guest if the guest and host compositor never wants result and still have that fragment sent to output. This also can reduce volume of traffic that need to be transferred back to host and guest for virtio-gpu.

Its really simple to forget the case of many small virtual machines doing many small light gpu dependant operations that user need to see the output of and that something like looking glass could run you out of the all import pci-e bus bandwidth.

[starshipeleven]

Clients will use the communication protocol that the compositor supports, and virtio-gpu will assist with making buffers available in both sides, and copying content as needed. It this sentence on the patch you have not read or you at least missed the last bit. Why is it copying content as needed. With virtio-gpu due to having access to hosts gpu render tasks performed on host gpu the instruction to host compositor from guest might be using a frame that was generated host side and has not been transferred back to guest. So this virtio-gpu patch will not always be moving rendered windows over to host as host might have rendered the buffer following the guest instructions and now has a message to display that buffer so the buffer might never be transferred back to guest because virtio-gpu avoids pointless coping.

Hm, this is very interesting, thanks.

I didn't consider that if you offload the rendering to host GPU with virtio-gpu (instead of using a KVM/Xen passthrough GPU the host has no control over) then the rendered frame is already on the host and can be sent to compositor directly.

[oiaohm]

virt-io GPU is a way to access a host GPU from the guest, which is required to have the guest render stuff on its own in the first place. Then this patch adds a way to move over rendered "windows" to the host.

Please type it name right its not virt-io gpu its virtio-gpu absolute correct and virtio gpu slightly wrong. There is something else out there called virt-io gpu and its totally a different thing.
Clients will use the communication protocol that the compositor supports, and virtio-gpu will assist with making buffers available in both sides, and copying content as needed. It this sentence on the patch you have not read or you at least missed the last bit. Why is it copying content as needed. With virtio-gpu due to having access to hosts gpu render tasks performed on host gpu the instruction to host compositor from guest might be using a frame that was generated host side and has not been transferred back to guest. So this virtio-gpu patch will not always be moving rendered windows over to host as host might have rendered the buffer following the guest instructions and now has a message to display that buffer so the buffer might never be transferred back to guest because virtio-gpu avoids pointless coping.

Wat?
Looking Glass is still a way to pipe rendered frames to the host and integrate them seamlessly in it as if they were native programs, but per-se isn't exposing host GPU.

While virtio-gpu is much more than just this feature (as it also exposes the GPU in the guest, which is still required because the guest still renders stuff on its own before sending the buffer to the host's compositor).

One of the big difference between looking glass and virtio-gpu is the fact that under virtio-gpu the buffer the host compositor is being told to display might be a buffer the gpu connected to host has generated.

Looking Glass at this stage would be passing a full video card in the very expensive cards then capturing and transferring. Virtio-gpu supports drm prime mode so you use a decanted a card to a virtual machine and use virtio-gpu to transfer the output of that card to the host as well.

https://wiki.debian.org/VGAPassthrough When you say guest renders stuff on own this sound to me like VGAPassthrough as this would not be using the host resources. Using virtio-gpu gpu functionality is using the host resources. It is important to understand when you are using host or guest because this can provide very different resource starvation points.

Looking glass is lot more primitive item.

[starshipeleven]

This is what happens when you don't read the patch.

https://lists.freedesktop.org/archives/dri-devel/2017-December/160309.html

Note this is a modification virtio-gpu.

You must read things I don't see, because I still see it makes a way to send guest-rendered frames to the host's compositor.

And rendered frames on the guest still need some kind of access to a GPU.

Yes this is last year where virtio-gpu was allowing virtual machine linux guests to access host gpu unit.

That is unrelated to this patch (as it is what virtio-gpu can do already), and still fits within what I stated.

virt-io GPU is a way to access a host GPU from the guest, which is required to have the guest render stuff on its own in the first place. Then this patch adds a way to move over rendered "windows" to the host.

Really that Looking glass stuff is just another group attempting to-do what virtio-gpu is up to using a qemu particular part.

Wat?
Looking Glass is still a way to pipe rendered frames to the host and integrate them seamlessly in it as if they were native programs, but per-se isn't exposing host GPU.

While virtio-gpu is much more than just this feature (as it also exposes the GPU in the guest, which is still required because the guest still renders stuff on its own before sending the buffer to the host's compositor).

[oiaohm]

Wait.
1. it's done on the ChromeOS kernel, and Google isn't running that thing in cloud VMs unless I'm mistaken.
2. this works for integrating a Wayland application on the host's compositor, which makes it look native. Not providing acceleration to the application.

Last time I checked Wayland applications are responsible of their own rendering, so they make OGL or Vulkan calls on their own, render the frame of their window, then send this frame to the compositor that deals with it to create the full screen image. Sooo, this can't be accelerating. It can only be integrating applications from host and guest(s) running Wayland in the most native way possible. Because the application (in the VM) still needs to have access to a GPU (real or otherwise) through other means to render itself first, which is outside of the scope of this feature.

This is what happens when you don't read the patch.

https://lists.freedesktop.org/archives/dri-devel/2017-December/160309.html

Note this is a modification virtio-gpu.

https://www.youtube.com/watch?v=Ir_to-SuwXE

Yes this is last year where virtio-gpu was allowing virtual machine linux guests to access host gpu unit. So virtio_wl idea from google chromeos is now merged into virtio-gpu.

So this adds compositor support some gpu accelerated programs refuse to work if they don't find a X11 server or Wayland compositor.

Features of virtio_wl merged into virtio-gpu as Collabora developer had done makes a very interesting modification. Buffers in a virtio-gpu context are way more covering this is all your shaders, textures..... but it was not your compositor until now. Reality is one of the missing pieces from the virtio-gpu mix has just been added.

virtio-gpu + vir3d gets interesting.

http://www.studiopixl.com/blog/index.php/posts/3d-acceleration-using-virtio

Really that Looking glass stuff is just another group attempting to-do what virtio-gpu is up to using a qemu particular part.

[starshipeleven]

Well, then call these guys Wizards or Warlocks:

I said "magic OR someone wrote a windows component to redirect its GUI over" and yeah it seems they did.

And that Looking Glass is Linux-Host-with-Windows-guest only.

I don't understand why VMware, Oracle and Microsoft did not came up with it.

I guess there isn't much incentive to do that when the only way to get decent 3D performance in the VM is to passtrhough a GPU (which isn't even supported by VMWare and Virtualbox). That is clearly not what they envision most of their customers will want to do.