Intel KVMGT 2018-Q1 Release Offers Mediated GPU Pass-Through Improvements

[phoronix]

Phoronix: Intel KVMGT 2018-Q1 Release Offers Mediated GPU Pass-Through Improvements

While the relevant bits for supporting Intel GPU mediated pass-through to virtual machines with KVM are now upstream in the Linux kernel as well as in QEMU 2.12, Intel developers have just announced their quarterly release of "KVMGT" for those wanting the officially blessed configuration for running Intel virtual GPU support with KVM virtual machines...

http://www.phoronix.com/scan.php?page=news_item&px=Intel-KVMGT-2018-Q1

[kuco]

So, is GVT like PCI-passthrough, where the "guest-gpu" is completely isolated and you can not integrate your guest-desktop into your host-desktop? So your only option on a desktop-system is RD'ing into a headless host or buying a second GPU(+monitor+KVM-switch)?

I'm regularly playing a handful of ("esport")-games with low requirements and no linux support. Wine is stressful and is breaking alot because the games are still regularly patched/changed. I'm trying to avoid dual booting. GVT+Intel Iris+Windows guest sounds like a great option if one could integrate the guest like a regular non-3d VM? I guess that's not the case?

The situation is really frustrating ^^ I dont want to stop playing those games, but PCI-passthrough is so much work and usability is kinda bad. I might aswell dual-boot then. My hopes are on "looking-glass" (https://www.youtube.com/watch?v=okMGtwfiXMo) and improvements in Wine (like DXVK).

[Aleksei]

So, is GVT like PCI-passthrough, where the "guest-gpu" is completely isolated and you can not integrate your guest-desktop into your host-desktop? So your only option on a desktop-system is RD'ing into a headless host or buying a second GPU(+monitor+KVM-switch)?

I'm regularly playing a handful of ("esport")-games with low requirements and no linux support. Wine is stressful and is breaking alot because the games are still regularly patched/changed. I'm trying to avoid dual booting. GVT+Intel Iris+Windows guest sounds like a great option if one could integrate the guest like a regular non-3d VM? I guess that's not the case?

The situation is really frustrating ^^ I dont want to stop playing those games, but PCI-passthrough is so much work and usability is kinda bad. I might aswell dual-boot then. My hopes are on "looking-glass" (https://www.youtube.com/watch?v=okMGtwfiXMo) and improvements in Wine (like DXVK).

There is this writeup by one of the developers on how to use it - https://www.kraxel.org/blog/2018/04/...rged-upstream/. I didn't get around to fiddle with this yet. If you give it a shot, please post your results _{(so I don't have to figure it out myself)}.

[kuco]

Thanks, this looks awesome. So the guest is running in a spice-window, the iGPU is shared between host and guest and the host can actually access the frames rendered by the vgpu?

I've never used spice before and I'm not sure if I understand how it works. How is the performance/delay when using spice? Is it even usable for gaming? Also, I'm a bit puzzled about the "<gl enable='yes'/>"-option for spice. Will this work for Windows-Guests/D3d, too?

edit:
After skimming trough some wikis: I guess theres still alot of overhead/copying going on to get the guest-frames into the host-gpu, so it's not really usable for gaming (?). So I guess, running a "low-latency-3D-accelerated-windows-guest" inside a window on a linux-host is still an unachievable dream ^^ But project "looking-glass" is looking very promising (see link above).

[Aleksei]

Thanks, this looks awesome. So the guest is running in a spice-window, the iGPU is shared between host and guest and the host can actually access the frames rendered by the vgpu?

I've never used spice before and I'm not sure if I understand how it works. How is the performance/delay when using spice? Is it even usable for gaming? Also, I'm a bit puzzled about the "<gl enable='yes'/>"-option for spice. Will this work for Windows-Guests/D3d, too?

Say, if your games have low requirements - why not just use a windows VM with plain old QXL GPU and Spice?

[kuco]

Man, I'm getting so confused right now. So many options/technologies available. But as far as I understand, QXL is not really suitable for DX11 gaming? By "low requirements" I just meant, an Intel (Iris) GPU is OK, to run those games @60fps@fullhd@low-settings. And SPICE seems to add alot of delay, which is kind of bad for competitive multiplayer games. Sorry for being confused. This stuff is too technical for me and there are so many technologies available without "noob-friendly" documentation (as most gpu-virtualization-solutions are very business-focused).

[Aleksei]

But as far as I understand, QXL is not really suitable for DX11 gaming?

I don't know. Try and see.

And SPICE seems to add alot of delay, which is kind of bad for competitive multiplayer games.

I don't notice any delay in my windows 7 VM with QXL/Spice - but I don't run any games on it either. Again - try and see.

Man, I'm getting so confused right now. So many options/technologies available.

Welcome to Linux ArchWiki is your friend. I'd probably buy a cheap low-end GPU for the guest (or the host).

https://wiki.archlinux.org/index.php/QEMU

https://wiki.archlinux.org/index.php/PCI_passthrough_via_OVMF

[polarathene]

Thanks, this looks awesome. So the guest is running in a spice-window, the iGPU is shared between host and guest and the host can actually access the frames rendered by the vgpu?

I've never used spice before and I'm not sure if I understand how it works. How is the performance/delay when using spice? Is it even usable for gaming? Also, I'm a bit puzzled about the "<gl enable='yes'/>"-option for spice. Will this work for Windows-Guests/D3d, too?

edit:
After skimming trough some wikis: I guess theres still alot of overhead/copying going on to get the guest-frames into the host-gpu, so it's not really usable for gaming (?). So I guess, running a "low-latency-3D-accelerated-windows-guest" inside a window on a linux-host is still an unachievable dream ^^ But project "looking-glass" is looking very promising (see link above).

I've not looked into the Intel vGPU stuff yet, my understanding was that you got raw display output sent to the host OS that avoided the latency you'd get with something like QXL(no idea about virtual display like VNC/SPICE, is that required for vGPU?). I'm not sure about the gl enable tag, no experience with that yet, I know it's meant to improve 3D perf.

QXL doesn't support modern DirectX reliably in my experience, Uniengine benchmarks like Heaven is a good example, you'd use OpenGL. IIRC, OpenGL was getting 1-2FPS with QXL and Spice, but if I used my dGPU as passthrough(a GTX 1070), it would get something like 80FPS on max settings for OpenGL. If I had QXL/Spice also active I'd get less FPS(eg, 60-70) and the benchmark listed both the virtual GPU(QXL) and my dGPU as being used.

The iGPU passthrough with vGPU will let you split the resources of the iGPU between your host and guest OS, regular passthrough for dGPU is exclusive access to one OS. I assume the Intel vGPU will handle DirectX better than QXL, and performance is probably much better too. If you're not having a good time with the viewer, Looking Glass might be an option. If you seem to have input latency, you can use evdev for mouse and keyboard, that will allow you to press both ctrl buttons on keyboard to redirect input between host and guest OS.

I'm presently working on a Looking Glass port to Rust(because I'm more comfortable with Rust than I am C and C++ and would like to add some features and support Linux guests while utilizing the work of others easily via others libraries/crates). I've had a basic program capture my host desktop from one monitor and display it on another just using the iGPU and getting several hundred FPS at 1080p. Reading frames to my client program from the LG host/server(on the VM) which is powered by the dGPU has similar perf. Looking Glass will impact on the VM a bit though as it uses some CPU and GPU resources.

Note that when you display the VM display on the host and it's not 1:1, perf can take a dive. With my client, software(system RAM for frames/textures) is 2ms a frame currently, if windowed so that it's not 1:1 pixels, then that slightly smaller size brings the per frame time up to 5ms. Using OpenGL backend(frames/textures have to get uploaded to GPU) takes 4ms a frame, that can scale much better without the perf hit, but you already take a perf hit from uploading textures to GPU vRAM). If your VM uses a smaller resolution than the host Display, windowed 1:1 scale will be fine

I don't know what performance/overhead is like with the Intel vGPU, but it might be ok for you.

[kuco]

Thanks polarathene and aleksei for the input gonna try to get some sort of vgpu working when there is time

[m132]

kuco QXL (guest OS display driver) has no 3D acceleration at all, it only passes 2D rendering operations from guest to host, so these can be carried over through SPICE. This results in better performance over the network, but only if the desktop is rendered purely using 2D acceleration, so it works only for Windows < 8 or X11 without a compositor. Windows 8 and newer use software renderers for adapters without Direct3D support, which explains why you can get composited desktop and 3D benchmarks/games to run, but at horrible performance.

SPICE (the protocol used for communication between QEMU hypervisor and graphical frontend, eg. spicy, virt-viewer or virt-manager) has OpenGL support (<gl enable='yes'/>), but this doesn't do anything at all unless virtualized video adapter supports it.
As of now, the only adapter making use of SPICE's OpenGL support is the VirtIO GPU (with 3D acceleration codenamed Virgil). This one works in similar fashion to VirtualBox's and VMware's 3D accelerated adapters. It creates a virtual, OpenGL capable adapter in the guest, that passes GL commands and shaders to the host, which in turn translates and executes these using host's OpenGL driver. This is pretty slow and crash-prone right now, but it's vendor independent and has potential to improve in later years.

Intel GVT-g in turn, is a complete GPU (para)virtualization solution, that allows you to create virtual GPU on the host, and pass it to the VM using VFIO. This allows guest to use native Intel graphics driver, without depending on host driver like Virgil does. This means you can get native Direct3D support in guest, even if hosts driver does not support DirectX at all. This also means you get lots of things not related to 3D acceleration, like video de/encoding support, inside the VM.

Today, GVT-g is kind of half-baked, people are reporting problems with it when combined with UEFI inside guest and it doesn't support basic VGA mode (there's completely no video output until guest loads the driver, no BIOS screen, no "Windows is starting" message), so you have to install the guest with QXL graphics first, and switch to GVT-g once you install Intel driver.

Performance of GVT-g is not that bad, it struggles with some 3D complex games (while there's no such problem when running them on bare metal), but achieves stable 60 FPS with desktop compositing + basic web browsing workload.

There are two ways I know to get the video output out of the VM. The older way (not upstreamed I think) is to use sysfs to switch whole display between host and guest. The new one (upstreamed in Linux 4.16 and QEMU 2.12) is to use DMA Buffer Sharing to get guest display nicely rendered inside QEMU window of the host. I've used dma-buf method, and it works pretty well when combined with QEMU's GTK UI, it crashes with SDL output though. There's absolutely no tearing there, but the video output is kinda choppy regardless of used resolution (I'd say it achieves ~30 FPS). That's not the problem with virtual GPU itself, I've recorded the screen inside the guest (Windows 10, using Action! screencast software), and the recorded material was buttery smooth, it had constant 60 FPS frame rate.

It's possible to use libvirt, but it doesn't yet support VFIO display output, so you'll have to use Cirrus display adapter, override it with some non-functional driver and use <qemu:commandline> to instruct QEMU to use GVT-g.
It's also possible to use SPICE (recommended when going with libvirt, as SELinux will block QEMU from creating GTK/SDL windows in this use case), but GVT-g doesn't yet support SPICE's OpenGL, so you'll get a black screen. To fix this, you have to use egl-headless as main output, and SPICE as second one, so the display is originally rendered on egl-headless and then copied to SPICE.
Checkout link about dma-buf above for examples.

TL;DR: GVT-g is not yet ready for comfortable gaming in it's current state, but maybe you'll find a way to overcome this by installing some game streaming software inside guest. If not, then there are no better solutions, other than complete passthrough.