Graphics Card Support For QEMU/KVM Comes Closer

One of the major limitations of QEMU/KVM when it comes to Linux desktop virtualization is that it hasn't provided any form of support for graphics acceleration -- either by passing OpenGL calls onto the host for execution by the host's driver and graphics hardware or allowing guest VMs to tap directly the graphics card. Support for the latter feature continues to be developed and is close to becoming a working reality.

Allowing QEMU virtual machines to directly interact with PCI devices has become a reality with the merging of the VFIO driver last August. VFIO for the Linux kernel provides a secure interface for device access using IOMMU so that, among other purposes, a physical PCI device could be assigned to a QEMU guest. Up to this point in the KVM world there were basically some x86 hacks to allow similar support.

QEMU 1.3 picked up VFIO support (among other interesting and fun features), but VFIO hasn't supported graphics cards but only more simple PCI-based devices. As I wrote about at the beginning of the month, QEMU is getting close to allow passing GPUs to guests.

Published today on the Linux kernel mailing list was a new patch for VFIO-PCI to add support for VGA region access. The to-be-added VFIO_PCI_VGA kernel option allows for a VGA extension to the VFIO PCI driver that exposes an additional region on VGA devices for accessing legacy VGA addresses used by BIOS and generic video graphics drivers.

While there's that kernel patch, published since the other Phoronix article earlier this month were QEMU patches. The QEMU patches rely upon the VFIO kernel support for accessing VGA ranges. Accelerated hardware graphics drivers don't work with these initial QEMU patches, but it's working so far for graphics adapters using the SeaBIOS text mode output with standard VGA drivers.

The QEMU patches explain:

Most VGA cards need some kind of quirk to fully operate since they hide backdoors to get to other registers outside of PCI config space within the registers, but this provides the base infrastructure. If we could identity map PCI resources for assigned devices we would need a lot fewer quirks.

To enable this, use a kernel side vfio-pci driver that incorporates VGA support (patches posted), and use the -vga none option and add the x-vga=on option for the vfio-pci device. The "x-" denotes this as an experimental feature. You may also need to use a cached copy of the VGA BIOS for your device, passing it to vfio-pci using the romfile= option.

Additionally:

Apparently graphics vendors need to come up with new ways to retrieve PCI BAR addresses on every revision of their chip. These are the ones that I've found on the following assortment of cards:

Advanced Micro Devices [AMD] nee ATI Cedar PRO [Radeon HD 5450/6350]
Advanced Micro Devices [AMD] nee ATI RV370 [Radeon X550]
NVIDIA Corporation G98 [GeForce 8400 GS]
NVIDIA Corporation G86 [Quadro NVS 290]
NVIDIA Corporation G72 [GeForce 7300 LE]

With these quirks, each is able to POST under seabios, display text mode VGA spew, and switch into graphics mode. All but the X550 also seem to work under Windows7 using the standard VGA driver (Windows automatically installed a new driver for the X550 and it hasn't worked since). Installing the AMD/ATI Catalyst driver for the HD 5450 hung the host, the NVIDIA cards report a yellow bang in device manager, so something is still not entirely happy there. The NVIDIA provided driver install, but reports Code 43(?).

It's relatively easy to figure out many of these quirks. First enable DEBUG_UNASSIGNED in exec.c, then enable DEBUG_VFIO in hw/vfio_pci.c. Log the output and can kill Qemu when Unassigned access errors start to spew (ignore the ones at very low offsets). If the unassigned access matches a range covered by the device (consult lspci or /proc/iomem), then look back in the vfio-pci debug output for read from the device that returned an address or partial address matching the unassigned access. Then follow these examples for creating quirks to trap that access and return the emulated BAR address.

None of these would be necessary if we could identity map devices.

Even with this patch, vfio-based VGA assignment is extremely experimental and quite fragile. Enable at your own risk.

At least they're slowly moving in the right direction...