AMD/Ryzen NPT Fix Discovered For Better Pass-Through Graphics Performance

[Niarbeht]

I have a question concerning the GPU passthrough. How can I get my mouse and keyboard to interact with the VM without having a second USB mouse and keyboard? I tried using a keyboard-mouse switcher (a KVM without video) and switching back to the VM causes the keyboard and mouse to malfunction. I don't have Linux handy, but I remember that every time I unplug my mouse and keyboard, the ID number increments and I did not want that to happen. I'm at World Services for the Blind with only a Windows laptop as I'm getting certified in A+ and Network+, but my question about VM in Linux with PCIe passthrough still stands.

Alrighty. I'm home now, and hopefully I can give you some ideas here.

  <devices>
...
    <input type='mouse' bus='virtio'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/>
    </input>
    <input type='keyboard' bus='virtio'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'/>
    </input>
...
  </devices>

(some more junk)

  <qemu:commandline>
    <qemu:arg value='-object'/>
    <qemu:arg value='input-linux,id=kbd1,evdev=/dev/input/by-id/usb-03eb_EasyAVR_Multimedia_Keyboard_v2.03.01-event-kbd,grab_all=on,repeat=on'/>
    <qemu:arg value='-object'/>
    <qemu:arg value='input-linux,id=mouse1,evdev=/dev/input/by-id/usb-Logitech_Gaming_Mouse_G600_F62C58A065B90017-event-mouse'/>
    <qemu:arg value='-object'/>
    <qemu:arg value='input-linux,id=kbd2,evdev=/dev/input/by-id/usb-03eb_EasyAVR_Multimedia_Keyboard_v2.03.01-event-if02'/>
    <qemu:arg value='-object'/>
    <qemu:arg value='input-linux,id=kbd3,evdev=/dev/input/by-id/usb-03eb_EasyAVR_Multimedia_Keyboard_v2.03.01-if01-event-mouse'/>
  </qemu:commandline>

So, what you see there is that I've created a virtio mouse and a virtio keyboard. I then add mouse and keyboard event thingies to the system via the QEMU command line. Then, some magic happens, and when I press both CTRL keys on my keyboard, I pass the keyboard/mouse inputs specified through to the VM. If I press both CTRL keys again, they pass back to the host. It's actually kinda nice, though it's a pain to set up and I wish there were a GUI-er way to do it. If I remember right, the mouse and keyboard virtio input devices can be added via virt-manager, but the QEMU command line magic to add the input devices still needs to be added via

virsh edit [I]nameofvirtualmachine[/I]

EDIT: Note that you'll need to customize the QEMU command line section to match the input devices present on your system. I kinda wish that you could do it all via the virt-manager GUI, and just have it enumerate input devices or something for you and have a tester to verify that you've selected the right input devices, that'd be really snazzy. I don't think these features are a priority for the big-name development companies like Red Hat, though.

[gnif]

Hi All,

I am Geoffrey McRae (aka, Geoff).

@AMD we have a proper patch thanks to Paolo Bonzini, please see:

https://patchwork.kernel.org/patch/10027523/

This article leaves a few things out.

1) This bug affects any passthrough of any device to a VM under KVM XEN, devices that failed to work or operate correctly in the past may now work (USB controllers for example)
2) This wasn't just a problem with Ryzen, but all AMD CPUs with NPT support over the last 10+ years.
3) There is a subreddit for this: https://www.reddit.com/r/VFIO/commen...ssed_on_iommu/

[gnif]

Posting this a second time sorry, seems my post is awaiting moderation due to containing links/urls and I am a new user.

I am Geoffrey McRae (aka, geoff or gnif), just like to point a few things out here the article left out:

1) AMD, this problem has been patched properly thanks to Paolo Bonzini, please see patchwork.kernel.org for patch 10027523 (KVM: SVM: obey guest PAT)
2) This doesn't just affect Ryzen processors, any AMD CPU going back 10+ years that has NPT had this problem.
3) This doesn't just fix VGA Pass through, it corrects the way all pass through devices are configured. You may find devices that prior didn't work such as USB controllers behave properly now, or even achieve better throughput.
4) There is an official subreddit for this, search for "possible_fix_for_the_npt_issue_discussed_on_iommu "

[polarathene]

Alrighty. I'm home now, and hopefully I can give you some ideas here.

<devices>
...
<input type='mouse' bus='virtio'>
<address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/>
</input>
<input type='keyboard' bus='virtio'>
<address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'/>
</input>
...
</devices>

(some more junk)

<qemu:commandline>
<qemu:arg value='-object'/>
<qemu:arg value='input-linux,id=kbd1,evdev=/dev/input/by-id/usb-03eb_EasyAVR_Multimedia_Keyboard_v2.03.01-event-kbd,grab_all=on,repeat=on'/>
<qemu:arg value='-object'/>
<qemu:arg value='input-linux,id=mouse1,evdev=/dev/input/by-id/usb-Logitech_Gaming_Mouse_G600_F62C58A065B90017-event-mouse'/>
<qemu:arg value='-object'/>
<qemu:arg value='input-linux,id=kbd2,evdev=/dev/input/by-id/usb-03eb_EasyAVR_Multimedia_Keyboard_v2.03.01-event-if02'/>
<qemu:arg value='-object'/>
<qemu:arg value='input-linux,id=kbd3,evdev=/dev/input/by-id/usb-03eb_EasyAVR_Multimedia_Keyboard_v2.03.01-if01-event-mouse'/>
</qemu:commandline>

So, what you see there is that I've created a virtio mouse and a virtio keyboard. I then add mouse and keyboard event thingies to the system via the QEMU command line. Then, some magic happens, and when I press both CTRL keys on my keyboard, I pass the keyboard/mouse inputs specified through to the VM. If I press both CTRL keys again, they pass back to the host. It's actually kinda nice, though it's a pain to set up and I wish there were a GUI-er way to do it. If I remember right, the mouse and keyboard virtio input devices can be added via virt-manager, but the QEMU command line magic to add the input devices still needs to be added via

virsh edit [I]nameofvirtualmachine[/I]

EDIT: Note that you'll need to customize the QEMU command line section to match the input devices present on your system. I kinda wish that you could do it all via the virt-manager GUI, and just have it enumerate input devices or something for you and have a tester to verify that you've selected the right input devices, that'd be really snazzy. I don't think these features are a priority for the big-name development companies like Red Hat, though.

I'm using Synergy myself, but thanks for the alternative info. Just from the QEMU args you listed, is this the same evdev approach mentioned elsewhere? It sounds like it, I just hadn't seen people mention virtio keyboard/mouse in relation to it. If not could be useful to share with r/vfio community

There are a few languages that support libvirt api, which presumably can do what virsh can? I know there is one for Rust at least and someone was writing something with GUI to ease setup on r/vfio in GO, putting that together might not be too much work to request, no idea about getting/verifying the correct mouse/keyboard devices to use though.

[Riyyi]

Hm, sad to hear that Vega is having GPU reset problems. Is RX 480 having the same problems too? One of the reasons why I want to replace my current GPU is because it's not capable of passthrough, would be nice if the replacement was...

Wait so my R9 Fury is having GPU reset problems, the 480, 580 and now Vega is having the same issues again? It's getting hard to keep liking AMD if you're into Linux.

[GraysonPeddie]

Thanks everyone about the mention of Synergy. And thanks Niarbeht for the alternative. I will give Synergy a try whenever I decide to build me a new PC with AMD Ryzen. I will be getting a second GPU as the Ryzen processors do not have iGPU built-in (Why, AMD, why? You could add at least a low-end iGPU in your Ryzen line of CPUs except for Ryzen 7, as I'll be getting Ryzen 5 instead).

Anyway, please pardon me for going off-topic here.

[nuetzel]

Thanks everyone about the mention of Synergy. And thanks Niarbeht for the alternative. I will give Synergy a try whenever I decide to build me a new PC with AMD Ryzen. I will be getting a second GPU as the Ryzen processors do not have iGPU built-in (Why, AMD, why? You could add at least a low-end iGPU in your Ryzen line of CPUs except for Ryzen 7, as I'll be getting Ryzen 5 instead).
[snip]

Try Ryzen Mobile then: https://www.phoronix.com/scan.php?pa...n-Mobile-Today
Starting with 4/8 cores/threads + 10 CUs, but 8/16 + XX CUs should be in the works, too.

[nuetzel]

Posting this a second time sorry, seems my post is awaiting moderation due to containing links/urls and I am a new user.

I am Geoffrey McRae (aka, geoff or gnif), just like to point a few things out here the article left out:

1) AMD, this problem has been patched properly thanks to Paolo Bonzini, please see patchwork.kernel.org for patch 10027523 (KVM: SVM: obey guest PAT)
2) This doesn't just affect Ryzen processors, any AMD CPU going back 10+ years that has NPT had this problem.
3) This doesn't just fix VGA Pass through, it corrects the way all pass through devices are configured. You may find devices that prior didn't work such as USB controllers behave properly now, or even achieve better throughput.
4) There is an official subreddit for this, search for "possible_fix_for_the_npt_issue_discussed_on_iommu "

GREAT work, Geoffrey!
Thank you very much for your patience.

BTW Welcome to Phoronix.

[asdfblah]

Unrelated, but...
Did AMD ever find a (microcode, compiler) fix for the segfaults bug?

[GraysonPeddie]

Try Ryzen Mobile then: https://www.phoronix.com/scan.php?pa...n-Mobile-Today
Starting with 4/8 cores/threads + 10 CUs, but 8/16 + XX CUs should be in the works, too.

I wouldn't expect to find mobile Ryzen in ATX motherboard, but thanks.