AMDGPU and virtualization

[DoctorWho]

To bridgman and anyone else who can answer this:

Assume we have a hypothetical hypervisor that has no overhead. Furthermore assume the hypervisor uses the AMDGPU kernel module.
The question is this: Assuming we run Windows, like 7 or 10, in a virtual machine. Can the AMDGPU module provide all the functionality needed to run the Windows driver, Vulkan or Catalyst, at near native speeds (assuming there is no other overhead like memory, disk, etc).

I think my question basically may be does the AMDGPU module provide all the functionality to the hardware so that you can run either Windows or Linux drivers at near native speeds on top of it (again assuming there isn't any other overhead)?

Note: For the purposes of this hypothetical example assume that near native speeds means running a fairly modern game with moderate to high settings.

[DoctorWho]

To clarify , the windows virtual driver should have all the features and functionality as if it was running on bare metal.

[bridgman]

If your question is whether the amdgpu kernel driver exposes the same-ish user/kernel interface as the Windows driver, the answer is "no". If the question is whether a shim layer running over amdgpu could support Windows userspace drivers the answer is "maybe but iffy, the conventions for building & submitting command packets and managing contexts is fairly different between the two".

That said, as hardware takes on more of what was traditionally OS functionality the drivers and interfaces will need to become more similar as a consequence, so the answer may change over time.

[nanonyme]

If your question is whether the amdgpu kernel driver exposes the same-ish user/kernel interface as the Windows driver, the answer is "no". If the question is whether a shim layer running over amdgpu could support Windows userspace drivers the answer is "maybe but iffy, the conventions for building & submitting command packets and managing contexts is fairly different between the two".

That said, as hardware takes on more of what was traditionally OS functionality the drivers and interfaces will need to become more similar as a consequence, so the answer may change over time.

Wonder if you could detect full-screen case though and just start pass-through'ing then. Similarish in semantics but not details to compositor bypass. Most games run in full-screen mode anyway

[bridgman]

I think we are talking about different levels of abstraction here... the actual drawing of content on the screen, not the "wrapper" protocol that manages where content is drawn.

[DoctorWho]

If your question is whether the amdgpu kernel driver exposes the same-ish user/kernel interface as the Windows driver, the answer is "no". If the question is whether a shim layer running over amdgpu could support Windows userspace drivers the answer is "maybe but iffy, the conventions for building & submitting command packets and managing contexts is fairly different between the two".

That said, as hardware takes on more of what was traditionally OS functionality the drivers and interfaces will need to become more similar as a consequence, so the answer may change over time.

CPU have hardware virtualization to speed up guest OS but for graphics hardware it seems like you have to pick one OS to have full features, speed, and functionality and the rest don't get it.

So by what you wrote then I guess that the VMM hypervisor has to probably implement a special interface, not AMDGPU or Windows but something else, that can provide access to low level functions for both Windows and Linux drivers. Assuming someone has the skill and knowledge to write such a driver/interface will the card, for example an M6100, function at near native speed for both Windows and Linux?

EDIT: Googled this just now. Didn't realize people were already working on this problem of GPU virtualization with near native speeds.

[nanonyme]

I think we are talking about different levels of abstraction here... the actual drawing of content on the screen, not the "wrapper" protocol that manages where content is drawn.

Just realized what I was thinking of can't work as you need *something* to expose the GPU context to guest and virgil is probably sanest approach to this thus far

[bridgman]

CPU have hardware virtualization to speed up guest OS but for graphics hardware it seems like you have to pick one OS to have full features, speed, and functionality and the rest don't get it.

Not so much "you have to pick an OS" as "there's a lot of additional work involved", and the work goes up a bit further once you start crossing OSes. On the other hand some GPUs like Tonga have hardware virtualization support via SR-IOV, and that reduces the work quite a bit:

http://www.amd.com/en-us/press-relea...2016feb01.aspx

The HW virtualization support exposes multiple instances of the GPU hardware and state (up to 8 on current HW IIRC), one for each guest, and allows the hypervisor to manage guest access to shared hardware resources. This allows relatively simple VM support (more like pass-through) to still share a GPU between multiple VMs.

So by what you wrote then I guess that the VMM hypervisor has to probably implement a special interface, not AMDGPU or Windows but something else, that can provide access to low level functions for both Windows and Linux drivers. Assuming someone has the skill and knowledge to write such a driver/interface will the card, for example an M6100, function at near native speed for both Windows and Linux?

It's certainly do-able - the most common approach is to hook it at graphics API level (OpenGL or lower) and then use OpenGL on the host machine for easy portability. You can get better performance if you use the same API on both machines but AFAIK the difference isn't too bad even if you use a low level interface on the guest and OpenGL on the host.

[Xaero_Vincent]

I think the more interesting question is will Polaris GPUs still permit being passed into a virtual machine (PCI passthrough) via IOMMU functionality while the Linux host has access to a seperate GPU, likely an integrated one.

[bridgman]

I think the more interesting question is will Polaris GPUs still permit being passed into a virtual machine (PCI passthrough) via IOMMU functionality while the Linux host has access to a seperate GPU, likely an integrated one.

You can do that with any GPU, can't you ? IIRC the question is more whether enabling the IOMMU causes problems with any other drivers, eg we ran across some network stacks which didn't make the right DMA API calls to function correctly with IOMMU enabled.