IOMMUFD Submitted For Linux 6.2 To Overhaul IOMMU Handling

[phoronix]

Phoronix: IOMMUFD Submitted For Linux 6.2 To Overhaul IOMMU Handling

After being in various forms of discussion since 2017, IOMMUFD has been submitted for the Linux 6.2 kernel as it lays the groundwork for aiming to overhaul IOMMU handling by QEMU and virtual machines on Linux...

https://www.phoronix.com/news/IOMMUFD-Linux-6.2

[erniv2]

Ähh the IOMMU in itself is allready a protection mechanism so that hardware cant directly dma to ram but has to go through those tables.

What they want to do here is that a VM can actually have a deeper more direct access to the IOMMU reducing overhead for devices the example here was qemu.

SO i guess they want to soften the boundary kernel/userspace but since it´s still a specific /dev and a kernel api they can get away with it, thats my layman understanding.

Edit: the userspace can send commands directly to specified cpu cores/pci/pcie devices thats how i see it.

Edit2: and that will probably ring some alarmbells well i dont know how all that pci passthrough voodoo works but thats probably part of it.

[erniv2]

Well i dont get hot and bothered about it anyway i still have that sr-iov setting in my uefi and it still does not work and you dont find hardware supporting it anyway ......

This maybe some prep work for a new supercomputer useing thousends of rtx 4090s

[Quackdoc]

im still trying to figure out if virtio-iommu can be used to safely use ACS override.

[Developer12]

This seems like the kind of thing that would be more at home in a microkernel. Neat.

[Developer12]

A microkernel can use this feature the same way as a monolithic kernel does. The only real difference is that in a microkernel, any application is by default almost running in a virtualized environment.

Since most applications do not need to manipulate the IOMMU, I don't see any special benefits of this tech in a microkernel (compared to a monolithic kernel). I am not claiming that there isn't any difference, but I just can't figure out what would that significant difference be.

In a microkernel every driver is running in userspace as an application. This implementation in linux is explicitly to allow hardware devices to DMA into userspace process memory, and DMA is core to how *all* drivers work. Linux drivers can get by using DMA from kernelspace, but for a microkernel this is absolutely necessary for decent driver performance.

[Developer12]

No, I don't see it that way. Perhaps I have misunderstood the article, or the purpose of IOMMUFD, because, honestly, I don't understand half of the low-level stuff mentioned in the article.

If a userspace process (microkernel or otherwise) wants to allow a hardware device to DMA into the memory of the process, the userspace process can simply ask the kernel to setup the required IOMMU mapping (hardware device to userspace memory). In this scenario, there is abolutely no need for the userspace process to directly access the IOMMU hardware.

My understanding is that the point of IOMMUFD is to allow a userspace process to directly access the IOMMU hardware. That functionality is, basically, only required for VMs.

This new facility in linux is to enable userspace processes to request DMA directly into their address space in the first place. There isn't really any other way to do it, at least not generically.

Nobody ever gets direct access to an IOMMU. That would be insanely dangerous.

[Developer12]

No, what you are saying appears to me as completely wrong.

As strange as it might sound, a userspace process can access an IOMMU directly, if the IOMMU implements hardware access protections. It is the same as with CPUs: an IOMMU can also have a "privileged mode", and a "userspace mode". But, not all IOMMUs have a separate "userspace" mode implemented (although I would guess that all the new ones do have it implemented).

So, there can be three types of IOMMUs:
type 1. "ordinary" IOMMU – has no separate "userspace" mode
type 2. "tree-structured" IOMMU – has no separate "userspace" mode, but the translation tables are arranged in a tree-structure
type 3. "hardware-virtualizable" IOMMU - has a separate "userspace" mode and a tree-structure

Your claim that the purpose of IOMMUFD is to "enable userspace to request DMA into the userspace" is false, because all types of IOMMUs can already do this without IOMMUFD (but, with a small amount of assistance from the kernel instead).

Instead, the article clearly states that the purpose of IOMMUFD is to enable VMs to more easily manage an IOMMU of type 3 (and possibly also of type 2, but I couldn't confirm this). Here are some quotes from the article (emphasis mine):

"aiming to overhaul IOMMU handling by QEMU and virtual machines on Linux"
"for performance, it has become quite popular to setup virtual machines so that HW (eg networking/storage devices) can have direct DMA into the virtual machine memory itself "
"A vIOMMU driver that can implement vPASID and vPRI to achieve vSVA within a VM "
"direct control over the IOMMU to implement, in userspace, a vIOMMU device emulation that can provide all these services to the IOMMU driver running in the VM "

Also note: drivers in a microkernel are not VMs. That's why a microkernel driver doesn't need the functionality of IOMMUFD.

You're missing that VMs have previously had access to various virtualization extensions.