Qualcomm MSM Driver With Linux 5.19 Adds DSC, Preps For Mesa Driver Within A VM

[phoronix]

Phoronix: Qualcomm MSM Driver With Linux 5.19 Adds DSC, Preps For Mesa Driver Within A VM

MSM DRM driver and Freedreno creator Rob Clark continues leading the charge on open-source Qualcomm Adreno graphics/display support for Linux in this effort that started out as a reverse-engineering project years ago. This past week Rob sent in the last batch of MSM Direct Rendering Manager driver updates intended for the Linux 5.19 kernel...

https://www.phoronix.com/scan.php?page=news_item&px=MSM-For-Linux-5.19-Changes

[Quackdoc]

the virtualized native driver thing will be so cool if other vendors decide to adopt this. ofc the various vmms need to adopt it as well.

[robclark]

the virtualized native driver thing will be so cool if other vendors decide to adopt this. ofc the various vmms need to adopt it as well.

I hope we end up seeing this at least for other mesa drivers. It is not transparent to the usermode driver in the guest, so it is a thing that might not be so easy to add if you don't have the usermode driver src code. (But OTOH, you really don't want it to be a transparent emulation of the kernel uabi.. for running in a VM, the host<->guest synchronization is expensive so you want the interface from guest usermode driver to host kernel mode driver to be as asynchronous as possible.. the tradeoffs are different compared to host kernel <-> host usermode.) But the performance wins are well worth it.

As far as various VMMs.. crosvm is further along than qemu.. but what is needed for native guest usermode drivers is pretty much the same as what is needed for venus (api virtualized vk) at this point, so hopefully qemu will catch up on the required virtgpu features.

[Quackdoc]

...But the performance wins are well worth it.

As far as various VMMs.. crosvm is further along than qemu.. but what is needed for native guest usermode drivers is pretty much the same as what is needed for venus (api virtualized vk) at this point, so hopefully qemu will catch up on the required virtgpu features.

I certainly find it quite exciting, I have been looking at purchasing an arm laptop recently for the cost efficacy of them. It would also be really nice to see this take off in qemu for sure as that is what I am most familiar with. though maybe I should try giving crosvm a try again and see if I can actually get it to work properly this time.

though I am curious, should either driver support video acceleration, whether it be vulkan video or gallium based one, is it safe to assume that they would be usable within the guest too? I am aware of crosvm having some kind of virtio device for video decode/encode, but I know of it's existence not much more. and guest video acceleration is something I find to be quite an appealing feature. especially if it is something that could be implemented as... I guess as "natively and generically" as possible would be a good term to use? hmm, not sure the right way to put it... especially if qemu and other VMMs in the future could utilize it too.

regardless though, even if I can't get my hands on a supported device anytime soon, It's still really cool tech to see come to fruition.

[robclark]

I certainly find it quite exciting, I have been looking at purchasing an arm laptop recently for the cost efficacy of them. It would also be really nice to see this take off in qemu for sure as that is what I am most familiar with. though maybe I should try giving crosvm a try again and see if I can actually get it to work properly this time.

I will mention, so that you don't have to find out the hard way.. for vm support on snapdragon things, stick to the chromebooks.. the windows snapdragon laptops boot the kernel in EL1 so linux kvm support is not possible. I'm not entirely sure how VM stuff like WSL2 works on arm laptops, but apparently it is in a way that linux hosts cannot easily support currently. (Maybe someone who is more familiar with kernel kvm side of things has looked into how it works on windows.. I'd be curious.)

though I am curious, should either driver support video acceleration, whether it be vulkan video or gallium based one, is it safe to assume that they would be usable within the guest too? I am aware of crosvm having some kind of virtio device for video decode/encode, but I know of it's existence not much more. and guest video acceleration is something I find to be quite an appealing feature. especially if it is something that could be implemented as... I guess as "natively and generically" as possible would be a good term to use? hmm, not sure the right way to put it... especially if qemu and other VMMs in the future could utilize it too.

regardless though, even if I can't get my hands on a supported device anytime soon, It's still really cool tech to see come to fruition.

On all the arm things, video decode is completely decoupled from the "gpu".. even separate power and clock domains. Which makes sense, there is basically zero overlap. For *rendering* video, like converting from YUV to RGB, deinterlacing, etc.. *that* part is done on the GPU but it can be done with shaders, it doesn't really require tight integration with the GPU. Any more than it makes sense for the camera to be part of the GPU. Other than discreet video cards with separate VRAM, it doesn't make sense for the video decode/encode block to be part of the "gpu".. I think it is just a hold-over from the windows device model established in the early days when all GPUs were discrete pci/etc cards.

That said, crosvm does have virtual video and camera devices. I'm not sure what the state is on qemu. Longer term, at a high level I could imagine that camera and video just end up similar to vm wayland integration where it is just dma-buf (and dma-fence) fd passing between host and guest. Video doesn't really have the high rate of API calls that gl/vk has so I don't expect API level virtualization (vs native usermode driver in guest) to have much difference from performance standpoint.

[Quackdoc]

I will mention, so that you don't have to find out the hard way.. for vm support on snapdragon things, stick to the chromebooks.. the windows snapdragon laptops boot the kernel in EL1 so linux kvm support is not possible. I'm not entirely sure how VM stuff like WSL2 works on arm laptops, but apparently it is in a way that linux hosts cannot easily support currently. (Maybe someone who is more familiar with kernel kvm side of things has looked into how it works on windows.. I'd be curious.)

ah I see, thank you very much. I will certainly have to keep that in mind. though it still leaves a large selection of laptops

On all the arm things, video decode is completely decoupled from the "gpu".. even separate power and clock domains. Which makes sense, there is basically zero overlap. For *rendering* video, like converting from YUV to RGB, deinterlacing, etc.. *that* part is done on the GPU but it can be done with shaders, it doesn't really require tight integration with the GPU. Any more than it makes sense for the camera to be part of the GPU. Other than discreet video cards with separate VRAM, it doesn't make sense for the video decode/encode block to be part of the "gpu".. I think it is just a hold-over from the windows device model established in the early days when all GPUs were discrete pci/etc cards.

ah I see, I had never thought about that before, I was loosely aware that they weren't the same thing, but I didn't realize they were so separated on arm. I guess it makes sense.

That said, crosvm does have virtual video and camera devices. I'm not sure what the state is on qemu. Longer term, at a high level I could imagine that camera and video just end up similar to vm wayland integration where it is just dma-buf (and dma-fence) fd passing between host and guest. Video doesn't really have the high rate of API calls that gl/vk has so I don't expect API level virtualization (vs native usermode driver in guest) to have much difference from performance standpoint.

I remember reading about virtio-video possibly getting an implementation, I think it was a thread on the mailing list 2 years ago now(?) though I'm not sure if anything came of it sadly. one of the things I was worried about is not having a generic interface when exposed to the guest, but now that I have awoken, I guess since it is a virtio protocol and all, I wonder if it has a libva backend?

in any case, thanks for the explanations, it was very enlightening.