AMD Begins Posting More KFD Patches For Discrete GPUs

[bridgman]

OpenCL doesn't actually require coherency except for (IIRC) fine-grained system SVM... I thought we already had OpenCL running over graphics paths for those systems.

[fusionS]

That would be the famous amdgpu-pro which doesn't seem to cover all APUs, especially not embedded systems ones (merlin falcon, steppe eagle, eKabini). But, maybe those has been added now?

[Jumbotron]

I have two AMD Bristol Ridge APU ( Carrizo refresh ) based computers. An HP Pavilon desktop and a Lenovo 310 laptop. Just last night I had to reinstall Gnome Ubuntu 17.04 after playing around with Ubuntu 17.10 ( 17.10 is still way wonky with Wayland bugs and more importantly Synaptic package manager is completely broken on 17.10 and that's a deal breaker for me ).

Anyway....at the end of the 17.04 install and at the point of rebooting the Lenovo laptop it threw up these three lines of code.

kfd kfd: error getting iommu info. is the iommu enabled?
kfd kfd: Error initializing iommuv2 for device (1002:1318)
kfd kfd: device (1002:1318) NOT added due to errors

( Ignore the device numbers....this is from someone else's bug report. I didn't screen cap my messages in time....just remember this is what mine said about kfd and iommu )

I don't remember at the time when I originally installed 17.04 on the HP Pavilion desktop in April if this error message came up as well. I may reinstall 17.04 on the desktop just to see if it does.

I found a bug report on Red Hat Bugzilla ( Bug # 1404139 ) to Red Hat from 2016 detailing someone else's issue with kfd iommu and Red Hat closed it a year later with an insufficient data reason for not fixing it.

https://bugzilla.redhat.com/show_bug.cgi?id=1404139

Also found a SUPPOSED fix ( you know if you read it on the internet it HAS to be true ) .....but not sure about this at all....

0 down vote

FOUND FROM A GOOGLE SEARCH: https://ubuntuforums.org/showthread.php?t=2275207 UEFI boot was a red herring. The real problem was that you need a kfd.rules file in /etc/udev/rules.d . The file must say: KERNEL=="kfd", MODE="0666" Installing mainline 3.19 in Ubuntu 14.10 created this file during package installation (we think?), but Ubuntu 15.04 does not create this file by default. You must do it by hand. Then the HSA software stack works! Problem solved.

[fusionS]

That is lack of IOMMU error. Bios is not enabling IOMMU. Check if HP provides a bios setting to enable it.

[Jumbotron]

Thanks. The Lenovo laptop threw the error. Can't remember if the HP desktop did it back in April. But I imagine it did too. Anybody have any ideaa about that kfd flag above?

[Marc Driftmeyer]

Eventually, the OpenCL Stack from amdgpu-pro will be usable in a default Mesa config w/o manually installing ubuntu deb files out of the stack of deb files from the pro driver. Apps like Blender, Natron, GIMP, RawTherapee, Darktable, etc., which leverage OCL will see it and leverage your GPU. Currently, Blender nighly leverages OpenCL on both the CPU/GPU using the amdgpu-pro stack for compute on my Debian Sid system.

[Imroy]

Currently, Blender nighly leverages OpenCL on both the CPU/GPU using the amdgpu-pro stack for compute on my Debian Sid system.

Any tips on getting this working? The AMDGPU-PRO OpenCL ICD hasn't worked on the stock kernel for a while now, and the DKMS module doesn't compile either. Are you merging something into the stock kernel, patching the DKMS module, or using an old kernel? Or something else?

[odedgabbay]

I have two AMD Bristol Ridge APU ( Carrizo refresh ) based computers. An HP Pavilon desktop and a Lenovo 310 laptop. Just last night I had to reinstall Gnome Ubuntu 17.04 after playing around with Ubuntu 17.10 ( 17.10 is still way wonky with Wayland bugs and more importantly Synaptic package manager is completely broken on 17.10 and that's a deal breaker for me ).

Anyway....at the end of the 17.04 install and at the point of rebooting the Lenovo laptop it threw up these three lines of code.

kfd kfd: error getting iommu info. is the iommu enabled?
kfd kfd: Error initializing iommuv2 for device (1002:1318)
kfd kfd: device (1002:1318) NOT added due to errors

( Ignore the device numbers....this is from someone else's bug report. I didn't screen cap my messages in time....just remember this is what mine said about kfd and iommu )

I don't remember at the time when I originally installed 17.04 on the HP Pavilion desktop in April if this error message came up as well. I may reinstall 17.04 on the desktop just to see if it does.

I found a bug report on Red Hat Bugzilla ( Bug # 1404139 ) to Red Hat from 2016 detailing someone else's issue with kfd iommu and Red Hat closed it a year later with an insufficient data reason for not fixing it.

https://bugzilla.redhat.com/show_bug.cgi?id=1404139

Also found a SUPPOSED fix ( you know if you read it on the internet it HAS to be true ) .....but not sure about this at all....

0 down vote

FOUND FROM A GOOGLE SEARCH: https://ubuntuforums.org/showthread.php?t=2275207 UEFI boot was a red herring. The real problem was that you need a kfd.rules file in /etc/udev/rules.d . The file must say: KERNEL=="kfd", MODE="0666" Installing mainline 3.19 in Ubuntu 14.10 created this file during package installation (we think?), but Ubuntu 15.04 does not create this file by default. You must do it by hand. Then the HSA software stack works! Problem solved.

Thanks for pointing that.
I replied to that bug.
As fusionS mentioned, the root cause is most probably the IOMMU being disabled in the BIOS.

[Jumbotron]

After some extensive searching on this subject I found out a few more things. First of all it looks like my HP Pavilion desktop DOES have IOMMU enabled but NOT the AMD kvm module even though Hardware Virtualization is available, it is NOT enabled nor even an option in BIOS. That should only effect me using KVM and QEMU properly, NOT HSA.

The thing is...even my BIOS did NOT show anything about IOMMU at all. I only found out about it being enabled from a blog post at scottlinux.com He posted a way to enable IOMMU on Fedora. I use GNOME Ubuntu so I gave it a shot anyway.

Open Terminal and do this.....

sudo virt-host-validate

This should return a BIG page on everything you need to properly run a virtual session ( KVM, QEMU, XEN.....)

HOWEVER....when I first ran this it threw up an error message stating that I did NOT have the libvirt libraries installed so I installed them and then re-ran sudo virt-host-validate

After that I got a BIG page of every setting needed to run virtual sessions and whether they passed or failed. It passed everything EXCEPT for finding either an Intel KVM module or AMD KVM module ( think Intel VT-d or AMD Vi). As I said....hardware virtualization is NOT available to me in the HP desktop BIOS. That's ok for now as I was looking for IOMMU enabling. And it did !

Amongst the PASS / FAIL results you should see this if you passed for IOMMU enabling......

QEMU: Checking for device assignment IOMMU support : PASS
QEMU: Checking if IOMMU is enabled by kernel : PASS_

If you did....great. If you did NOT....there still is a way to possibly enable IOMMU by editing GRUB. However...at this point I will just leave you all the link to the blog post because it was for enabling Intel's IOMMU and NOT AMD and my AMD IOMMU is already enabled. Therefore I do NOT know if editing GRUB will work for this instance.

https://scottlinux.com/2017/05/10/how-to-enable-iommu-support-in-fedora-linux/

So....apart from the Hardware Virtualization NOT being available in BIOS, my HP desktop with AMD Bristol Ridge APU DOES have IOMMU enabled for things such as HSA. I haven't tried this on my Lenovo 310 laptop which also has an AMD Bristol Ridge APU. This is the computer that threw up the IOMMU error message at the end of re-installing GNOME Ubuntu 17.04. I will run sudo virt-host-validate on the Lenovo and see what happens. Will let everyone know what I find later.

[Jumbotron]

Ok....finally worked on my Lenovo Ideapad 310. After loading the libvert libraries as these are needed to perform sudo virt-host-validate to determine if IOMMU is enabled or not
( along with finding out if hardware virtualization is also available )

< chuckle >..... and this time unlike my HP Pavilion desktop hardware virtualization WAS turned on. BUT....unlike the HP desktop IOMMU IS NOT available in BIOS nor is it enabled.

sudo virt-host-validate threw up this error message on the Lenovo.......

WARN: No ACPI IVRS table found, IOMMU either disabled in BIOS or not supported by this hardware platform.

And I confirmed in BIOS that there is NO mention of IOMMU....only hardware virtualization which was turned on. Both the HP and the Lenovo have nothing about the IOMMU in their respective BIOS. However, the HP has its IOMMU enabled none the less but hardware virtualization turned off. The Lenovo was just the opposite. IOMMU was NOT enabled but hardware virtualization WAS enabled.

So....if I want HSA goodness I use the HP desktop. If I want Hardware passthrough in a virtual session I use the Lenovo laptop. Well....I got what I paid for....only $250 U.S. brand new off of Ebay for each of those machines. Two low end machines just to keep the kids going in their school work and a cheap work laptop for me. If I need BOTH hardware virtualization AND IOMMU I'm going to have to spend more money and build my own rig with a motherboard that can be confirmed to have both functions available in BIOS.

Still sucks though that AMD gets short shrift when it comes to hardware OEMs. I can't imagine that it would cost that much more to have a BIOS with the ability to activate IOMMU AND hardware virtualization with AMD CPU's that are capable of both.

BTW.....is there a way on computers such as my Lenovo laptop that does not have IOMMU enabling in BIOS to be able to GRUB a software IOMMU on AMD APU's such as Carrizo and my Bristol Ridge APU ?