AMDGPU-PRO OpenCL Compiler Hacked Into Mesa's Clover

[phoronix]

Phoronix: AMDGPU-PRO OpenCL Compiler Hacked Into Mesa's Clover

While AMD developers worked on the Radeon Gallium3D "Clover" OpenCL support for some time, that really hasn't been the case in years with the AMD's open-source OpenCL effort these days being focused upon their ROCm compute platform. Some within the community though still work on this OpenCL Gallium3D state tracker from time to time and this New Year's weekend is an interesting project pairing Clover with AMD's proprietary OpenCL compiler...

http://www.phoronix.com/scan.php?page=news_item&px=AMDGPU-PRO-Hack-Clover

[tildearrow]

Isn't Vega supposed to be ROCm-only?

[bridgman]

That's a pretty interesting project. Nice work.

Isn't Vega supposed to be ROCm-only?

In terms of what we ship today yes, but there is no technical limitation stopping the use of other stacks.

[matszpk]

I made that feature as an alternative in small level to the ROCm platform. I have RX VEGA and Ivy bridge which doesn't support PCIE atomics. Therefore, I can not use ROCm OpenCL on my hardware. Ofcourse, this hack do not solve many issues of the Clover OpenCL (like, poor support, poor performance in some cases).
A ROCm platform due to PCIE atomics requirements is unusable for some users even if they has a motherboard and CPU that supports PCIE atomics. Some users just can not run sample programs under a ROCm, if a graphics card is in second PCIE slot or PCIE riser. This small effort from me can help run some the OpenCL applications.
However, code generated by AMDOCL2 can be not fully compatible with RX VEGA ISA (some instructions has been removed or their behaviour has been changed), despite the almost code will be working correctly.

[darkbasic]

Some users just can not run sample programs under a ROCm, if a graphics card is in second PCIE slot or PCIE riser.

WTF!? Are you saying that if I cannot use 2 Vega cards to run a computing intensive OpenCL application?

bridgman can you confirm?

[matszpk]

WTF!? Are you saying that if I cannot use 2 Vega cards to run a computing intensive OpenCL application?

Look at: https://github.com/RadeonOpenCompute/ROCm/issues/237

In low-entry or mid-range mobo's, the second PCIEx16 (really PCIEx4) 3.0 slot is connected to southbridge (not to the CPU) and is not suitable for the ROCm platform

[bridgman]

My guess would have been that the second slot was only PCIE Gen2 (rather than "Gen3 but not supporting atomics" which I didn't think was allowed) but IF the second slot is not a properly functioning Gen3 with atomic support then there would be a problem with ROCm support on that slot.

[darkbasic]

My guess would have been that the second slot was only PCIE Gen2 (rather than "Gen3 but not supporting atomics" which I didn't think was allowed) but IF the second slot is not a properly functioning Gen3 with atomic support then there would be a problem with ROCm support on that slot.

https://github.com/RadeonOpenCompute/ROCm/issues/237

It seems slots other than first one are routed via chipset and reject atomics. That's a very big problem.

[bridgman]

I don't think those PCIE lanes are "routed by chipset" - AFAIK they are actually implemented on the chipset rather than the CPU and therefore do not necessarily have the same features as PCIE lanes implemented on the CPU.

In other words while PCIE lanes on "Haswell and up" CPUs have the required support, PCIE lanes on a chipset connected to a "Haswell and up" CPU may not - and in this case it sounds like the B250 does not. It's not clear if any chipsets support atomic ops on their PCIE lanes right now.

I have to admit, the idea of connecting a GPU to chipset-implemented PCIE lanes never occurred to me, except for older systems where all PCIE lanes came from chipset, since the performance would be significantly limited by the shared interconnect between CPU and chipset.

The ROCm documentation does specify in a few places that GPUs should be connected to CPU root ports but I'll see if we can add a more in-your-face note to the effect that PCIE lanes from the chipset should not be used.

To support ROCm programming model, the GPUs are installed in PCIe slots with PCI Express 3.0 or higher capabilities with transfer rates of 8.0 GT/s in either x16 or x8 lanes. The system configuration can have the PCIe slots directly on CPU’s root port or a PCIe switch port. The CPU root must indicate PCIe AtomicOp Completion capabilities and any intermediate switch must indicate PCIe AtomicOp Routing capabilities.

https://rocm.github.io/hardware.html

[darkbasic]

I don't think those PCIE lanes are "routed by chipset" - AFAIK they are actually implemented on the chipset rather than the CPU and therefore do not necessarily have the same features as PCIE lanes implemented on the CPU.

In other words while PCIE lanes on "Haswell and up" CPUs have the required support, PCIE lanes on a chipset connected to a "Haswell and up" CPU may not - and in this case it sounds like the B250 does not. It's not clear if any chipsets support atomic ops on their PCIE lanes right now.

I have to admit, the idea of connecting a GPU to chipset-implemented PCIE lanes never occurred to me, except for older systems where all PCIE lanes came from chipset, since the performance would be significantly limited by the shared interconnect between CPU and chipset.

The ROCm documentation does specify in a few places that GPUs should be connected to CPU root ports but I'll see if we can add a more in-your-face note to the effect that PCIE lanes from the chipset should not be used.



https://rocm.github.io/hardware.html

One way or another AMD cards CANNOT be used for OpenCL computing, because you'll be limited to one card only.