AMD ROCm 6.0 Now Available To Download With MI300 Support, PyTorch FP8 & More AI

[Djip007]

https://www.phoronix.com/forums/forum/linux-graphics-x-org-drivers/open-source-amd-linux/1426660-amd-details-the-mi300x-mi300a-announces-rocm-6-0-software?p=1429009#post1429009

Needed link:

https://www.forbes.com/sites/karlfreund/2023/12/13/breaking-amd-is-not-the-fastest-gpu-heres-the-real-data/?sh=64db064c3a6f

It looks like ROCm 6.0 is incompatible with AMD's iGPUs: https://rocm.docs.amd.com/projects/i...u-install.html



That means you have to disable iGPU on Ryzen 7000 desktop series.
So, it will be impossible to use ROCm 6.0 with (AMD GCN5.1 or AMD RDNA2 or RDNA3) iGPU + (AMD GCN5.1 or AMD RDNA2 or RDNA3) dGPU on laptops.
That means all modern laptops are incompatible with ROCm 6.0.
Possibly it works OK with GCN5.0 - ROCM just ignores it.
Otherwise it will be impossible to use ROCm 6.0 with AMD iGPU + AMD dGPU on laptops.
Also that means needless for patches that enable Blender + HIP on APUs.
I can suggest to use older ROCm.

for me it is "only" :

impossible to use ROCm 6.0 with AMD iGPU + AMD dGPU.

(laptop or desktop)
old time ago I made a path on kfd driver to sellect which GPU is use for rocm : https://github.com/ROCm/ROCK-Kernel-...ment-743742192
Didn't test that from long time nor update the patch for newer Kernel.
Note: last test was 3 year ago so not sur if we can made same type of patch.
But pretty sur it is not possible to deactivate iGPU on laptop, the iGPU is use for display/screen connection (well if you did it you don't have display anymore)

But now MI300A is a APU... it may change in future.

[oleid]

Ah, I thought that you tested it with these changes and saw that 2.6x speed increase.

Nope, I was merely pointing out that you missed information from the video.

[bog_dan_ro]

So far not, only the video you shared.

Ah, I thought that you tested it with these changes and saw that 2.6x speed increase.

[oleid]

[...] only cuda + hipify [...]

If cuda comes with such libraries as well, then that is enough. If not, well, then software changes are required to use those libraries.

I have no idea about ROCm's "graph stuff" , if you have more info please share it.

So far not, only the video you shared.

[Panix]

I know there are still challenges ahead, but I would like to thank the entire Linux team at AMD for the work you are doing.

What work are they doing?

[Panix]

https://www.phoronix.com/forums/forum/linux-graphics-x-org-drivers/open-source-amd-linux/1426660-amd-details-the-mi300x-mi300a-announces-rocm-6-0-software?p=1429009#post1429009

Needed link:

https://www.forbes.com/sites/karlfreund/2023/12/13/breaking-amd-is-not-the-fastest-gpu-heres-the-real-data/?sh=64db064c3a6f

It looks like ROCm 6.0 is incompatible with AMD's iGPUs: https://rocm.docs.amd.com/projects/i...u-install.html



That means you have to disable iGPU on Ryzen 7000 desktop series.
So, it will be impossible to use ROCm 6.0 with (AMD GCN5.1 or AMD RDNA2 or RDNA3) iGPU + (AMD GCN5.1 or AMD RDNA2 or RDNA3) dGPU on laptops.
That means all modern laptops are incompatible with ROCm 6.0.
Possibly it works OK with GCN5.0 - ROCM just ignores it.
Otherwise it will be impossible to use ROCm 6.0 with AMD iGPU + AMD dGPU on laptops.
Also that means needless for patches that enable Blender + HIP on APUs.
I can suggest to use older ROCm.

Most AMD users I talk to say they can't use their gpus w/ ROCm 5.7 or 6 - there's always some problem or problems* (plural). Kudos to AMD with their FANTASTIC LINUX SUPPORT! LMAO!

Uh oh, AMD fanboys are gonna be mad at me now! /s

[bog_dan_ro]

Thanks! So that 2,6x speed up in vLLM seems to occur when using optimised libraries (attention and graph stuff) that come with ROCm. Question is if llamacpp is using them as well.

If you're talking about PyTorch's flash attention, then the "optimized" code was reverted a few days ago .
Also, if you check PyTorch's srcs (and most of the AI libs/apps out there that supports ROCm e.g. llama.cpp, mlc-llm, etc.), you'll see that, sadly, there is no ROCm code, but all of them are using CUDA + hipify...
I have no idea about ROCm's "graph stuff" , if you have more info please share it.

The point is that so far 2,6x speed up in LLMs is not visible...

[darkbasic]

It should at least work with rusticle, should it not?

Sort of... you need at least 64GB of ram to use it with darktables: https://gitlab.freedesktop.org/mesa/mesa/-/issues/9959

[Svyatko]

https://www.phoronix.com/review/amd-...ct-mi300-rocm6 ?
https://youtu.be/pVl25BbczLI?si=poMTbfa-TsQB_yhU&t=1684 ?

https://www.phoronix.com/forums/forum/linux-graphics-x-org-drivers/open-source-amd-linux/1426660-amd-details-the-mi300x-mi300a-announces-rocm-6-0-software?p=1429009#post1429009

Needed link:

https://www.forbes.com/sites/karlfreund/2023/12/13/breaking-amd-is-not-the-fastest-gpu-heres-the-real-data/?sh=64db064c3a6f

It looks like ROCm 6.0 is incompatible with AMD's iGPUs: https://rocm.docs.amd.com/projects/i...u-install.html

Warning

ROCm currently doesn’t support integrated graphics. Should your system have an AMD IGP installed, disable it in the BIOS prior to using ROCm. If the driver can enumerate the IGP, the ROCm runtime may crash the system, even if told to omit it via HIP_VISIBLE_DEVICES.

That means you have to disable iGPU on Ryzen 7000 desktop series.
So, it will be impossible to use ROCm 6.0 with (AMD GCN5.1 or AMD RDNA2 or RDNA3) iGPU + (AMD GCN5.1 or AMD RDNA2 or RDNA3) dGPU on laptops.
That means all modern laptops are incompatible with ROCm 6.0.
Possibly it works OK with GCN5.0 - ROCM just ignores it.
Otherwise it will be impossible to use ROCm 6.0 with AMD iGPU + AMD dGPU on laptops.
Also that means needless for patches that enable Blender + HIP on APUs.
I can suggest to use older ROCm.

[paulocoghi]

I know there are still challenges ahead, but I would like to thank the entire Linux team at AMD for the work you are doing.