Harnessing Incredible AI Compute Power Atop Open-Source Software: 8 x AMD MI300X Accelerators On Linux

[phoronix]

Phoronix: Harnessing Incredible AI Compute Power Atop Open-Source Software: 8 x AMD MI300X Accelerators On Linux

A few days ago I had the chance to indulge on an incredible compute nirvana: eight AMD Instinct MI300X accelerators at my disposal for some albeit brief testing. Not only was it fantastic from the shear compute performance, but for Phoronix fans, all the more exciting knowing it's atop a fully open-source software stack from the kernel driver up through the various user-space libraries (well, sans the GPU microcode). This first encounter with the AMD MI300 series was eye-opening in seeing how far the ROCm software stack has come and the increased challenges for NVIDIA going forward with the rising competitiveness of AMD's hardware and software efforts.

https://www.phoronix.com/review/amd-instinct-mi300x-rocm6

[CochainComplex]

Fun ))

[onlyLinuxLuvUBack]

use rust to reverse engineer the Linux nvidia driver

[S.Pam]

Oh nice! Can we get some pictures of the whole server and the insides? Whats the total power draw at the wall?

[Michael]

Oh nice! Can we get some pictures of the whole server and the insides? Whats the total power draw at the wall?

As mentioned I tested it in AMD's cloud, so I am unable to take pictures of the physical server... And usually the companies aren't too open to taking pictures inside their data centers.

Similarly, no idea about total power... The server might expose it via IPMI but I didn't have root hardware access in the cloud.

[ReaperX7]

Too bad TinyCore and George Hotz didn't use Instinct GPUs for their projects and like noobs used off the shelf gaming GPUs for this same workload and had issues. 🤣

[Drago]

Too bad TinyCore and George Hotz didn't use Instinct GPUs for their projects and like noobs used off the shelf gaming GPUs for this same workload and had issues. 🤣

Do you have the slightest idea how much each of these MI300X cost? TinyCore is cost constrained.

[Jumbotron]

Too bad TinyCore and George Hotz didn't use Instinct GPUs for their projects and like noobs used off the shelf gaming GPUs for this same workload and had issues. 🤣

Firstly…that’s kind of a noob statement.

Secondly…what exactly are you doing in the Neural Network space ?

Thirdly…George Hotz is putting real skin and expertise in the game. He also has exposed glaring weaknesses in AMD’s engineering and has publicly and rightly called them out. This will only help all us other “noobs” who want to personally pursue GPU compute for not only the lulz but for personal needs and personal knowledge and skill set improvements without having to take out a second mortgage to build out a personal MI300 rig or rent space on a cloud in order to help pay for a billionaire’s great grand child’s education.

[GPTshop.ai]

I can`t wait for the benchmarks especially Mi300A. Please AMD make it happen. The best way to attack the competition is by showing the performance as is it is to everybody. I promise we will sell AMD hardware too when the performance is there.

[ReaperX7]

The problem is regardless of how you look at it, using off the shelf hardware that isn't optimized for advanced and precision calculations is haphazard and ill conceived.

Everyone and anyone who has used professional grade GPUs knows they're not built for speed, but for precision and accuracy. I have Quadro cards I've used in systems for testing and analytics and they are a different breed than consumer grade cards by a mile. Radeon Pro, Instinct, and even older FireGL cards differ greatly in capabilities than standard Radeon.

I don't have to be doing anything in any space to know why they had issues and were bitching at AMD about their firmware issues.

It's like buying a Ferrari without an engine. You don't buy a Ferrari without an engine, drop just any old engine into it, and expect it to run the same as a Ferrari normally should. Same goes for A.I. which is why Nvidia even refuses to support GeForce cards being used in stuff when Quadro and Tesla style cards should be used. So yes they made a noob decision. They tried to "make it work" in a way that maximized profit over cost and it bit them in the ass. They did what amounts to dropping a VW Bug engine in a Ferrari and expected it to run like a Ferrari. It was a stupid decision and I'm surprised AMD even bothered to work with them.