Harnessing Incredible AI Compute Power Atop Open-Source Software: 8 x AMD MI300X Accelerators On Linux

[S.Pam]

As mentioned I tested it in AMD's cloud, so I am unable to take pictures of the physical server... And usually the companies aren't too open to taking pictures inside their data centers.

Similarly, no idea about total power... The server might expose it via IPMI but I didn't have root hardware access in the cloud.

Thanks for clarification. I misread the article. I thought you had gotten the physical unit for a short while, and that you mentioned amd also has the cloud thing.

Nevertheless, my takeaway is how hugely power hungry all this ai stuff is....

[AmericanLocomotive]

Well I suppose it's obvious now why ROCM doesn't seem to work so well on consumer cards - it seems they've hyperfocused on making sure the top-end Instict stuff works. Which is fair enough I guess, given how much these parts cost and the potential quantities involved.

But there's a reason why CUDA-capable cards became so popular, and that's because just about anyone could get CUDA working at home. It quickly spread to researchers tinkering around with the home and office rigs, and next thing you know it exploded in popularity. AMD should not forget about supporting consumer cards - even if they might not accel at certain math operations.

When Valve found weaknesses in AMD's software stack, they didn't just make some outrageous tweets and issue public ultimatums because they're the almighty Valve and AMD should be counting its blessings to have them as a partner! No, Valve rolled up its sleeves and quietly got to work!

Hotz might be smart, but he's also a toxic narcissist. By catering to his demands, AMD is basically inviting copycats and we can basically expect a small army of vocal AMD detractors to amass and overwhelm AMD's ability to cope, on either the PR front or tech support.

AMD is a $22B/year company.

They shouldn't need some video game company and some random guy on the internet to write their drivers and software stack for them. AMD officially endorses and supports their open source stacks. They should be the ones largely responsible to make sure they work good.

It's one thing for the community to roll their owner drivers if they're making a weird esoteric custom open-source operating system for hobby use. But at this point, we have real companies, selling commercial hardware, utilizing AMD's open-source driver stack. They need to step up. The number of people and companies who are not AMD and/or not employed by AMD that are working on AMD's drivers and software is absurd. Valve is literally paying other companies to improve AMD's software.

It's nuts.

AMD should be paying every single person who has touched or contributed to the AMD kernel driver and the open-source userspace component that essentially every distro (including the Steamdeck) uses.

[Kimmono]

This is awesome, but with consumer hardware it's a completely different experience: https://github.com/ROCm/clr/issues/61

AMD RX 570 on Gentoo running on IBM Power 9?

[stiiixy]

But there's a reason why CUDA-capable cards became so popular, and that's because just about anyone could get CUDA working at home. It quickly spread to researchers tinkering around with the home and office rigs, and next thing you know it exploded in popularity. AMD should not forget about supporting consumer cards - even if they might not accel at certain math operations.

I remember seeing CUDA as an option on HP desktop workstations in 2007. No discrete cards, but the BIOS offered it regardless. A bare bones government MS Office machine.

Accessibility is the king. Flip a switch, and enable the power, should you choose to accept it.

And here now AMD has a great foundation (fabrique) to make it happen at a cost effective, and controlled rate, and it would be interesting to know just how fast they could apply section's of hard silicon to early adopters, without making themselves liable.

Bypass emulation style pre-emptive testing like Qe.u and get straight to the goodies.

And create a by default of any 'failed' sections in the future should they (the testing silicon) not pan out. Or make bilug fixes in future driver revisions, I guess. Whatever works.

[ms178]

AMD is a $22B/year company.

They shouldn't need some video game company and some random guy on the internet to write their drivers and software stack for them. AMD officially endorses and supports their open source stacks. They should be the ones largely responsible to make sure they work good.

[...]

It's nuts.

I agree. It is even more absurd with Red Hat and Collabora writing open source drivers for Nvidia which is even a slightly bigger and more profitable company.

[darkbasic]

AMD RX 570 on Gentoo running on IBM Power 9?

Yes. Rusticl works fine in most workloads, but no chance with ROCm.

[coder]

there's a reason why CUDA-capable cards became so popular, and that's because just about anyone could get CUDA working at home. It quickly spread to researchers tinkering around with the home and office rigs, and next thing you know it exploded in popularity.

Almost right. It wasn't home users, but rather university students who used their gaming GPUs to accelerate their research projects.

AMD is a $22B/year company.

But they were teetering on the brink of bankruptcy, only about 7 years ago.

Also, I'm not sure you understand how business works. Just because a company has big revenues doesn't mean they can afford to spend unlimited amounts of money on anything, without even so much as an RoI case.

They shouldn't need some video game company and some random guy on the internet to write their drivers

Valve's contributions go back a long way, well back into that "bankruptcy" era I mentioned. At the time, AMD was basically the only way Valve could get involved, due the sorry state of Nvidia's open source situation and Intel not having a dGPU.

AMD officially endorses and supports their open source stacks. They should be the ones largely responsible to make sure they work good.

They are.

It's one thing for the community to roll their owner drivers if they're making a weird esoteric custom open-source operating system for hobby use. But at this point, we have real companies, selling commercial hardware, utilizing AMD's open-source driver stack. They need to step up.

They did. bridgman said AMD hoped they would get more community engagement, on this front, but ended up having to do most of it themselves.

The number of people and companies who are not AMD and/or not employed by AMD that are working on AMD's drivers and software is absurd. Valve is literally paying other companies to improve AMD's software.

What about Sony and Microsoft? Do you think they spend a lot of money customizing and optimizing software for the AMD GPUs in their consoles? The main difference with Valve is that because Valve runs Linux on Steam Deck, it can contribute its work back upstream.

AMD should be paying every single person who has touched or contributed to the AMD kernel driver and the open-source userspace component that essentially every distro (including the Steamdeck) uses.

First, I'm sure it's not as many as you think. Second, they really shouldn't. People will do what's in their own incentive to do. Not that there's not a certain amount of altruism in open source, but in order to pay someone, you would want to have some direct say in what they're working on and how they go about doing it.

Just to ground this back in reality, what about Asahi or the open source drivers for ARM and Vinte GPUs? Should Apple, ARM, and Verisilicon have to pay people who, of their own personal volition, decided to undertake or contribute to those tasks? And if there are companies involved, you'd better believe they undertook the effort in their own self-interest. Should they have to be paid, too?

Also, as for Valve's efforts... Valve knew the state of AMD's software stack, when it opted to use their hardware. If Valve demanded better quality from the software, don't you think AMD would've had to charge them more, in order to fund the additional software development?

You started out at a good place, with this post. But, then it seems like your sense of righteous indignation overtook your rational capacity and you started beating up the patient in an effort to make them walk.