Harnessing Incredible AI Compute Power Atop Open-Source Software: 8 x AMD MI300X Accelerators On Linux

[sophisticles]

Just to be clear, even though there are no benchmark results in this article, Michael was able to run a few tests?

I am inferring this from the fact that he seems to have been impressed with what he experienced and unless you actually run a few benchmarks and have some known results to compare to then there is nothing to be impressed by.

Also, and this goes for the NVIDIA hardware that was tested remotely not to long ago, unless I am able to physically confirm the configuration of the hardware, I tend not to trust what I am being told by an interested party regarding configuration.

AMD gave remote access for 3 days but there's really no way of know how the system was configured, Michael is relying on AMD being honest about how many Instincts there were and he is relying on a software readout as to how much power is being drawn.

I'm the type of guy that wants to confirm with my own eyes how many cards there are and use a hardware meter at the outlet to monitor power draw.

Maybe AMD can supply a single server with a single Instinct card for testing and Michael can then draw more accurate conclusions.

[ms178]

Once upon a time, with GCN, many issues found and fixed in server products have benefited consumer products from AMD as well. And consumers could enjoy server-grade features on consumer cards (e.g. fp64 performance). The good old days.

[muncrief]

Well, using something a company setup in a cloud doesn't mean anything to me, even if benchmarks were included.

What is impressive is when companies give or loan reviewers hardware to setup and configure themselves, and then benchmarks comparing that hardware to competitors offerings are published.

[varikonniemi]

8*750W for the compute units could heat a large house in the middle of arctic winter.

[geerge]

Once upon a time, with GCN, many issues found and fixed in server products have benefited consumer products from AMD as well. And consumers could enjoy server-grade features on consumer cards (e.g. fp64 performance). The good old days.

Man the Radeon VII was and still is a beast.

[darkbasic]

This is awesome, but with consumer hardware it's a completely different experience: https://github.com/ROCm/clr/issues/61

[Jabberwocky]

8*750W for the compute units could heat a large house in the middle of arctic winter.

That's why Sam Altman (OpenAI) and Satya Nadella (Microsoft) are investigating nuclear power dedicated for their compute.

OpenAI is estimated to use 1GWh per day which isn't nearly enough for them.

[coder]

George Hotz is putting real skin and expertise in the game.

That doesn't give you license to be an asshole on social media.

He also has exposed glaring weaknesses in AMD’s engineering and has publicly and rightly called them out.

When Valve found weaknesses in AMD's software stack, they didn't just make some outrageous tweets and issue public ultimatums because they're the almighty Valve and AMD should be counting its blessings to have them as a partner! No, Valve rolled up its sleeves and quietly got to work!

Hotz might be smart, but he's also a toxic narcissist. By catering to his demands, AMD is basically inviting copycats and we can basically expect a small army of vocal AMD detractors to amass and overwhelm AMD's ability to cope, on either the PR front or tech support.

This will only help all us other “noobs” who want to personally pursue GPU compute for not only the lulz but for personal needs and personal knowledge and skill set improvements without having to take out a second mortgage to build out a personal MI300 rig or rent space on a cloud

If you're doing enough training that it's more cost-effective to buy a multi-RX 7900 XTX rig... I guess I doubt you're really just into it as a hobbyist. I think his customers are mostly small businesses and institutions, some of whom could certainly afford to rent the resources they need.

[gukin]

Typo fixed.

[quaz0r]

Hopefully sooner than later we get to the point where desktop/workstation chips are more like a scaled-down mi300. Memory a mile away and GPUs hanging out of PCI slots is definitely not the future.