The State Of ROCm For HPC In Early 2021 With CUDA Porting Via HIP, Rewriting With OpenMP

[qarium]

Yep, I have the same concern - the ROCm documentation has grown huge, and it seems like it has grown past the point of maintainability without chaining an architect or two to the documentation task. I'm strongly in the "documentation has to be small enough to be kept accurate and relevant" camp but that increasingly feels like a minority view (not just inside AMD).
I'll try to find out who is maintaining the docco these days and pass this along.

there is simple fix for this: if you mainline something in upstram just drop the legacy support.

and just work on this task: "It uses an NVidia/Mellanox API (which is what the NICs support) that requires application memory pages to be pinned in place from userspace. I think the recent upstreaming of GPU P2P dmabuf may allow us to re-implement in an upstreamable form, at least that was one of the goals for the work IIRC."

to make it upsteam with the DMABUF.

people will like it.

[smitty3268]

AMD has never understood there is a big market of scientists, grad students, small software companies that are buying FLOPs on a budget - three Radeon VII's instead of one Radeon VII Pro gets you more FLOPs and they don't care about ECC and enterprise features. This is what nVidia understood - AMD on the other hand just had to EOL the VII and go for the hard upsell to the VII Pro. AMD could have kept the VII around at $700 and grown it's market instead...

I think that presumes that AMD actually wanted to sell lots of VII cards, which I'm not certain is the case. I think they viewed it more as an experiment with TSMC's 7nm node, and something they could show shareholders that was a bit less uncompetitive with Nvidia than their other cards at the time. But I think I saw somewhere that AMD likely wasn't making much (any?) money on those cards, so it makes sense that they'd only really want to sell them for higher prices - enter the pro version.

As someone who doesn't really care that much about graphics compute, I don't really have a bone to pick with AMD in this but I will say as someone who roots for AMD it's sad.

I'd really like to be able to say they are competitive in compute, but the truth is if I was making purchasing decisions for a company that needed to use gpu compute I wouldn't even consider using AMD. Nvidia is the only viable option at the moment, and it's been that way for a while. I gave up on AMD and OpenCL a year ago, and the situation is essentially the same now as it was then. If (and i do mean IF, not when) AMD ever gets their ROCm platform into decent shape then it may make sense to reconsider, but they're a lost cause right now.

Intel's Xe graphics cards might be the best hope for anyone wanting to have competition in the market vs Nvidia, but it remains to be seen how they'll stack up hardware-wise. I think it's pretty obvious their software side will be better than AMD though.

[qarium]

Intel's Xe graphics cards might be the best hope for anyone wanting to have competition in the market vs Nvidia, but it remains to be seen how they'll stack up hardware-wise. I think it's pretty obvious their software side will be better than AMD though.

I bett money on AMD against intel. they hire people soon their software will be better.

the hardware of intel will not be competive. end of story.

[bridgman]

My point about the VII Pro was that Bridgman thinks it is an entry level compute card, but really the VII was the entry card. Compute is also about FP32, despite HPC focus on FP64. You can do a lot with 32 bit floats. AMD has never understood there is a big market of scientists, grad students, small software companies that are buying FLOPs on a budget - three Radeon VII's instead of one Radeon VII Pro gets you more FLOPs and they don't care about ECC and enterprise features. This is what nVidia understood - AMD on the other hand just had to EOL the VII and go for the hard upsell to the VII Pro.

I think we understand it, but we haven't had a chance to spend money on it yet. That is changing though.

AMD could have kept the VII around at $700 and grown it's market instead.

I don't think $700 would have been enough, unfortunately, unless we were willing to lose money in order to grow a fairly small market (which might still have been a good idea, just saying it's not the no-brainer you suggest).

Radeon VII was not something we could make money on at consumer pricing but it did fill a real gap in our gaming product line at the time and was worth selling for a while just to have a higher performance card in the lineup. I was surprised we didn't offer it from day one as a workstation product though - part of the reason for being expensive was HBM, but another part was that it was the first full-RAS GPU we had made since Tahiti and Carrizo.

[qarium]

I don't think $700 would have been enough, unfortunately, unless we were willing to lose money in order to grow a fairly small market (which might still have been a good idea, just saying it's not the no-brainer you suggest).
Radeon VII was not something we could make money on at consumer pricing but it did fill a real gap in our gaming product line at the time and was worth selling for a while just to have a higher performance card in the lineup. I was surprised we didn't offer it from day one as a workstation product though - part of the reason for being expensive was HBM, but another part was that it was the first full-RAS GPU we had made since Tahiti and Carrizo.

I am not an insider but i also think 700€ was a money loser for AMD. without raytracing making more money was impossible
the 6900XT makes 1550€ in a time AMD sells it for 1000€ this proof that only raytracing can rise the profitability for AMD in the consumer market.
i see no different between radeon7 and 6900XT both are the highend gpu cards at their time but one costs 700€ lose money for AMD the other makes 1550€ makes profit for AMD.

anyone with a brain should accept this that AMD can not sell products they lose money on.

"I was surprised we didn't offer it from day one as a workstation product though"

for me this was clear at day 1... the 2080 had raytracing to in fact the only benefit for a radeon7 was to be a compute card.
no real gamer missed the raytracing feature at that time but many compute people did not care about raytracing at all.

sometimes it looks like the managers at these companies like AMD sleeping to not see this in the same clear way.
but this is history and we can't change the history we can only change the future.

in my point of view AMD can only do this: sell a radeon7 pro with 4096 shaders and 32gb vram and if some cheaper version is in need drop the ECC feature. look what he said: he run deep learning stuff on radeon7 it run out of VRAM... so 32gb is needed.

and ROCm is not ready for RX5700 and 6800/6900XT yet this means there is still demand in radeon7 pro cards.

but see what he wrote he thinks he can get ECC for free... insane... you can only build a radeon7 pro without ECC to make it cheaper.

[bridgman]

The problem is that ECC is not something that can be easily left out - every memory / register file, every functional block, every data path... pretty much everything has ECC or parity on it.

[bridgman]

in my point of view AMD can only do this: sell a radeon7 pro with 4096 shaders and 32gb vram and if some cheaper version is in need drop the ECC feature. look what he said: he run deep learning stuff on radeon7 it run out of VRAM... so 32gb is needed.

and ROCm is not ready for RX5700 and 6800/6900XT yet this means there is still demand in radeon7 pro cards.

but see what he wrote he thinks he can get ECC for free... insane... you can only build a radeon7 pro without ECC to make it cheaper.

The problem is that ECC is not something that can be easily left out - every memory / register file, every functional block, every data path, HBM controllers, PCIE bus interfaces... pretty much everything has ECC or parity on it along with all of the control logic to manage it and gracefully handle errors at any point.

Vega20 also has all of the Infinity Fabric logic that we use on the workstation and datacenter SKUs.

[qarium]

The problem is that ECC is not something that can be easily left out - every memory / register file, every functional block, every data path, HBM controllers, PCIE bus interfaces... pretty much everything has ECC or parity on it along with all of the control logic to manage it and gracefully handle errors at any point.

Vega20 also has all of the Infinity Fabric logic that we use on the workstation and datacenter SKUs.

yes and i like it... i could had buy 128gb ram for my system for 400€ but i spend 600-700€ to get ECC...
i really don't unterstand why these people don't want ECC... i think they don't really unterstand how important ECC is.

ok lets adress his VRAM problem... could AMD release a Radeon7 PRO with 32gb vram ?

apple sells VEGA20 with 32GB vram... so it should not be a problem if APPLE not bought the exklusive right for it.

[qarium]

The problem is t

and please make a DDR4/DDR5 GPU with "Infinity cache" it should perform well in my point of view
and the DDR4/DDR5 could mean that you can put cheap DDR4 ram on it 128GB for like 400€

you have to accept sometimes people do not need fast ram instead they need a lot ram.

like he said he was sad because his 16gb VRAM card run out of memory...

people like this would be really happy with a DDR5 GPU instead of GDDR6 or HBM2.

for sure the "Infinity cache" is needed to make it performant but AMD already have the "Infinity cache".,

[bridgman]

ok lets adress his VRAM problem... could AMD release a Radeon7 PRO with 32gb vram ?

apple sells VEGA20 with 32GB vram... so it should not be a problem if APPLE not bought the exklusive right for it.

We use the same chip for 32GB MI50/MI60 parts, so a 32GB workstation part should be fine.