Develop 3d interactive simulation with ROCm

[davide445]

First time posting here, also not really a Linux expert, my role is normally to design solutions and shape the tech and financial feasibility, so please pardon me if I'm not accurate enough.

Preparing a project proposal of a new 3d immersive interactive simulation solution, needing to realtime compute (i.e. updates every 0.1 sec) fluid dynamics using LBM/SPH models and soft body dynamics using FEM.

We have our 10+ years CUDA expert and will also collaborate with EU research institution, these having access to future AMD powered super but in fact so far porting their CPU code to CUDA.

For innovation interest, hate of vendor lock-in, cost reasons I was exploring the possibility to adopt ROCm/HIP in this project.

Wanted to kidy ask your ideas on these topics
- there is so far no interoperability btw ROCm/HIP and DX/OpenGL/Vulkan. But Blender just released Cycles X that have some form of interactivity.
- ROCm 5.0 plan adding official support oy to Radeon Pro W6800, without mentioning other consumer RDNA2 GPUs. So Blender case might remain a closed source driver with no public API available to make HIP work on i.e. an RX 6900 Xt. There is also no official release date
- even wanting to use HIP on CDNA there is no plan to release modern versions and also no availability on cloud (Azure only has the soon to be deprecated Mi25). CDNA it's not made for graphics and as first point there is no official interop with graphics oriented RDNA cards (Pro or consumer)

[bridgman]

- there is so far no interoperability btw ROCm/HIP and DX/OpenGL/Vulkan. But Blender just released Cycles X that have some form of interactivity.

We have had graphics/compute interop since the HSA days and have carried it forward to the ROCm stack. The API follows the GL/CL interop model where a graphics buffer is made addressable by compute code, since the variety of graphics formats is a superset of the variety of compute formats.

We use this API to support GL/CL interop in the AMDGPU-PRO stack, so most of the testing has been with the closed source GL driver, but we have used it with Mesa in the past and plan to include that in our regular testing at some point fairly soon.

- ROCm 5.0 plan adding official support oy to Radeon Pro W6800, without mentioning other consumer RDNA2 GPUs. So Blender case might remain a closed source driver with no public API available to make HIP work on i.e. an RX 6900 Xt. There is also no official release date

HIP support was added for all of the RDNA parts, although we prioritized RDNA2 for math library code and AFAIK that is in better shape on RDNA2 than RDNA1. I don't know the exact target date for 5.0 but it is paired with 21.50 AMDGPU-PRO (ie should be same ROCm code in both) and 21.50 is likely to come out in late January.

In the short term at least you should stick with the AMDGPU-PRO packages rather than the ROCm packages since we test graphics/compute interop in the AMDGPU-PRO stack.

Note that AFAIK Blender has only released HIP support for Windows in 3.0, while Linux support should be added in 3.1.

- even wanting to use HIP on CDNA there is no plan to release modern versions and also no availability on cloud (Azure only has the soon to be deprecated Mi25). CDNA it's not made for graphics and as first point there is no official interop with graphics oriented RDNA cards (Pro or consumer)

I don't understand the statement about "there is no plan to release modern versions" - current HIP releases all include CDNA support and releases are frequently planned around addition of CDNA support & features.

re: CDNA / RDNA interop can you give me a bit more context about the configuration you have in mind ? Are you thinking about CDNA card in a cloud and graphics card on your system (eg remote visualization), or are you thinking about having the cloud provider pairing a graphics and compute card on the same cloud node and using interop there ?

Certainly remote visualization is a common use case but that normally uses some kind of data stream between the two nodes rather than traditional compute/graphics interop.

[davide445]

HIP support was added for all of the RDNA parts, although we prioritized RDNA2 for math library code and AFAIK that is in better shape on RDNA2 than RDNA1. I don't know the exact target date for 5.0 but it is paired with 21.50 AMDGPU-PRO (ie should be same ROCm code in both) and 21.50 is likely to come out in late January.

In the short term at least you should stick with the AMDGPU-PRO packages rather than the ROCm packages since we test graphics/compute interop in the AMDGPU-PRO stack.

Note that AFAIK Blender has only released HIP support for Windows in 3.0, while Linux support should be added in 3.1.

Good to know but as my writing I didn't found any official note stating ROCm 5.0 will be available also on consumer RDNA2 cards such as the RX 6900 XT.
But possibly I simply misst the pont about AMDGPU-PRO including ROCm.
We are interested into Windows support more than Linux due our 3d app being developed on Windows, even if in case of necessity we can also switch the dev on Linux.

I don't understand the statement about "there is no plan to release modern versions" - current HIP releases all include CDNA support and releases are frequently planned around addition of CDNA support & features.

re: CDNA / RDNA interop can you give me a bit more context about the configuration you have in mind ? Are you thinking about CDNA card in a cloud and graphics card on your system (eg remote visualization), or are you thinking about having the cloud provider pairing a graphics and compute card on the same cloud node and using interop there ?

Certainly remote visualization is a common use case but that normally uses some kind of data stream between the two nodes rather than traditional compute/graphics interop.

A typo, this keyboard is creating a lot of problems and I was not able to modify the message. Was meaning that I didn't see the release to the retail channel of the modern Instinct PCIe Mi100 or Mi210, at least not as single item and not part of an OEM system. On the contrary I can let's say just purchase an A100 for my private testing.
At least just now I was able to track down a cloud provider supplying a bare metal machine with a Mi100 to test on.

The CDNA/RDNA topic it's just since in our app we will need to sim and visualize 30-40 millions of voxel data in realtime, so that was thinking will be better to split the simulation on CDNA and rendering on RDNA.

[davide445]

bridgman there is any support of Vulkan on CDNA2

[agd5f]

bridgman there is any support of Vulkan on CDNA2

CDNA2 GPUs have no graphics pipe. In theory you could write a vulkan driver that only exposed compute and transfer queues, but I'm not sure off hand if that is allowed by the spec.

[davide445]

CDNA2 GPUs have no graphics pipe. In theory you could write a vulkan driver that only exposed compute and transfer queues, but I'm not sure off hand if that allowed by the spec.

In the end with my great regret seems CDNA is completely unsuitable for our needs and NV is the only real option. Good to know anyway.