TBH, I see zero issues with rocm atm. It installs fine from the repos, you install PyTorch and tell it to use it. And that’s it. It just goes as does the NVIDIA stuff I’ve used on AWS.
-
-
FYI, the most recent Manjaro update broke ollama (0.4.1): it can't find its rocm libs. Maybe fiddling with LD_LIBRARY_PATH would fix it but I found that their official install works just fine. I dumped it into some tmp dir (instead of the suggested /usr), and I run it off a symlink. Works well.
The new qwen2.5-coder model is miraculous (for Python work at least), even the 14B variant. That runs with 33 tok/sec on my GPU (quite usable). The 32B one won't fit so it runs off CPU (some GPU load too) at 5.5 tok/sec (not really fun to use).Comment
Comment