Replace AMD with NVIDIA for LLMs, Manjaro Linux

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • lichtenstein
    replied
    As an alternative, I could keep the Radeon card and install a 2nd one e.g. a 16G Rtx 4060 to be used for the deep learning stuff e.g. HuggingFace inference. How are the chances of this working out better?

    Leave a comment:


  • lichtenstein
    started a topic Replace AMD with NVIDIA for LLMs, Manjaro Linux

    Replace AMD with NVIDIA for LLMs, Manjaro Linux

    Hey y'all,
    Atm, my Manjaro Linux rig has a Radeon 6900XT. It's a fine card, works well for desktop & gaming.

    What doesn't work well (or, at all) is GPU acceleration for deep learning (I want to run smaller LLMs locally, etc). For that I'd need something like Rocm actually working (it won't build from AUR, the damn thing) + ZLUDA. I'm tired of trying to make all of this work. So I'm thinking of switching to an RTX 4080S or similar.

    With the AMD open source drivers I've never had any issues on Linux desktop and such. What's the current state of the nvidia ones? Can I just put in the nvidia card (install their drivers) and stuff just works e.g. Wayland on KDE?

    What problems can I expect? I use two monitors.
    Last edited by lichtenstein; 02 November 2024, 01:25 PM. Reason: Added the two monitors bit.
Working...
X