ZLUDA Takes On Third Life: Open-Source Multi-GPU CUDA Implementation Focused On AI

The open-source ZLUDA project began life as a drop-in CUDA replacement that ran atop Intel GPUs using the Level Zero API. Then AMD quietly began funding it for several years as a viable CUDA implementation running atop AMD GPUs until discontinued funding earlier this year. ZLUDA for AMD GPUs was then made open-source but then in August the ZLUDA code was removed at AMD's request. Today it's taking on its third incarnation.

ZLUDA developer Andrzej Janik has continued working on the project albeit from a clean start before AMD's sponsorship/involvement to avoid any tainted source code. He's been working to secure project sponsor(s) and has now been successful in doing so. There is an unnamed organization that is sponsoring ZLUDA development moving forward but for now they want to be a "stealth" sponsor, so Andrzej Janik hasn't revealed the organization. The unnamed organization shares Janik's vision for the future of ZLUDA.

ZLUDA is being rebuilt to focus on multi-GPU vendor support and will take a particular emphasis on machine learning / AI type workloads. Previously ZLUDA was more focused on enabling professional creator workloads while now it will be more focused on CUDA-based AI/ML software.

The hopes are for getting Llama.cpp, PyTorch, TensorFlow, and other AI/ML software using NVIDIA code paths to work atop ZLUDA. Among the early work being pursued with this newest ZLUDA rewrite is improving the NVIDIA PTX parser and other changes with his experience and insights gained from the prior ZLUDA implementations.

While ZLUDA development moving forward is focused on supporting multiple GPU architectures, most development at least initially is being done with AMD GPUs. The new ZLUDA code will be focused on RDNA1 and newer support along with ROCm 6.1+ compute stack support.

ZLUDA's development will remain open-source and welcome community contributions. In a conversation with Andrzej ahead of his planned announcement for today, he thinks it will take about one year to get the new ZLUDA code up into decent shape though could take longer for enabling different applications.

Track the progress moving forward of ZLUDA development via the ZLUDA GitHub. Of course, as ZLUDA hits new milestones you'll be able to read about it on Phoronix. For now the ZLUDA sponsor remains stealth but will be interesting to find out more about that organization when revealed.