AMD ZenDNN 5.0 Software For AI Delivers "400% Performance Uplift"

While writing about ZenDNN 5.0 back in November the day it was released and then forgetting about it alongside all of the other exciting Linux benchmarking and testing like their AOCC 5.0 compiler, earlier this month AMD put out a blog post formally announcing the ZenDNN 5.0 software library. Not only do they talk up its Zen 5 / Turin CPU support but that ZenDNN 5.0 can provide a 400% performance uplift on average.
Earlier this month AMD engineer Shailen Sobhee outlined the ZenDNN 5.0 release and its ability to provide a 400% performance uplift. Testing across a range of models like Llama 2/3.1 compared to ZenDNN 4.2 yielded a 400% performance uplift on average. This was measured with the ZenDNN plug-in for PyTorch. There were also very nice gains over IPEX 2.4 as Intel's Extension for PyTorch.
In addition to enabling 5th Gen AMD EPYC "Turin" / Zen 5 CPU support, ZenDNN 5.0 delivers on advanced auto-tuning for LLMs, INT4 weight-only quantization, new APIs for generative LLMs, and other optimizations.
More details on the ZenDNN 5.0 software release and its 400% performance uplift via the technical article on AMD Developer Central. I'll be working on some of my own ZenDNN 5.0 independent benchmarks as soon as time allows.
6 Comments