Rusticl Capable Of Running Tinygrad For LLaMA Model

Written by Michael Larabel in Mesa on 14 July 2023 at 06:05 AM EDT. 18 Comments
Mesa's Rusticl OpenGL implementation written in Rust it turns out can already run the Tinygrad open-source software with its OpenCL back-end for running the LLaMA model.

Tinygrad is the deep learning framework out of Tiny Corp that is run by George Hotz. He's been working on Tinygrad with an emphasis on using the AMD ROCm compute stack but it turns out Mesa's Rusticl OpenCL state tracker for Gallium3D is good enough for Tinygrad too.

Red Hat's David Airlie has begun testing and profiling Tinygrad both with the ACO compiler back-end and the AMDGPU LLVM back-end. In the coming week Airlie is also likely to test Rusticl against the ROCm OpenCL back-end for seeing how the Mesa performance is compared to AMD's official Linux open-source GPU compute stack.

Airlie's testing has been with a Radeon RX 6700 XT graphics card thus far. He's found using Rusticl with ACO for Tinygrad is around four times faster at compiling than the AMDGPU LLVM back-end but the run-time performance is lower and less optimized -- at least until the ACO experts get to optimizing it further for compute.

Details for those interested on Airlie's blog.

tinygrad logo

Those wanting to learn more about Tinygrad can do so via tinygrad on GitHub.
Related News
About The Author
Michael Larabel

Michael Larabel is the principal author of and founded the site in 2004 with a focus on enriching the Linux hardware experience. Michael has written more than 20,000 articles covering the state of Linux hardware support, Linux performance, graphics drivers, and other topics. Michael is also the lead developer of the Phoronix Test Suite, Phoromatic, and automated benchmarking software. He can be followed via Twitter, LinkedIn, or contacted via

Popular News This Week