PyTorch 2.0 Now Shipping With Better CPU & GPU Performance

[Mike Frett]

Almost everything using AI is using Python in some fashion. Hardly garbage. ChatGPT, Stable Diffusion...

[coder]

BTW, if you've ever used Matlab or GNU Octave, you should have a look at numpy. You might consider some aspects of it a hack, but it's pretty impressive what they've been able to shoehorn into Python.

[Jabberwocky]

Triton: very wow, true bandwidth, much LLVM. CPUs and AMD GPUs are not supported at the moment, but we welcome community contributions aimed at addressing this limitation.
Me: Fu**** you Nvidia!

[schmidtbag]

I just can't take anything seriously that uses python as a base, even if the gpu and whatever obviously isn't running python. It's like the world has been overrun by this "teach every man woman and child to code" mindset, but then all you end up with is a bunch of rubes making dumb garbage in toy languages. Has anything really been gained?

Go ahead and uninstall Python from any *nix system you use and come back to us about your experience.
Better hope you don't use Dropbox, Facebook, Netflix, Spotify, Reddit, Pinterest, Instagram, various Google services (including Youtube), and so on, because they all use Python.

[brucethemoose]

Triton: very wow, true bandwidth, much LLVM. CPUs and AMD GPUs are not supported at the moment, but we welcome community contributions aimed at addressing this limitation.
Me: Fu**** you Nvidia!

PyTorch is backed by Facebook, so you can thank them :P

And TBH performance on anything but the AMD 7000 series or the datacenter GPUs would be bad anyway, as is the case with the Vulkan/MLIR backend for Stable Diffusion.

[Weasel]

I just can't take anything seriously that uses python as a base, even if the gpu and whatever obviously isn't running python. It's like the world has been overrun by this "teach every man woman and child to code" mindset, but then all you end up with is a bunch of rubes making dumb garbage in toy languages.

Amen.

[Jabberwocky]

PyTorch is backed by Facebook, so you can thank them :P

And TBH performance on anything but the AMD 7000 series or the datacenter GPUs would be bad anyway, as is the case with the Vulkan/MLIR backend for Stable Diffusion.

Facebook using another/new domain specific language :-O hahaha.

Is the reason for RDNA 3 doing well because of better traditional compute or does this specific backend use FP16 WMMA?

My grievance is idealistic. Companies, big and small, hobbyists, everyone for the past decade have been writing vendor specific code. People aren't just writing apps they wrtie entire libraries, documentation etc... Now we even have multiple vendor specific languages. There's a point in time where you develop expand and then decide if you want to turn some useful things into standards, we are many years past that point.

Currently Nvidia seems untouchable. AMD can compete but as you've mentioned the barrier to entry is sky high. I don't think it's unrealistic for Intel or other vendors to take the crown in AI workloads in the next decade. The problem is that the PTX ecosystem still is growing which makes the scene very anti-competitive. This issue has been repeated many times in the past on hardware and software. E.g. Macromedia Flash (which is still used today). Direct3D11 async issues still effecting most popular games today. The cost of staying with and then later moving away from Xeon CPUs prior to Zen/ARM improvements (Lack of PCIe lanes in Intel was a big oef!). We all know supporting monopolies isn't very wise.

Why everyone and their dog needs to make their own MLIR layer codegen I cannot comprehend. In a few years people are going to say, yeah we have to rewrite this or that stack from scratch because it was designed for legacy environments. Maybe I'm just too ignorant to understand MLIR. I can see the appeal with PoC prototyping or a university experiment. Robust long term maintainable solutions surely more fundamental intrinsics are needed?

Currently I am compelled to use PyTorch-DirectML / Antares. Maybe IREE works too, I have not tried. I'm trying to remember which life choices I made that got me here.

Rusticl is really good for Linux, but the industry needs something that everyone can use.
Sylkan is an interesting test, it could be useful for making things work until there are better solutions. https://dl.acm.org/doi/fullHtml/10.1145/3456669.3456683
OpenSYCL, is very limited in terms of hardware support. It makes an attempt to work on Windows at least:

Building the OpenMP runtime fails for LLVM 11 and probably 12, if using MSVC, but seems to be fixed on main in this commit. Building the runtime separately, using Clang might work, otherwise, it also is possible to just copy libomp.dll and libomp.lib from a binary distribution of LLVM to <INSTALL_PREFIX>/bin and <INSTALL_PREFIX>/lib folders respectively

By the time you are done with that your rivals using Nvidia hardware has finished their project.

We should have taken this approach: Make it work, Make it right, Make it fast.

A decade ago thanks to Khronos and Nvidia *****ing up. Khronos make it right, AMD made it work, AMD made it fast. Nvidia refused to expose their already working SVM. Intel did nothing at the time, they didn't have hardware that would make use of it.

Now a decade later we are taking this approach: Make it work, make it fast. (This is my opinion about MLIR).

At some stage we have to go back and try to: Make it work, Make it right, Make it fast.

From an outsider SPIR-V looks like it has a lot of potential, but Khronos needs to sit with industry partners just like they did with Vulkan.

Looking forward to a day that we focus towards cross-platform vendor neutral compute implementations be that for AI, rendering, crypto, physics or other workloads.