Red Hat Acquiring Neural Magic To Bolster Open-Source AI Offerings

Red Hat announced today they have signed a definitive agreement to acquire Neural Magic, an AI software company behind the likes of DeepSparse and nm-vllm.

Red Hat is acquiring Neural Magic to bolster their AI offerings and leverage Neural Magic's engineering talent around GPU and CPU inferencing for generative AI "GenAI" workloads.

Red Hat wrote in today's press release:

"Red Hat intends to address these challenges by making gen AI more accessible to more organizations through the open innovation of vLLM. Developed by UC Berkeley, vLLM is a community-driven open source project for open model serving (how gen AI models infer and solve problems), with support for all key model families, advanced inference acceleration research and diverse hardware backends including AMD GPUs, AWS Neuron, Google TPUs, Intel Gaudi, NVIDIA GPUs and x86 CPUs. Neural Magic’s leadership in the vLLM project combined with Red Hat’s strong portfolio of hybrid cloud AI technologies will offer organizations an open pathway to building.
...
Neural Magic’s technology leadership in vLLM will enhance Red Hat AI’s ability to support LLM deployments anywhere and everywhere across the hybrid cloud with a ready-made, highly-optimized and open inference stack."

While Neural Magic does maintain some elements of open-source software, they do also have proprietary components. Once this acquisition is completed by Red Hat, it looks like it may all be open-source. From today's FAQ page:

"Neural Magic’s core offering is based around the vLLM open source technology, with additional proprietary code. Red Hat has long shown its commitment to open-sourcing the technology it acquires, and we have no reason to expect a change in this approach. Our specific plans and timeline will be determined over the coming months."

This is exciting and I've long been a fan of Neural Magic's DeepSparse and have used DeepSparse for CPU benchmarking with its speedy CPU-based AI inferencing. Hopefully under Red Hat's leadership it will more punctually support new Python versions and the like.

The NeuralMagic.com corporate site has already been updated to acknowledge Red Hat's planned acquisition and the "the future of AI is open-source."