Red Hat acquires AI optimization startup Neural Magic

Red Hat, the IBM-owned open source software firm, is acquiring Neural Magic, a startup that optimizes AI models to run faster on commodity processors and GPUs.

The transaction is subject to applicable regulatory reviews and other customary closing conditions. The terms of the deal weren’t disclosed.

MIT research scientist Alex Matveev and professor Nir Shavit founded Somerville, Massachusetts-based Neural Magic in 2018, inspired by their work in high-performance execution engines for AI.

Neural Magic’s software aims to processes AI workloads on processors and GPUs at speeds equivalent to specialized chips. By running models through off-the-shelf processors, which usually have more available memory, the company’s software can realize speedups.

Big tech companies like AMD and a host of other startups, including NeuReality, Deci, CoCoPie, OctoML, and DeepCube, offer some sort of AI optimization software. But Neural Magic is one of the few with a free offering, and a collection of open source tools to complement it.

Prior to the Red Hat acquisition, Neural Magic managed to raise $50 million in venture capital from backers like Andreessen Horowitz, New Enterprise Associations, Amdocs, Comcast Ventures, Pillar VC, and Ridgeline Ventures.

Red Hat CEO Matt Hicks says Neural Magic’s work on vLLM, an open source project for model serving, is of particular interest. With Neural Magic, Hicks says that Red Hat’s getting a vLLM-based “enterprise-grade” stack that lets customers optimize, deploy, and scale custom and pre-built models across cloud environments with control over infrastructure and security policies.

Red Hat is already involved in the vLLM project, which it leverages to run models in products like Red Hat Enterprise Linux AI and Red Hat OpenShift AI. With Red Hat and Neural Magic, Hicks says that infrastructure partners will be able to better scale AI across platforms and accelerators; integrated service provider partners will gain more robust inference and performance to integrate with their offerings; and OEM partners will be able to take advantage of improved open source infrastructure for AI.

“AI workloads need to run wherever customer data lives across the hybrid cloud; this makes flexible, standardized and open platforms and tools a necessity, as they enable organizations to select the environments, resources and architectures that best align with their unique operational and data needs,” Hicks said in a statement. “We’re thrilled to complement our hybrid cloud-focused AI portfolio with Neural Magic’s groundbreaking AI innovation, furthering our drive to not only be the ‘Red Hat’ of open source, but the ‘Red Hat’ of AI as well.”

Subscribe for the industry’s biggest tech news