Three Ways to Run AI Beyond Nvidia’s Chips

SAN JOSE, Calif., Oct. 20 — InIn two days, three companies moved to let artificial-intelligence models run on chips other than Nvidia’s. On Tuesday, at the PyTorch Conference here, Google made TorchTPU open source; it runs PyTorch, a widely used AI toolkit, on Google’s own chips, called TPUs. Amazon announced TorchNeuron, which does the same for its Trainium chips, on Monday and released it as an open-source public beta on Tuesday. And DeepSeek, the Chinese AI company, published the engine behind its own service, built to run on Huawei’s Ascend 950 chips as well as Nvidia’s.

The stakes are in the software. Most AI apps are written by default to run on Nvidia’s chips, by TechCrunch’s account, and many of PyTorch’s features were built to work specifically with them.PyTorchMeta, then Facebook, developed PyTorch in 2016 and turned it over to the Linux Foundation, a nonprofit, in 2022. Nvidia’s own platform, CUDA, is what it calls “the software layer that enables applications to harness the power of GPUs.” Moving that work to another chip has meant, in Amazon’s words, “rewriting and revalidating work that took months to perfect.” Google and Amazon are trying to make the switch from Nvidia cheap, and the tools could also draw work to their clouds. Google says TorchTPU runs “on Google Cloud or in your own infrastructure.”

Google’s TorchTPU, co-developed with the PyTorch team at Meta, runs existing PyTorch work on TPUs “with minimal code changes,” Google said. With it, Google wrote, “TPUs join GPUs and CPUs as a native option for PyTorch,” working with tools developers already rely on, among them vLLM, SGLang, Ray and Hugging Face. “At Meta, advancing the frontier of AI requires immense compute scale and deep hardware diversity. We cannot be constrained by a single hardware architecture,” Jana van Greunen, Meta’s director of PyTorch engineering, said in Google’s announcement.

Amazon says TorchNeuron runs PyTorch code on its Trainium2 and Trainium3 chips “with a single configuration change and no rewrite,” and that 85 percent of AI researchers and developers write their code in PyTorch. It is also contributing changes to PyTorch itself, it said, to remove code that ties the framework to specific chips.Amazon’s ChipsAmazon says it has built four chip families since buying the chip designer Annapurna Labs in 2015, and that its custom-silicon business has passed a $25 billion annual revenue run rate.

DeepSeek’s release is a different kind of thing: the machinery behind its own service rather than a bridge for developers. DeepInfer “powers DeepSeek’s large-scale production services,” its GitHub page says, and a single cluster serves 155 trillion tokens a day, by the company’s count. It runs one model, DeepSeek-V4.1-Flash, on Nvidia’s SM100 GPUs, using CUDA, and on Huawei’s Ascend 950, using software called CANN. The Huawei support comes as Washington moves against DeepSeek: on Tuesday the Department of Homeland Security published an order barring federal civilian agencies from buying DeepSeek’s products and ordering them removed from their systems.

The release has gaps. It leaves out pieces a complete service needs, among them a cache that spares the system from recomputing repeated context and load balancing across copies of the model; DeepSeek says those parts are “tightly coupled to our internal infrastructure.” Its published speed benchmarks were measured on Nvidia GPUs, and two of its features work on Nvidia’s chips only. DeepSeek says code for its future models will continue to be open-sourced in DeepInfer.

“None of this means AI app developers will automatically ditch Nvidia’s hardware en masse just because they can now easily move to alternative chips and clouds,” Julie Bort wrote in TechCrunch. Google itself pitched TorchTPU as giving customers “choice” across TPUs, GPUs and CPUs. The test now is whether developers who can move, do.NVDANasdaq204.64▼9.02−4.2%Nvidia · Friday’s session