Google open sources TorchTPU to run PyTorch on TPUs

By VK

Published 21 Oct 2026, 10:58 UTC

Reporting from TechCrunch, Hugging Face

Google released TorchTPU, a native PyTorch backend for its TPUs, a day after AWS shipped TorchNeuron, widening the push against Nvidia.

What it means for founders

Your compute bill now has two credible challengers to Nvidia. With Google and AWS both offering PyTorch paths onto their own chips, a team spending heavily on GPUs can run a real benchmark on TPUs or Trainium and bring the numbers to its next pricing conversation.
Portability is cheap insurance. Keep model code free of hardware specific calls and lean on libraries such as Transformers and vLLM that already support several chips. That helps keep a GPU shortage or price rise from becoming an outage.
Ask for migration help in writing. Both clouds want your workload on their own silicon, so ask Google Cloud and AWS what credits or engineering support they will commit before you move a training or inference job.
Watch the test pass rate and the open issue list. If the remaining TorchTPU issues close and the TPU results on the Transformers CI dashboard turn green, that is a sign the stack is maturing for production inference.

The story

Google released TorchTPU as open source on Tuesday, October 20, at PyTorch Conference 2026. It is a native PyTorch backend that treats the company's Tensor Processing Units as an ordinary device, next to GPUs and CPUs. The announcement says the code was built together with Meta's PyTorch team, and the repository is under the Apache 2.0 license, with release 0.2.0 on PyPI.

The timing matters. AWS shipped TorchNeuron for its Trainium chips one day earlier, a release Enki covered in AWS opens TorchNeuron beta to run PyTorch on Trainium. TechCrunch framed the two launches as parallel attacks on Nvidia, whose GPUs remain the default target for most AI apps.

What Google shipped

Developers debug in eager mode, then switch on torch.compile when they want speed. The backend handles data and tensor parallel training, and Google wired it into widely used open serving and training projects: vLLM with Inferact, SGLang with RadixArk, plus llm-d 1.0, TorchTitan, VeRL and Ray. Google claims TorchTitan hit model FLOPs utilization as high as 52% when training Qwen 3 8B on Ironwood TPUs.

Two extras stand out. A TPU agent suite, now in preview, installs into agent tools such as Claude Code and Antigravity, with support for Cursor and Codex promised later, and walks users through onboarding, profiling and kernel tuning. Google also brought LiteRT, its on-device runtime, into the PyTorch ecosystem for phones, laptops and embedded hardware.

Hugging Face drops its workaround

Hugging Face says the older torch_xla path was hard to debug, with lazy tensors and recompilations set off by small edits. Now a Transformers v5 model lands on a TPU by passing a single device argument, Diffusers pipelines run unchanged, and the old Optimum TPU layer is no longer needed. Hugging Face reports that 96.5% of the Transformers tests it runs on TPU pass, and says 19 of the 34 issues it filed against TorchTPU are fixed.

Meta's director of PyTorch engineering, Jana van Greunen, said in Google's post: "We cannot be constrained by a single hardware architecture."

What we don't know yet

How TPU pricing compares with GPU rental for the same job, since Google published utilization figures but no cost per token or per training run.
How much real code changes when a team moves an existing GPU project, beyond the Hugging Face libraries.
When the agent suite leaves preview and starts accepting outside contributions.