Google adds TPU 8tx for confidential inference and TPU 8i Max for long-context work

Google Cloud introduces TPU 8tx and TPU 8i Max, two new systems for confidential inference and large context workloads, detailed in a company blog post. The additions expand its eighth-generation portfolio to four systems, alongside TPU 8t for model training and TPU 8i for inference.

TPU 8tx uses the same silicon as TPU 8t and pairs it with custom AMD CPU hosts. It protects data and model weights while they are in use and supports private model tuning, distillation, reinforcement learning, and agent execution on one system. Google says it delivers up to two times lower latency than TPU 8t with a near-zero performance penalty.

TPU 8i Max combines two TPU 8i chips in one package. It provides twice the compute, SRAM, and HBM bandwidth of TPU 8i while fitting twice the silicon into the same rack footprint. The system is designed for workloads involving million-token contexts, large codebases, and hours of video.

Google also said it has open-sourced TPU Sync, a layer that manages model state across chip memory, host memory and storage, and made Cloud Pathways and GKE dynamic slicing generally available. It is the first time Google has introduced four TPU systems in one year.