Google splits its next TPU in two: 8t for training, 8i for inference

Google's latest TPU announcement is a useful signal about where AI infrastructure is heading: the one-chip-fits-all era is giving way to specialization. With its eighth-generation TPUs, Google is splitting the line into TPU 8t for training and TPU 8i for inference. That is not just product segmentation. It is an admission that agent-heavy workloads now pull hardware in two different directions at once: massive model training on one side, low-latency multi-agent serving on the other.

The training chip is the blunt-force machine. In the announcement, Google says TPU 8t is built to cut frontier model development cycles from months to weeks, with nearly 3x compute performance per pod over the previous generation. A single superpod scales to 9,600 chips, two petabytes of shared high-bandwidth memory and 121 exaflops of compute. Google also says it paired the system with 10x faster storage access and software including Virgo Network, JAX and Pathways to push toward near-linear scaling at far larger cluster sizes.

TPU 8i is tuned for the opposite problem: keeping inference responsive when many specialized agents are calling one another, sharing context and bouncing through long workflows. Google says 8i pairs 288 GB of high-bandwidth memory with 384 MB of on-chip SRAM, triples on-chip SRAM versus the prior generation, doubles interconnect bandwidth to 19.2 Tb/s and uses a new Collectives Acceleration Engine that can reduce on-chip latency by up to 5x. The message is clear: in an agent market, wasted milliseconds become a systems problem, not just a UX problem.

The engineering language here is dense, but the commercial meaning is simple. Labs and cloud vendors are no longer optimizing only for bigger single-model benchmarks. They are building for continuous loops of reasoning, retrieval, tool use and inter-agent coordination. Google even says the designs were created with Google DeepMind to handle the demands of agentic workloads and evolving model architectures at scale. That is a stronger statement than "faster chips." It is a blueprint for how cloud AI stacks are being rearranged around autonomous software behavior.

Availability still matters: Google says both chips are coming later this year, so the real test will be how much of this performance shows up outside launch-stage examples. But the spec sheet already tells a story. Training hardware is being stretched toward ever larger shared memory pools and goodput targets above 97%, while inference hardware is being reworked to kill the waiting room effect in live agent systems. This launch is less about one chip drop than about the new shape of AI compute.