Google to launch new AI chips with separate training, inference

Google said on April 22 it will release separate eighth-generation TPU chips for training and inference later this year.

The company said it is replacing its combined design, as it targets AI agent workloads and offers Google Cloud customers an alternative to Nvidia hardware.

The training chip delivers 2.8 times the performance of its seventh-generation Ironwood TPU at the same price, while the inference chip is 80% faster and carries 384 MB of SRAM, triple Ironwood’s amount.

The move comes as Apple, Microsoft, Meta, and Amazon expand custom AI chip efforts, while Nvidia remains the market leader.

🔗 Source: CNBC

🧠 Food for thought

Implications, context, and why it matters.

Google’s new AI chips already have a large buyer

Google’s new Tensor Processing Units (TPUs), its custom AI chips, come from a long-term effort with Broadcom and Anthropic, the startup behind Claude 1.
Anthropic plans to use about 3.5 gigawatts of TPU computing through Broadcom starting in 2027 2.
Broadcom will turn Google’s TPU designs into chips that can be made at scale. It will also provide networking parts for Google’s next AI server racks through 2031 3.
The deal rests on Anthropic’s rapid growth, with annualized revenue recently passing US$30 billion 2.

A three-company chip setup challenges Nvidia

The arrangement suggests a model where an AI developer like Anthropic, a cloud company like Google, and a chip maker like Broadcom work as a dedicated group 4.
Top AI firms are building custom chips to secure large computing supply and find cheaper, more efficient options than Nvidia graphics processing units (GPUs), which are common in AI work 5.
Broadcom is also working with OpenAI on custom chips. Anthropic still relies on Amazon Web Services (AWS) as its main cloud and AI training partner, including a Trainium 2-based supercluster, a large connected group of servers built for AI processing 3.
These multi-year agreements help Broadcom plan production, though they tie more of its business to the commercial results of a small group of customers such as Anthropic 3.