AWS builds new cooling system for Nvidia AI GPUs

Amazon Web Services (AWS) has developed new hardware called the In-Row Heat Exchanger (IRHX) to cool Nvidia’s GPUs used for AI tasks.

These GPUs consume substantial energy, prompting AWS engineers to build an in-house cooling system instead of using liquid-cooled data centers.

The IRHX addresses concerns about space efficiency and water use, according to Dave Brown, AWS’s VP of compute and machine learning services.

The IRHX can be installed in both existing and new AWS data centers.

Customers can access this cooling system through AWS P6e computing instances, which support Nvidia’s GB200 NVL72.

🔗 Source: CNBC

🧠 Food for thought

1️⃣ The evolution of GPU cooling reflects the growing thermal demands of AI

AWS’s development of the In-Row Heat Exchanger follows a decades-long progression in cooling technology necessitated by increasingly powerful processors.

In the 1990s, early graphics chips like the Rage Pro (1997) required no dedicated cooling whatsoever, while by 2003, the Radeon 9800 series introduced heatpipes and dual-slot coolers to manage rising thermal output 1.

The industry’s first experiments with liquid cooling appeared around 2005-2007 with products like the Radeon X1950 XTX featuring All-in-One water coolers, highlighting the limitations of air cooling 2.

Today’s AI-focused GPUs generate unprecedented heat, with modern systems reaching thermal design powers of up to 350W per GPU, multiple times higher than previous generations and requiring entirely new cooling approaches 3.

This trend shows how thermal management has become a critical factor in AI infrastructure, pushing even the largest cloud providers to develop custom cooling solutions rather than rely on conventional approaches.

2️⃣ Custom infrastructure has become a competitive necessity in the AI cloud wars

Amazon’s decision to build proprietary cooling hardware continues its long-standing strategy of vertical integration that began with its first custom silicon initiatives around 2012 4.

AWS has progressively developed custom components including the Nitro System for virtualization, Graviton ARM-based processors, and networking equipment, all delivering significant efficiency improvements while reducing dependency on third-party suppliers 5.