AWS unveils new Trainium3 AI chip

Amazon Web Services has introduced Trainium3, its latest AI chip designed for advanced generative AI applications including reasoning and video generation.

Trainium3 is AWS’s first 3nm AI chip and powers the new Trn3 UltraServers, which the company says deliver up to 4.4x higher performance and 3.9x higher memory bandwidth compared to Trn2 UltraServers.

AWS claims Trainium3 achieves over 4x better energy efficiency than its predecessor, with each chip providing 2.5 petaflops of compute and 144GB of HBM3e memory.

The chip is built for both dense and expert-parallel workloads and supports frameworks such as PyTorch, allowing developers to train and deploy models without code changes.

AWS said early adopters are testing the new chip, but did not name specific companies.

🔗 Source: Amazon Web Services

🧠 Food for thought

Implications, context, and why it matters.

Trainium3 performance claims lack independent checks

AWS says Trainium3 delivers up to 4.4× over Trainium2 and 2.52 petaflops FP8 per chip 1, yet no MLPerf (an industry‑standard AI benchmarking suite) head‑to‑head results versus NVIDIA H100 or AMD MI300 2.
NVIDIA H100 led MLPerf Training v3.0, finishing a GPT‑3 based large language model (LLM) in under 11 min on 3,854 GPUs. Intel’s Habana Gaudi2 needed 311 minutes for the same job 2.
AWS scaled to 1,024 chips but has not shown runs at the size of Microsoft Azure’s 10,752 H100s or Google’s 50,944 Tensor Processing Unit v5e (TPU v5e), so frontier competitiveness is unclear 3.
Without direct results on standard jobs like LLaMA2‑70B training, a 70‑billion‑parameter open‑source LLM, or inference throughput, buyers cannot assess Trainium3 price‑performance versus rivals holding 82 of 90 MLPerf submissions 2.

Custom kernels open niche optimization work

Trainium3’s Neuron Kernel Interface (NKI), part of AWS’s Neuron software stack, offers direct instruction access and an open‑source compiler built on Multi‑Level Intermediate Representation (MLIR) 4.
The General‑Purpose Single Instruction, Multiple Data (GPSIMD) engine speeds up custom operators, reaching 2 TB/sec of memory bandwidth for memory‑bound work like TopK (selecting the k largest values) or LayerNorm (a neural‑network normalization technique) 5.
For AI consultancies and model optimization firms, PyTorch support plus NKI will spur migration projects that port CUDA (NVIDIA’s GPU programming platform) kernels to Neuron using the AWS Neuron software development kit (SDK) 4. Tool vendors can add debugging extensions on Neuron Explorer (AWS’s profiling and visualization tool), similar to Datadog (an observability platform) integrations with the SDK 6.