What Google’s chief scientist says AI teams get wrong

This article summarizes an episode of Y Combinator’s video series featuring Jeff Dean, Google chief scientist.

Photo credit: Shutterstock

In AI, the most successful companies test new ideas faster than their competitors.

Google chief scientist Jeff Dean argues that the next major advantage lies in high-volume, automated experimentation.

This means using AI to measure progress, test many options, and integrate the best results into products quickly.

Automated experiments drive new research

Companies cannot scale research by hiring more people. Instead, when results are easily measurable, AI systems can run their own experiments, learn from failures, and compound small victories into massive breakthroughs.

This requires a fast, repeatable testing cycle:

• Set clear, often measurable goals to guide decisions.
• Break large goals down into smaller, individually testable problems.
• Treat failed tests as inexpensive learning opportunities.
• Use AI models to estimate results when running a full physical test takes too long.
• Combine small successes into overall system improvements.

Dean views this as an automated version of the scientific method. The loop of proposing, running, and evaluating experiments speeds up discovery when it runs on its own.

“Instead of [a simulation] taking a night, they made [a learned validator] that was 300,000 times faster and nearly as accurate,” Dean explains. This speed allows teams to screen millions of options over a lunch break.

Hardware limits product potential

Faster testing creates new delivery challenges. Complex AI programs run continuously in the background, requiring significant computing power while remaining highly responsive to the user.

Dean notes that hardware will drive adoption, as speed and energy efficiency dictate how widely these tools can be deployed.

“You’re going to see more and more high-performance and low-energy inference hardware systems,” Dean notes, emphasizing that fast response times are critical for interactive AI.

A company with highly efficient custom chips can deliver faster responses or charge less for the same service.

High energy costs restrict interactive AI

AI agents require a clear work environment