Sam Altman: AI needs attention span, not higher IQ

This article summarizes an episode of Databricks’s video series featuring its CEO Ali Ghodsi, and OpenAI CEO Sam Altman.

OpenAI CEO Sam Altman/ Photo credit: Antonello Marangi / Shutterstock

According to Sam Altman, CEO of OpenAI, the measure of an AI’s ability in a business is how long it can work on its own on a task before its chance of success drops below 50 percent. This perspective changes what automation means and shows the main problem for most businesses.

Intelligence is not enough

Adding a powerful AI to a business won’t solve everything. Even a brilliant model can be useless without understanding how the company works, its past information, and how its teams work together.

The genius with no context
Altman explains, “You could bring a very brilliant physicist and drop them into [a company] and maybe they wouldn’t get that much done on the first day. They’d be missing all of that understanding even though they had crazy intelligence.”

Giving AI the right information
Ali Ghodsi, co-founder and CEO of Databricks, argues the answer is inside the company itself.

Ghodsi notes, “There’s this data that’s not available to the LLMs, which we now can bring together. Providing the data that [companies] have that’s proprietary in the enterprise as context to the agents is going to be the big unlock.”

A new way to measure AI

Recognizing this context gap revealed a more accurate way to measure performance. The goal became making AI work longer on hard problems.

A system to measure AI agents

OpenAI’s progress can be seen in a simple system based on how long a model can work on a task and be likely to succeed.

• Level 1 (GPT-3.5): Agents can successfully complete tasks that take approximately 5 seconds.
• Level 2 (GPT-4): Agents can successfully complete tasks that take approximately 5 minutes.
• Level 3 (GPT-5): Agents can successfully complete tasks that take approximately 5 hours.
• Level 4 (Future State): Agents can successfully complete tasks that take months or even years.

The 50% success rule
Altman outlines, “The way that I think about this is, for a particular class of task… [we ask,] ‘For what length of task does the model have a 50% chance of success?’ We’ve gone from 5-second tasks at the launch of GPT-3.5, to 5-minute tasks with various GPT-4 iterations, to 5-hour tasks with GPT-5. That’s remarkable.”

Automation is moving beyond code

As agents get better, they change how whole departments work. The focus is changing from just doing technical tasks to automating the entire thinking process behind the work.

Engineering is more than just syntax
Take software engineering. People think it’s all about writing code, but building software for a business is more complicated.

The surprising reason adoption is slow

Unlocking previously impossible tasks