The OpenAI plan: Cheap AI tests will revive old research

This article summarizes an episode of a16z’s video series featuring Mark Sellke, OpenAI mathematician.

Photo credit: Shutterstock

Cheap AI testing lets companies revive old research ideas that once cost too much to explore. Mark Sellke and Meethab Sawhney, a mathematician and a researcher at OpenAI, argue that handing off boring calculations to AI forces teams to rethink how they pick projects.

By using AI models to think through problems, separating big plans from daily work, and having humans check the results, leaders can grow innovation without running out of money.

AI reasoning revives abandoned research

Momentum on promising research ideas historically stalls when concepts require weeks of expensive validation.

Sawhney points out that “it is not uncommon to find out a year or two later that somebody else got the idea to work,” highlighting the frustration of discarding correct theories.

To capitalize on affordable testing and bypass these barriers, research leaders must deploy the following shifts:

• Abandoned idea retrieval: Revisit stalled projects that were previously ignored due to manual effort.
• Route navigation: Allow the system to try a path, change course upon failure, and narrow down options.
• Performance evaluation: Review the summarized chain of thought to ensure the AI applies judgment instead of trying everything.

Isolating strategy from execution

As AI takes on a larger role in problem-solving, teams need new operational boundaries. Because models now utilize judgment to navigate these revived projects, research leaders must implement rules that divide strategy from execution:

• Rely on human supervisors to choose project direction before initiating work.
• Assign lengthy calculations to an AI model to maintain strategic clarity.
• Execute session restarts after a failure to ensure fresh attempts.
• Demand improvements even after the system hits its initial target.

Sellke measures the success of this method by speed, arguing that “if you’re able to solve problems faster by making better judgments, that’s the best proxy I have for taste.”

Shifting human roles toward output verification

While separating strategy from execution improves speed, humans must still manage the resulting data. Generating data is only valuable if a company can explain the results, meaning talent must pivot toward making vast amounts of information usable.

To prevent information overload and maintain an edge, organizations must prioritize these skills:

• Verify complex results. Check AI-generated work to ensure accuracy before integrating findings into daily operations.
• Connect data to goals. Translate model outputs into knowledge for corporate problems.
• Train internal teams. Teach staff how to absorb AI results that are typically clear.