Meta PM’s playbook for AI-on-AI code reviews

This article summarizes an episode of Lenny’s Podcast’s video series featuring Zevi Arnovitz, a product manager at Meta.

Zevi Arnovitz, product manager at Meta/ Photo credit: Lenny’s Podcast

Zevi Arnovitz, a product manager (PM) at Meta, has created a system where AI models review each other’s code. This method helps people who can’t differentiate code and changes how teams of programmers work.

A method for building products without code

Arnovitz created a step-by-step process for building products without writing code, relying on slash commands in the AI code editor, Cursor.

• /create issue: Records a bug, feature, or idea and sends it to a project management tool like Linear.
• /exploration phase: The AI studies the task, reads the existing code, and asks questions to understand the goal and its limits.
• /create plan: The AI creates a text file that explains how it will do the work, including the steps and technical choices.
• /execute plan: The AI writes the code using the approved plan. Arnovitz uses different models for different tasks.
• /review: The AI model reviews its own code to find the first bugs.
• /peer review: Different AI models review the code, finding problems the first model missed.
• /update docs: After the code is added, the AI updates the instruction documents so other AIs know what changed.

Why AI can’t just ‘wing it’
It can be tempting to let an AI work without a plan, but that often leads to challenges.

Arnovitz says, “Planning is really important when you’re implementing something technical… If the coding agent is just like, ‘All right, I got it,’ and just starts writing code, this always results in terrible things.”

Learning from AI’s mistakes
Even the best models will fail. The important part is to create a process that learns from every mistake, so the system gets better over time.

Arnovitz notes, “When Claude fails to do something… I’ll ask it what in your system prompt or tooling made you make this mistake… [and say,] ‘Let’s update your tooling and documentation so that this mistake never occurs again.'”

The problem with code quality

AI tools can create code instantly, but founders who can’t code have a big problem checking if it’s good and finding small mistakes.

The intimidation of code
For someone who isn’t an engineer, a page of code can be impossible to understand. This makes it challenging to check an AI’s work for themselves or trust the final product.

Arnovitz explains, “If you’re nontechnical like me, code is terrifying. It’s the scariest thing in the world to look at… It’s very difficult for me to catch mistakes.”

The first line of defense
Before sending the code to others for review, Arnovitz does a simple first check. He has the AI that wrote the code check its own work to find easy-to-spot mistakes.

He adds, “After I’ve manually tested it, I’ll have Claude review its own work… What I’ll do is basically /review. And this tells Claude to start reviewing its own code.”

A team of AIs to review code

A new chance for new people

Learning skills is faster with AI