Why nobody follows your AI coding policy and what works instead

A few months ago, I witnessed a leadership team at a startup revoke its engineers’ AI access after a week of dealing with production fires. But the move changed nothing as junior engineers kept coding using AI on their personal accounts, where no one was tracking them.

I’ve learned the hard way how systems can make or break things. For four years, I led the innovation unit at a food delivery platform where we shipped around 100 solutions in total. That made me see that what makes a tool succeed or fail is usually the system around it.

Image credit: Timmy Loen

Before that startup took AI access away from its engineers, it was rapidly deploying new code as it hurtled toward a launch date for a payments project. Nobody was trying to break things, but the faster the company shipped generated code, the more issues the team had to address.

The new payment integration, mostly vibe-coded, was failing when used by real customers about 27% of the time. Cutting off AI access sounded like the sane thing to do.

But while it addressed the issue, it forced senior developers to work overtime. Worse still, new problems kept emerging. I then realized we wouldn’t get through it without changing the system underneath.

Revoking AI access didn’t change how people got work done. Instead, it just hid the process from view and never revealed why the bad output happened in the first place.

Same pattern at scale

Similar patterns have emerged at scaled enterprises like Amazon. In late 2025, the ecommerce giant made its in-house tool Kiro the default AI assistant and set a target: 80% of developers had to use AI every week, with a leaderboard ranking engineers by utilization.

While creating this environment boosted AI usage, it also allowed unreviewed code to slip through, resulting in several incidents.

In December of that year, Kiro was asked to fix a minor bug in a production system. But instead of patching it, the agent deleted the whole environment and rebuilt it, faster than anyone could read the whole rewrite. This caused a 13-hour outage that Amazon called a “user error.”

Photo credit: bluestork / Shutterstock

In March 2026, AI-linked changes that skipped formal review took down the retail site twice in one week. This resulted in around 120,000 lost orders on the second day and a 99% drop in orders on the fifth. An estimated 6.3 million orders were also lost in a single day.

Now, Amazon requires a senior engineer’s sign-off before junior and mid-level engineers ship AI-assisted code. By late May, the company had scrapped the leaderboard – an indicator that the environment, not a policy, had been steering people’s behavior the whole time.

Environment > policy

How to build it

The half that no gate can enforce