AI Writes the Code, but Humans Can't Review It All. Now What?

"Make mistakes cheaper" part works until you make irreversible changes. As I'm building solo with an agent's help, I always try to control what the agent actually can or can't do.

For instance, all DB migrations and access policies are applied by me. In this case, feature flags don't help because if the read policy is too permissive, the data will be out by the time you notice it.

Before I figured it out, I shipped to issues.

The first of them was a policy that was letting anyone read a board's member list.

The second was a "with check" in the policy that pinned a column's value, so any other user's unrelated updates on that row started failing, and neither was visible in the diff. The first one showed up only in a test that asserts the attacker fails. The second one only in a test that asserts a different legitimate user still succeeds.

As for now, I pass reversible stuff to the agent, and the sensitive changes I apply myself. Also, I ask the agent to cover negative cases and make a security review before applying sensitive changes.

That's it.

Now what happens is that basically-working but poorly-designed code gets pushed anyway, with half-assed bot review. What happens next is that complexity and defect rates gradually climb until the software becomes too fragile to change without imploding. If we’re lucky, eventually people conclude that the situation isn’t sustainable.