What he built was, almost by definition, a prototype.

The problem is that, as it often happens, the prototype was then deployed to production. In this case, it was partly the fault of your non-technical friend who didn't know better, but software engineering history shows us this sometimes happens because of customer/boss pressure.

So it's often the case prototypes designed to prove feasibility or to demo to customers become production software, and AI makes this undesirable result even more frictionless.

It seems in our line of business we are doomed to repeat past mistakes -- forever.