You deploy an application that ostensibly has the desired capabilities but it fails at scale, or the architecture is not extensible to add new capabilities or fix bugs. Those are expensive mistakes and in safety-critical domains could be deadly.

The thing is I've worked on plenty of code like this written by humans. AI just accelerates the process and puts this type of code within reach of more people. It could be seen as a step backwards in good software engineering practices we've painfully learned over the last two decades. I see it as a cycle repeating itself in a new iteration, with the hope that we'll end up with still better tools and processes in another few years. But I agree it can and will result in real harm in the meantime.

A giant pile of unmaintainable, brittle code nobody understands. At some point, even LLMs struggle to make changes without introducing bugs.

The next decade of software consulting will be defined by repairing the Big Ball of Mud https://www.laputan.org/mud/

Waste of time and therefore money is a big one.

For example, doing a marketing push assuming that the app works, and then having it fail in trivial ways at release.

Code that accidentally ships every 100th order out for free... exploits releasing customer PID and causing regulatory nightmares... self-driving cars that reboot at 70mph... The only limit is your imagination!

Corrupted or lost customer data, security vulnerabilities, incorrect financial transactions, regulatory breaches, production outages, irreversible migrations, important product change effectively impossible without a full rewrite. Every feature takes longer, incidents take longer to resolve. Engineers spend their time fighting the codebase instead of building things, burn out and quit. I could go on forever.

Time lost when the real developers have to pick up the pieces and the viber has just moved on to the next shiny thing.