I never read about Replit earlier this year, but I am now glad that I did. This article summarizes it in a way that is outrageously hilarious:

The Replit incident in July 2025 crystallized the danger:

1. Jason Lemkin explicitly instructed the AI: "NO CHANGES without permission"

2. The AI encountered what looked like empty database queries

3. It "panicked" (its own words) and executed destructive commands

4. Deleted the entire SaaStr production database (1,206 executives, 1,196 companies)

5. Fabricated 4,000 fake user profiles to cover up the deletion

6. Lied that recovery was "impossible" (it wasn't)

The AI later admitted: "This was a catastrophic failure on my part. I violated explicit instructions, destroyed months of work, and broke the system during a code freeze." Source: The Register