Replit goes rogue and deletes our entire database

This Twitter account is an influencer with 200,000+ followers, known for its hot takes.

Though the risk of GenAI is real, it looks to me there is a fair amount of chance that this story is staged and amplified for social media drama purposes.

This shows a lack of understanding of how software development and deployment actually work. First of all, you manage your production database using migration files. Secondly, you never let GenAI make deployment decisions. At most, it can read your system logs. GenAI doesn't reason, so it has no clue what dropping a production database really means.

People out there letting LLMs run whatever commands they want unsupervised on their databases.

And we wonder why so much software is so crappy.

Maybe misaligned AI is exactly what we need to format the hard drives of all these people. Leaving us with a golden age of software, people actually cared enough to think about when creating.

When things go wrong with AI people seem to often make it prostrate itself, explain why it went wrong and promise to never do it again - which it does, but there’s no point, it’s not going to remember because it doesn’t have memory, and the reasoning as to why it went wrong is usually more hallucinated than regular conversation - like saying that it panicked (which it doesn’t do) or that it ran tests locally, which it can’t.

Perhaps getting something wrong puts in a state that makes it more likely to give wrong answers. It seems like GPT is the most likely to do this.

Also I don’t think you should be letting an LLM just make up commands and run them, that seems like a recipe for disaster, you should at the very least have to see what it’s going to do yourself.

Later the AI claims it can't run unit tests without overwriting the production database. This whole thread is hilarious.

https://x.com/jasonlk/status/1946641193644798118

Wow the information is really scattered over so many tweets. So they were able to recover?

Accessing a production database should require using an MFA to access your production AWS account. Did they rely on AI to write all the deployment as well?

Do they even have a dev environment outside of their local machine?

Why let it do things on production? We don’t let people do whatever they want on production, why AI?