Comment by Pewpewarrows | Hacker News Reader

Comment by Pewpewarrows | Hacker News Reader

Happy to hear that you had backups and not too much data was lost because you caught it quickly.

But besides the obvious takeaway of always having recent, tested backups, I think it's also important to get out of the habit of ever using a REPL (or SSH for that matter) to connect to production servers. Write a one-off script, document when and why you made it, check it into your VCS of choice, and run it against the target server(s). Of course within the script itself feel free to make any connections you need; my point is to never do it "manually" from the command-line. This also prepares you for when you move past having a single live server for web or db. If you're in Python-land, Fabric is the go-to tool these days for automating this.

I usually write my scripts in a temporary file and run them through Django with something like:

    $ python manage.py shell <~/tmp-script.py

In this instance, I didn't even consider that what I was doing could be catastrophic. But you're right, you want to avoid doing things manually. In the future, I'll track all the scripts I run. And thanks for the tip about Fabric, I'll check it out.

For bonus points, clone your production db locally and test your script against it a few times first.

Or, you know, have a staging server.

Exactly.

This is the reason why enterprises always use staging servers with test suites.

It is irrelevant if you use NoSQL, SQL, or magic, if you don't have development+staging+production platforms for a business reliant on an information system, you're doing it wrong and it will bite you hard eventually.

The advice isn't about how you do the script.

You should never, ever be running test scripts and code against a production db.

Reply on news.ycombinator.com