Hacker News Reader

Comment by londons_explore | Hacker News Reader

For anyone reading this who has login access to a production SQL database...:

* Change your account to readonly. Make a new admin account, and put its credentials somewhere hard to get (and audited!).

* Make a directory in git for 'one off sql statements'. Make them all go through code review and have an automated system run them on merge/deploy.

* Enforce style rules with a linter/test, like "UPDATE must have a LIMIT"

* Anything the above process is too burdensome to do should have an API or admin interface built for the purpose.

* Aim to eventually get rid of your readonly account. A leaked customer data dump could kill the company and shouldn't be available to any malware on your machine. You aren't as secure as you think you are.

View on HN

I agree with everything you said except "a customer database leak could kill a company".

Do we have any example of any company going under due to a data breach? Unfortunately, there seem to be a lot of examples of enormous breaches that did essentially nothing. (like Experian)

That Italian malware vendor closed down after all it's internal documents were leaked. As did the law firm behind the Panama papers

Hacking Team is still alive and kicking.

Well yes, but I would attribute the first to simply doing illegal things and getting caught. The data breach just helped them get caught faster. The second, yes, although privacy was literally their number one product and I wouldn't categorize a leak quite the same as a breach.

You forgot one thing, if you’re ever executing one off statements make sure they’re in a transaction if it’s an insert or update.

BEGIN; Statement COMMIT; or ROLLBACK

etc

You can also do changes in the transaction and check that it did what you expected before committing.

That is smart, I would add "limit 2" to a single row update, so if it returns 2 rows updated I know it was wrong.

Fair warning about this: if your transaction isolation level in your database is set to serialized hanging out in mid transaction while you verify your changes will block other calls to the table(s)!

I always recommend that you are wrapping calls in transactions if you are touching prod data. I only left the WHERE clause of my update query off once in my life before I started doing that.

If you fear this, you can put it into script like this:

BEGIN; UPDATE... SELECT for verification ROLLBACK;

And then just change ROLLBACK to COMMIT, after you're sattisfied. There's no need to do it interactively.

> * Enforce style rules with a linter/test, like "UPDATE must have a LIMIT"

As someone who has never had access to a DB with any serious number of users, can you explain this one further? What if you really do want to update every row? Do you just do LIMIT INT_MAX or the like, and just force people to write that so that they always know they're updating the entire table? Or are you saying you should only ever use UPDATE on a known finite (and small?) number of rows?

I think it might just be like a psychological thing to try to prevent mistakes. Like a dialogue box asking "Sure?" before you delete your harddrive.

Updating every row is usually a bad idea anyway. That means every row needs to be rewritten, which involves reading and writing all data in the table. That will take locks which mean old data needs to be kept (in case of a transaction abort), doubling disk usage. Since half of rows are now dead, all other queries to the table will take double the disk access time.

Basically, if your table is over a few gigabytes, updating every row in one query on a production instance is a really bad plan.

So how do you make the change then? Do SQL dialects provide built-in tools to do an UPDATE to a large table in a saner way, or do you end up having to write an application to do so?

It's the notion of a staggered rollout. You should never initiate a global change without first validating it on some small, controlled-impact subset.

Reply on news.ycombinator.com