I kind of hope the CEO of Gitlab isn't reading HN right now
I kind of hope the CEO of Gitlab isn't reading HN right now
For every backup system you have, test restores, from scratch, periodically. The more critical the data is to your business, the more frequently and more automated you want those backup checks.
Of course you also want procedures to try to prevent tired admins from deleting production databases. To the greatest degree possible, implement systems to prevent manual tampering with production data; often there's an alternative. But databases can get broken or corrupt without admin interference, so preventing manual database removal or corruption is not an ultimate solution.
Do you have DBAs? Are they completely inept at their job? Don't answer.