The database is at the core of our business.
But there has to be a better way to create automatic backups.
The way I currently do it is with cronjobs and scripts that move the transaction logs to another cloud provider (every xx minutes) and do a full backup every night.
Is this really the way to do this?
How many transactions are you doing on the DB? What's writing to it?
How big is the DB? https://mariadb.org/cyber-protection-best-practices/
A common pattern with Postgres databases is to replicate the WAL to a read replica. If the primary goes down you promote the replica to master. I have to imagine there is a similar recipe for MariaDB.
If the way you are doing it works, there isn't a better way.
So to me, the place to start is to make sure what you are doing is working (if you haven't).
I mean having one copy on another cloud provider isn't very deep.
As in only one cancelled credit card from failure (or maybe two).
Or two compromised credentials, at best.
Backing up either sucks or it isn't really backing up. It is not a good place to look for efficiency, because not backing up is always going to be easier than what you are doing.
Good luck.
If I want to check if everything works, I have to check several places. My own documentation of this backup process is several pages long.
Considering how many people need to backup databases on linux, I was hoping there'd be a better, simpler practice.
1. The place to check if everything works is a running system after a restore. None of the rest matters if that doesn't work.
2. If I wanted something simple and reliable, I would spend the money and use tape.
3. If I was rolling my own cloud backup, I would abstract it to work with the Linux Tape API.
https://linux.die.net/man/1/mt
https://access.redhat.com/documentation/en-us/red_hat_enterp...
4. AWS seems to have anticipated my intuition
https://aws.amazon.com/storagegateway/vtl/
5. The caveat is that this is all theory and may not work for your particular needs.