First of all, most privacy regulations are surprisingly toothless and rarely enforced, so in practice, you get away with anything until you somehow do draw the ire of a regulator, and then they'll enforce their random favorite pet rules that don't make much sense, and may also change arbitrarily. (Think "must use fax, can't use e-mail unless it's PGP-encrypted" followed by "can't use fax either".)

German regulators that I spoke with acknowledge the need for backup, but require you to keep a separate "deletion log", so that when data is restored from backup, you can replay the deletions and keep supposed-to-be-deleted data from entering live systems. You also can't keep the backups forever, but if you have a reasonable plan, you're most likely going to be OK (even if they disagree, they'll probably just tell you to change your retention periods).

This applies to privacy in general - as long as you show a minimum of genuine care, and deal with a few formal requirements, you're probably going to be OK. GDPR is most scary for people who a) think it will be enforced to the fullest extent that a pessimistic reading of the law might allow, or b) are actively doing privacy-hostile things that they know they shouldn't be doing but want to keep doing.

For example, storing the data for backup purposes, then going back in and using it to build ML models because the backups have more data than the live system would (rightfully) have a decent chance of getting you in trouble. You can keep data for backup purposes, but if you're allowed to store data for purpose A, that doesn't mean "I'm allowed to store it so now I can use it for any purpose" as some people seem to think.