Restoring it took 24 hours, during which I got bombarded with text messages and emails warning me that someone was restoring my backup, and that if it wasn't me, I should immediately click or reply to prevent it from happening.
Seems like that might help - a 24 hour waiting period on any significant account changes for verified accounts.