Is it really that much of a big deal though?

I would expect that each authority server's configuration data is securely and reliably backed up, so if the physical servers are seized, then a replacement can be fairly easily provisioned. Maybe some operators will even have one waiting on standby, in case of a normal hardware failure or similar.

Presumably the IP address space isn't being seized or otherwise disbanded, so it could be dropped in exactly as specified in the hard-coded configuration - that is, the same IP address and port.

Any ongoing and persistent impersonation of the server wouldn't be viable, as the long-term directory authority identity keys are kept offline.

Maybe I'm missing something here but it sounds like more of a symbolic violation, rather than a potentially catastrophic disabling event that brings down Tor.