Restoring is just as simple as flipping the switch again, but access to that switch is another matter when your internal network is also down and you cannot even get access to your office or datacenters.
I suppose with something like BGP it would be very difficult to get such a fallback working given how distributed the system is, and even more difficult to keep it exercised and tested.
https://www.juniper.net/documentation/us/en/software/junos/c...
But Facebook's network is certainly much more complex and automated than just doing one commit on one device.
But I think they do still uses Juniper devices at the edge.
<snark> Software defined networking. In PHP.
iptables-apply reverts to previous config instead of flushing all the rules.
How do you know if you waited long enough to see negative effects (think caches, your own and caches of others).
Waiting too long with a bad config also costs you.