How do you do this on the DNS layer? Very short TTL and switching on failure detection? Do you use both providers at the same time when both are up via DNS round robin or similar?
dig a www.google.com
www.google.com. 300 IN A 74.125.237.178
www.google.com. 300 IN A 74.125.237.177
www.google.com. 300 IN A 74.125.237.176
www.google.com. 300 IN A 74.125.237.179
www.google.com. 300 IN A 74.125.237.180
Browsers these days will resolve and try each one, on a first-come-first-served basis. If one fails, it will try the request on the next server.If you're relying on DNS then short TTLs are pretty reliable, but there can be extended outages due to erroneous client-caching.