The magic happens at BGP level.
I considered CF as a domain registrar, but they don't allow setting the NS records. So you must use them. (They basically use sane no-nonsense domain registration as a way to gain leads for their main product. Pretty smart actually, because it's a great high-level add-on for their main product, but they just went ahead and made that the bait for everyone.)
Anyway, ideally, if you add 2 separate sets of NS servers to your NS records then you eliminated this SPoF, great. Sure, it's your job to keep them updated, and in sync (preferably, to avoid problems like half of your users landing on a different CNAME/IP/etc).
And recursive nameservers will handle the failover.