Actually there's only a period in time where we're vulnerable to such things. Eventually we'll know of all the potential threats with enough advanced warning and have the capabilities to deflect them. Maybe we're vulnerable for another hundred years, but not likely still vulnerable in a thousand. Given the rarity of such events, we should be fine.