It might seem outlandish to us, but then again, so would a hydrogen bomb.
Though lets be honest, anything in orbit is dangerous. A crowbar (aka Rods from God aka https://en.wikipedia.org/wiki/Kinetic_bombardment ) is even smaller and just as dangerous.
As space becomes easier to access the risk increases exponentially.
And it doesn't make much sense either. A billionaire can just pay a few millions to send a guy in a car bomb or a cessna full of explosives. It's way simpler and just as undetectable until the explosion happens.
Also, the idea of changing an asteroid's trajectory just before it hits the Earth is crazy. You need much more warning, because even a big bomb isn't going to move the asteroid that much, so you need to use it long before it's projected to collide.