P.S. I you fragmented an asteriod - would at least _some_ of the pieces tend to change heading/speed enough so they miss earth?
Far away in time doesn’t even need to mean far away in space, since many of the objects of concern are orbiting the Sun along with us. They may even have multiple close passes before being perturbed enough to become a threat.
Also, the masses involved don’t have to be huge. Two rocks intersecting in space are likely to be doing so with very different velocities. Kinetic energy is mass times velocity squared, so velocity rapidly overtakes mass as the dominant factor.
That's how you turn a fission bomb into a fusion bomb. If you just put LiD in a fission bomb, it doesn't squeeze hard enough. If you focus the x-rays from a fission bomb onto an ablation drive, it does squeeze hard enough and you 1000x+ the yield with fusion. Which is why fusion bombs look like metal peanuts.
Conveniently, this means we are also really good at designing x-ray reflectors and lenses that focus the primary output of a nuclear bomb for the purposes of building ablation drives. Depending on just how good, we could probably persuade an asteroid pusher to be far more unidirectional than one typically pictures from a "bomb," if we set it off some distance away and focused the x-rays in the direction of the asteroid.