So as you said, maybe the solution to the Fermi Paradox is that every surviving civilization is hiding in cloaked habitats within the vastness of intergalactic space.
I think if you look ahead at feasible technology, you get self-sustaining pure-space civilizations long before you get civilizations that can launch relativistic planet killers.
(The other problem with the "planet killer" hypothesis, as we've discussed in other HN conversations, is that there's no reason to wait until there's a civilization on the other end to launch one, given how fast they can pop up. As soon as you see life, you should launch one. But Earth has been broadcasting fairly conclusive proof that a civilization with just a bit more tech than we have now is alive ever since the Great Oxygen Catastrophe, yet nobody has killed Earth.)
This is more dangerous. If weapons are absolutely but selectively destructive - I can wipe your planet out quickly before you have time to respond and my planet is just fine - it incentivizes me to attack first and hunt down other potential threats and destroy them before you or they destroy me.