"least interacting" is the key here. Trillions are emitted by the source, and nearly zero impact anything along the way, so most of them make it through the planets, stars, etc. Occasionally one hits an atom of ice/water (depending on which detector we're talking about), allowing us to detect it. The impacts on the detector are pretty rare, relatively speaking.