I'm curious why this makes sense to you? Does it make sense that you could use an ordinary film camera and take a picture with an arbitrarily short exposure, say a picosecond?
With very, very low SNR it can take a long time to build up data. The reason why this is possible is because the real data, the signal, is correlated with itself, it builds up consistently with each addition of measurements. Noise, however, is uncorrelated, it's random, so it tends to evenly cancel (or even) itself out with additional measurements.
That's because whatever signal is out there is quite a bit fainter than the initial estimates predicted it would be. If the signal were as strong as the initial estimates predicted, it would have been seen by now.