Yep, if we haven't observed it, it is effectively not happened.

Same as finally observing some of the first galaxies created after the big bang. Speed of light ~= speed of information. If we haven't seen indication of it directly or indirectly, its at best an inference.