It's as if we never experienced those ~4B years. Or the ~13B years for the supposed age of the universe. Neither do we experience the ~8 hours of unconsciousness we have every night. We only know how much time has passed based on memory of previous sleeps, how groggy we feel, and how sunny it is behind the curtain.

The status quo belief today seems to be as you said: the nothingness that awaits us after death is also what came before our birth. But what came after the supposed nothingness of pre-birth was your birth, so why wouldn't there be another birthing which becomes you after the nothingness that comes after your death? To believe otherwise is tending towards solipsism. It would be thinking there is something special about "you" and why "you" were born into and inhabit "your" particular region of spacetime. When our feeling of personal identity, the feeling of "you"ness is really mostly an illusion of memory.

Imagine what it was like to be patient HM, with your memory being wiped every ~30 seconds. But imagine even further that you had no long term memories at all. It would be almost as if you were being reborn every time you have a memory lapse.