We have fossils that are > 3 billion years old. While subduction does steadily erase the past, not everything subducts. See: https://en.wikipedia.org/wiki/Craton

That depends where you leave them.