I think you've reached this statement too eagerly but would be interested to discuss this point. Do you mean that the materials they used would have disintegrated and their (presumably carbon-based lifeform) bodies wouldn't have left any fossils?
My main thoughts on this come from reading Vernor Vinge's [1] excellent Marooned in Realtime which discusses some of the condundrums resulting from transferring information over massive periods of time. I think part of it talks about subduction zones where everything is eventually riven back into the Earth's mantle, essentially lost to any kind of current archaelogical techniques.