Here's another factor: over 4.5 billion years we really don't have a good idea of what close neighbours the Solar System has had. Stars are orbiting the Milky Way and I seem to remember reading not that long ago (hundreds of thousands of years) our closest neighbour wasn't Proxima Centauri but some other star that is now further away.
What other neighbours have we had? How bright have they been? How close have they been? Could they have been bright enough to melt ice? If so, would this impact the dust layer? Could they be massive enough to disrupt the "dusting"?