A simplification of the process:

Tectonic plate movements create mountains, as they climb on top of each other (and where space was opened, lava there turns into new rock)

Then erosion carry material from high places, to low places.

Thus over time places in high areas get "shorter", while places in low areas get "taller".

Meanwhile some parts of tectonic plates are sinking again and melting again.

But as the planet core cools down, this process get slower and slower, and erosion speed remains "constant", so over time the tendency is the planet get rounder and rounder, eventually the planet would have cooled 100% and no new mountains would form, while erosion would make all mountains become flat over time.

In non-desert areas, dead biomass (fallen leaves, etc.) creates soil. My parents had a stone path through their yard that they didn't maintain, and within a decade it was covered over with soil and grass grew over it.

Over the course of a few thousand years, structures can be buried by several meters of earth.

Inorganic matter largely gets blown from other places, or washed down from mountain tops.

(had to reply with a gross out amusement, it's not sand, it's dead people!)