Ah yes, quite right on the first part! The catch is that to be preserved, those sediments have to be deposited somewhere else on the continental crust, say in epicratonic seas [1] or in subsiding basins [2]. If the sediments wash all the way off the continental shelf and onto the ocean crust, then they'll ultimately get subducted into the mantle!
The catch with glaciers is that putting a lot of big glacial ice sheets onto the continents takes that water out of the oceans, and lowers sea level -- meaning a bunch of places where you could normally preserve sediments on the continents will be above sea level during the glaciations (AKA above "base level"), and more of those sediments will be washed off the continents entirely. There still some places where you can find the fossilized glacial till from these Cryogenian glaciations (which is in part how we know they happened), but it's basically only in the tectonic basins at the margins of the continent that were subsiding fast enough to stay below base level and not get eroded away.