The momentum point is very valid. But the slower speeds provide more time for drivers and others to react. That in turn gives you much more time to break and scrub off that extra momentum.
At 50kph your reaction distance is about 15 meters, at 25kph it’s half that at 7.5meters. (Assuming reaction time of 1 second, which seems to be a realist value).
7.5 meters is quite a bit of distance, and in a busy city will make a lot of difference.