Couldn't have been since US cities (at least on the east coast) predate the very existence of cars.
> But in a way that's worse because it means they used to have this privilege and it has been taken away, and cars-first has become the norm.
Yep, the entire concept of jaywalking was invented for the convenience of car owners and the car industry, and pedestrians were alienated from the street surface.
> But the roads would have been used for something, at a guess, was it trams and horse carriages, and that these didn't pose much of a problem?
Much less so, horses would not trample people unless spooked (and they don't have anywhere near the acceleration or speed of a car, especially as mass increases). And streetcars generally keep to pretty low speed (bike range) in pedestrian areas.
You can see what it looks like in low-speed / pedestrian streets of europe, where streetcars are pretty common e.g. https://www.youtube.com/watch?v=oXrGbZ6LKzs