No, you can't replace all existing power sources now and into the future by pumped hydro.
Pumped hydro has a massive role to play in power supply, time shifting electricity from renewable sources, providing system resiliency, black start, inductive and inertial load, you-name-it. It also exists. It's in widespread use, across the globe. I have one less than 100km from me here in Qld, there is one under construction in the snowy hydro project (with issues) complementing an existing unit at Tumut, Another at Shoalhaven in NSW and there are two large sites in the UK power grid Dinorwig and Cruachan, both of which supply near-instantaneous (<20sec) power against demand shifts.
Compared to gas turbine load start (minutes), pumped hydro is fast (seconds). Compared to batteries (milliseconds), It's slow. Batteries have capacity limits too. In the capacity stakes, Pumped hydro can beat them hands-down but for a massive initial capital outlay. Snowy 2.0 will supply DAYS of power. 2Gigawatts, as 350GWh of power. 3 million homes equivalent, for a week. People argue about the economics, not about the underlying capability.
Worldwide there are plenty of good sites, including mine shafts. And, gravity models for power include trains, and cranes. Neither of which will replace all sources of power for all burdens in the network either. The idea this is limited to existing land-surface bodies of water simply isn't true.
Not all useful things have to be able to solve the problems "at the scale we need" to be both useful, and able to solve problems.