You've got to have utility poles, trim trees away from power lines, etc. regardless of whether you serve 100% or 25% of the customers in that area.
If you lose a lot of customers, you will have to raise prices on the ones that remain. If you're competing against something that already took away some of your customers, then raising prices could have a snowball effect and cause you to lose more.
https://www.peninsulacleanenergy.com/for-residents/
https://www.peninsulacleanenergy.com/wp-content/uploads/2017...
It'll still be a thing in cities because you don't have enough area to collect sunlight for tall buildings on the buildings themselves, but that's also where the costs are lowest because the density is highest. If technology makes it so you no longer need a power grid in rural or suburban areas, and the economics are no longer viable there, why should we pay a lot of money to continue to have one there?
Hypothetically, if the capex to install PV is low enough, and the transmission costs are low enough, then you could build a type of decentralized generator where suburban and rural generation feeds urban demand. There's early work being done on "smart grids" which enable this pattern, but I could imagine a future company that markets to a private homeowner, one who already had sufficient generation for their own needs, with the offer of upgrading their private PV in exchange for most of the generation profits.