It seems that rents are going down a bit in San Francisco because some people who can suddenly work remotely are moving out (among other reasons). UBI is an income that you can take with you wherever you go, much like social security, so it's likely to make living in cheaper places more attractive.
1) Allow actual competition. Adopt Japan-like zoning to make it easier to build. Landlords cannot freeload if they have competition. Make it harder for small special interest groups to veto new development.
2) Set property taxes appropriately or implement a land-value tax so that landlords have to pay for part of the surrounding improvements.
City expansion is hard because there isn't enough money to uniformly develop every growing areas in all directions.
This is very similar to any investment you will ever make. Including things like education.
If we walked down this route. Everything needs to be free.
You're right that cities do not expand uniformly, and city improvements are also not uniformly distributed. This is why a flat property tax does not make sense. Use land-value-tax to balance. If one area is very desirable, increase land taxes in that area and use the revenue to fund improvements like parks, libraries, schools all around the city.