The problem economists would point out is that this is a fully general argument against
any development. Which is exactly what leads to the negative side of gentrification - people eventually being forced out of their communities, because there's way too little developed real estate and the rent is too damn high. Georgism lowers rents by pushing towards the highest and best use of land, which is the anti-gentrification move.
In your specific example, you shouldn't assume that the supposed change in the "highest and best use" will be enough to price you out if you're a typical homeowner (not a vulnerable renter in a sharply gentrifying area): the numbers don't really pencil out that way. High land values exist in some places, but they're quite rare and quite concentrated.