It looks like there's about 20x more ag land, by sq km, than urban land in the US (4M to about 200k). So even if urban areas doubled in size, purely at the expense of ag (and not forests), it would be about 5% loss overall.
I have a feeling these numbers don't account for suburban land use, and that's actually an issue... Not urban use.
I'd guess overall agricultural landuse would drop to 1/4th or so due to the huge inefficiencies of animal-based products, but would have to dig for proper numbers at e.g. Our World in Data.
I hear this a lot and trust it is a widespread occurrence. Here (gulf-coast FL), farms are commonly non-viable when sold to developers. Our tree-based crops seem especially prone to farm-busting economics, diseases and weather.