Over the last half-century (probably more) we've seen a big part of the economic base for these small cities and downs either disappear or move away. Agriculture used to play a much bigger role in the economy as a whole, and a particularly large role in the economy of many small towns. And manufacturing used but much more distributed across the country in smaller cities and towns. But a lot of those factories have closed down.
As a consequence the tax base in may of these towns and cities didn't go away simply because retail businesses moved out of the city limits. They may have gone away because the entire GDP of the surrounding area dropped.
For reference: https://commons.wikimedia.org/wiki/File:Sectors_of_US_Econom...