I think identity trumps economics. People are willing to die for their identity, whereas it's harder to get people to die for some money when it's not a matter of survival. People would rather be proud and poor than getting by under what they've mentally envisioned as a yoke, instead of owning it.
Sure, economics is important. In global currency terms, the value of my home has declined substantially, I've lost a notional mid-5-figure amount expressed in EUR or USD terms. But if you're not internationally minded, if you don't visualize the possibility of yourself living outside of the UK, you're unlikely to consider it a real loss. It's a lot like how poor people can be self-limited by their self-conception, something you might have seen in teaching. (I grew up poor myself, it took some time to expand my horizons, and the process still isn't complete.)
If you live your life locally, why would you care if French cheese and Spanish ham gets more expensive? You probably think it'll be better for British producers. In fact you'd probably consider it just fine if the soft southerners have to pay more for their wine.
The big irony is that pro-Brexit regions have economies that are more dependent on the single market than pro-Remain London. Proving that's the case before Brexit, however, is hard - and if May's deal goes through, it'll go some way to softening the blow. So the decline will be even more imperceptible, until 20 years later, and finally you notice how shabby everything back home is when you come back from your Continental holiday.