It is slightly evocative of a kind of gentle conservatism which has mild undercurrents of racism/xenophobia for some people because rural England is overwhelmingly white and slightly old fashioned. (People call this "Jam and Jerusalem")
Basically it reminds people of the village council from Hot Fuzz.
(Source: grew up in Dorset)
Completely separately to this is has a recent history (80s) where it can be seen almost like a Confederate Flag in some contexts. It was aggressively adopted by fascists and also the subject of a tabloid conspiracy where they suggested immigrants hated it so people started flying it as an anti immigrant "this is our land" type thing.
This isn't the context you saw it in, but it's impossible to completely disentangle this new meaning, everyone is aware of it.
The people of an area are mostly white and that makes them racist? Do you think black people in mostly-black areas are racist too or is it only racist to be white?
There is no way to get from my comment to your comment without bringing a mountain of your own pet grievances and preconceptions to the table.
As proper flag-based protests in the UK go, it'd be difficult to beat the bizarre sight of Cornish fishing boats and villages being covered with Canadian maple leaves back in the mid 90s when Cornish fishermen backed the Canadian side of a fishing dispute...
It used to mean (80s/90s) that you are on the right of politics and not keen on immigration.
However, because the UK does not have one Soccer team (England/Wales/Scotland/N.Ireland) If there is a major soccer tournament on the red cross is used to represent England team and support for that team - from a wider diverse proportion of the population.
It fluctuates between the two meanings, as things like Brexit ("We will rise up!" and Sport tournaments ("You are currently in England") happen.