In Spain the perception is that Brits in the south are low qualified and old and that they have driven up the price of housing with their real estate speculation. When young, they are perceived as violent and disrespectful (probably the image created by short-term young binge-drinking tourists). It's certainly prejudice and first impressions, but that's the kind of irrational reactions you could see after a UK exit if Spain is struggling with its own economy.
I had heard the opposite statement, by the way, that they took more from the healthcare than they contributed ... but my google-fu failed me. Do you have a reference? (I don't know what the answer is either way)