Except of course that a lot of the 'anger about immigration' in the way you put it is thinly disguised racism or xenophobia. It is quite hard to separate these (on both sides of the debate), and so have a sensible discussion on what a good policy on immigration should be.