I used to think that too. I don't anymore. People are anti-immigrant. The law determines whether the immigrant is legal or not, but that does not matter to the anti-immigrant. Let us be generous and say the anti-immigrant is so because of economic reasons, depression of wages and what not. Then does it matter that the immigrant is here legally? No. He is nonetheless contributing to the depression of wages (ex. H1-b). And then maybe if the economy is super healthy and there is labour shortage, would it then matter if the now desired immigrant is legal or not? No.
People are anti-immigrant because they see immigrants are others, as competition who has no right to compete with them, or similarly, they seem them as another people encroaching on their territory. Issues of legality concern the state but they do not concern the individual anti-immigrant.