The vast majority of people who are "anti-immigration", and who are not immigrants themselves (or recent descendants thereof), are not "simply anti-immigrant" either. Most "immigration restrictionists" want to reduce the levels of immigration significantly compared to current levels, but very few of them want to drive it all the way down to literally zero with no exceptions whatsoever ever allowed. Also, many "immigration restrictionists" want to reduce immigration by certain groups (whom they view as "culturally alien" or "problematic"), but are quite happy for continued (or even increased) immigration by groups they like. So if it is true that "immigrants are never simply anti-immigrant", well that's true of non-immigrants too.

No, I truly mean people who are against any sort of immigration. A subset of which who say that they escaped from worse cultures and don't want the new culture to be diluted.