Freedom doesn't mean just what you think is acceptable. If a society feels like they let too many people into the country they have the freedom to undo it.
I do not consider that as a requirement free society. While representational democracies give governance to a small amount of people compared to the overall populace theoretical you could have a society where everyone was on equal footing and all contributed to decisions being made.
However, support for migration is not a left or right issue. Accepting refugees might be (and I fully support that), but non-refugee immigration (the vast majority of it) is not an unalloyed good.
Immigration has a number of negative effects, on housing, employment and wages, inequalities, social cohesion, political direction.
Being a radical leftist and against immigration is not contradictory.