Yeah, I don't even think most people in the US would use the word migrant to talk about someone from Canada. When I hear "migrant worker", the image in my head is agricultural workers coming from Mexico.