At that time, a lot of people (supporters or not) were under the impression that Trump was okay with legal immigration.

The travel ban made it clear that's not the case (it affected legal immigrants).

In fact, Bannon had spoken against legal immigration before the election, and continues to do so:

"Legal immigration is the real problem"

https://www.vox.com/policy-and-politics/2017/2/2/14472404/st...