Maybe another way to put it is: the Russian empire was reasonably “successful” (not actually good, ethically, IMO) in assimilating areas that had a stronger tie to the historical Eastern Orthodox core of the Moscow state. Tie also meaning the expulsions you mentioned. They had a more difficult time on the borders of and outside this zone. (Poland, for example.) This continued into the Soviet era.
A good metric of assimilation might be the alphabet. Seems to me that if the imperial core is using one alphabet (Cyrillic) but outlying areas with a hostile history are using another (Latin), then the assimilation process wasn’t working and perhaps never would work. And more broadly this alphabet difference is really a religious-cultural difference going back a millennium.
In short, IMO the Latin-using Catholic Poles had a long history of being different from the Cyrillic-using Orthodox Eastern Slavs, and were large enough as a group to not get relocated or assimilated, and therefore basically never would have been properly assimilated into the Russian-dominated Soviet identity.