Everyone in a shitty position has a right to speak up. You're absolutely right that some people cannot exercise that right - they can't risk losing what they have. That is unfair to every single person in that situation (and personally, I find this to be a shameful reflection on the world).
But using that to justify that other workers should stay quiet is equally wrong. It's using something wrong happening to one group of people to justify something wrong happening to another group. Even if you don't believe the moral reasoning, there's the pragmatic angle - this attitude leads to overall worse employee treatment. If 'better off' workers are shamed into not discussing poor treatment (because they should be embarrassed complaining about such trivialities), then minor poor treatment is allowed to escalate unchecked.
In my opinion, a better approach overall is to think "all of us have some rights, including good treatment by an employer and a right to a life not straddled by anxiety or debt". Anyone crusading against that is on the same side.