Not to completely deny your point, but we should be much more careful: in my opinion, a big part of mental illness stigma comes from the inability of society to see a culprit, and instead let the individual become the culprit. We are very biased to rationalize stuff. If we don't see an abusive family, economic problems, obvious disabilities, etc, we tend to rationalize something else and not admit that maybe there are psychological barriers and obstacles that are disabling a person. Being in social isolation is not easier than working hard. It's a complicated problem.
I personally believe the best attitude is to accept that if someone can't get out of a certain situation by themselves, they need help. No additional judgement required. Now, who you want to help or whether you want to help someone is your choice, as we often already have enough trouble in our own lives to also go helping others. That's ok. But anything else is trying to rationalize who needs more help, when we are always lacking context and perspective.