I don't think you have a need to judge anyone unless you are an actual judge. As a private person you could just decide you want to stay away someone and leave it at that. You have no need to dehumanize anyone by private labeling.
And, I think this is confused labeling. Substance abuse alone does not make anyone a danger to you.
So, substance abuse:
a) person goes to a daily job, drinks a bottle of wine in the evening, does some coke, goes to sleep. repeats ad infinitum.
b) person has a really bad addiction, needs money, so he robs you
I presume your talk about fear and judgement was more about scenario b rather than a, but both are about a person with substance abuse problems. The thing we don't actually like in b) is the stealing, not the substance abuse per se.