persistence of symptoms after discontinuation is the big one. after hormone levels return to baseline, there's no more pharmaceutical effect. thus the cause of those symptoms is unlikely to be pharmaceutical.

it's worth noting that candidates for finasteride treatment are already likely to be older and dealing with comorbidities like depression and anxiety, which makes it harder to say for sure if it's the drug in a lot of cases, but they do seem slightly higher than placebo. it is not surprising that a sudden change in hormone levels would cause a noticeable change in mood or sexual function, but there is usually improvement with continued treatment as levels stabilize.

for persistent effects, really there isn't a lot of reliable data to go on, and no plausible mechanism of action. we have a handful of anecdotal reports and some armchair hypotheses.

i'm happy to be proven wrong, and lots of drugs are indeed implicated in serious and lasting side effects, but in the case of finasteride i'm not convinced.