The trust of people like yourself is definitely going to be abused by people who want to smear others.
Yes, we can all make factually true, but totally irrelevant arguments that don't change anything.
A person does not have the right to be trusted, but can expect to be taken seriously on a claim of harassment if they have earned trust or if they provide evidence to back up the allegation.