I don't know the specifics of this, but vouching "against" the hive mind leads to your vouches not doing anything any more. I assume that either there's some kind of threshold after which you're shadowbanned from vouching or perhaps there's a kind of vouch weight and "correctly" vouching (comment is not re-flagged) increases it, while wrongly vouching (comment remains flagged or is re-flagged) decreases your weight.