I don't think anybody reading an Internet discussion has so little time not to add to the downvote: "This part of your comment is horribly wrong, I warn others against doing it."
Actually, I think I would be more convinced to be careful with that sentence than just seeing bunch of downvotes.
But commenting without downvoting skews the ordering algorithms, and of course if someone else has already made a similar enough comment that you wouldn't be adding materially to the conversation, adding a similar comment just makes the post look busy an therefore, potentially, interesting.
You got a downvoted comment with no replies. Now, why was it downvoted? You can reread it and see if you were rude/offensive, but besides that...