It’s nothing to do with ego.
HN downvotes cause comments to become effectively hidden and unreadable, and while some that receive this fate are just throw away “useless” comments, plenty are not.
The idea that a downvote “starts the conversation” is just plain wrong - people are actively told not to ask why they were downvoted. That’s the opposite of starting a conversation.
I disagree with what you said, but I didn’t downvote you,
I replied and told you why. If I, and the other people who Top-commented so far all just downvoted your comment your comment would be literally fading from view, and you’d have zero clue about why people did so.