I prefer to be replied to rather than downvoted - the discussion here is of greater value to me than blind attempts at operant conditioning without context, and some of the best conversations I've had here have been with people who disagree with me.
Unfortunately, downvoting without comment has been baked into HN culture since the beginning, and as long as it's not explicitly against the guidelines, people are free to do so without consequence.
However, given how diverse the userbase seems to be here, in terms of age, locale, income, political alignment, etc, expecting a single definition of what HN "should be" seems destined to fail, because there are multiple groups with multiple, contradictory ideals. Just look at the reaction to dang's attempt at banning political articles for a week to see an example of this in action.
What would be more likely, that requiring comments for downvotes would result in deeper engagement and higher quality discussion, or more flamebaiting and divisiveness? In comment threads that are already volatile (and garner a number of downvotes) I'm not certain the better angels of our nature would win out. It would just result in more noise and more downvoting, as people began to argue about the reasons provided for their downvotes.
Then again, I think HN should get rid of voting and karma altogether, so I'm even more of an outlier than many here. Simply ignoring a comment with which you disagree, but have no coherent or intelligent response for, is always an option, and this community would be a better place if more people considered it.