Not only does it require a bit more overhead on the downvoter's part, but the reasons can be used by the algorithm.
I also think that some sort of feedback for the comment's poster could really help comment quality.
edit to add: It also reduces the possibility for 'accidental' downvotes, which I see on a regular basis.
----------
On a related note, deleted comments should still count toward a user's Karma and avg.
The other day I saw a user making comments that struck me as factually questionable. I'm not an expert in that field, so I asked for an explanation. The user had an avg over 5, so there was reason to believe the user might have known something I didn't. All of a sudden, the user's comments on that topic started getting downvoted. Despite the initial downvotes one of the comments was still at the top of the thread (my understanding is that a user's avg impacts the comment ordering). The user never responded to requests for more information or reputable sources, and then all of a sudden, all of his/her comments on that thread were deleted.
Curious, I went and looked up the user's previous comments. In multiple fields this person put on an air of expertise. Maybe he/she is and then again, maybe not. But the comments were almost all disparaging, with this air of expertise. So, the user makes definitive claims that come across like an expert, if their comments start getting downvoted, they delete the comment and don't get penalized. Otherwise, they reap in the upvotes and have an avg that appears to the world to justify their expertise.