An interesting idea. But, unfortunately, it's very open to abuse. There is no guarantee the flagged comment has been flagged correctly.
Really, you're stuck with the same problem: just as there could be mass upvotes of "poor" comments and mass downvotes of "good" comments, there could be mass flagging of "good" comments as "spam", "dumb", or "mean".