Have to disagree.

This has already been mentioned elsewhere but comments with little value are already penalized when people downvote them.

If you notice more crap comments that aren't being downvoted that's because a) the community standards are changing and b) there's no good way to penalize all crap comments anyway - some will slip through the cracks. And as a result of b) great comments will slip through the cracks, which means people will lose points needlessly even when they're not trolling.

I'm not saying your proposal wouldn't have an impact on trolls. But there would likely be an awful lot of collateral damage.