Without the comment scores we're confronted with a wall of text that is hard to filter. Should I have to read every single comment just to find the gems? Comments scores have their downsides but they make filtering out noise much easier.

Unfortunately I don't think everyone has time to judge each comment on its own merits -- there are simply too many comments -- so we need a little crowd-sourced ranking. It does lead to some group think at times but that's a (relatively) small price to pay.