Maybe you get twenty-three down votes and forty-six ups; the point count isn't going to tell you that. I know that I have comments where the score hovers around one but have maybe ten votes that cancel each other out. What to make of that?
A voting system should have modest goals. (While the Slashdot community hasn't been my cup of tea for a while, I feel like they've muddled their way to a system that works pretty well.)
Meanwhile, the useful/useless votes are smaller or perhaps harder to access, like the current "flag" system for comments where you have to click through to the comment's own page first. Perhaps voting either agree or disagree automatically registers your vote for useful, and to vote useless you have to click through like with "flag".
It's even useful for topics I'm already familiar with. If a comment I don't necessarily agree with has a high score, I'll wonder why. It's a great opportunity to walk down new mental paths or explore different angles about the topic without feeling vulnerable. If any insights emerge while doing this, I'll then follow up with my own comment.
In either case, it's impractical to analyze or reflect on every comment, so points are a pretty good heuristic. Even if it is a broken system, it does help a lot of the time.