Points trigger a certain response among some people. The ones that get sucked into WoW for example simply because they have this impulse of getting all the best gear. Works with points as well.
In that case, you might as well just have numbers, since you can correlate numbers with colors, at that point.
I would rather see colors indicating the "heat" of a comment. That is, its numbers, in relation to other numbers in the thread. In other words, you wouldn't have the following:
Blue == 5+ Yellow == 10+ Orange == 20+ Red == 50+
Instead, you would have:
Blue == average score in this thread/topic Yellow == 5% higher score than other posts in this thread/topic Orange == 10% higher score than other posts in this thread/topic Red == 20% higher score than other posts in this thread/topic
This would help provide a more meaningful context for what posts are rising to the top in that particular conversation and the highest would always rise to the top. The top 20% of comments would always be the top 20% of comments, even if the lowest scored comment in the entire discussion is +10 points.
Anyway... really like your idea :).
Was going to write something about colouring entire threads or sub-threads as interesting based on the total sum of points or something else compared to everyone else. Would certainly help finding interesting stuff to read in 200+ comment threads. But not too sure how it would work without being invasive.