PG: I didn't say I'd do whatever got the most votes.
Keep in mind that this is a man that tries to give you startup advice.
pg added a useful feature, hidden comment scores -- then asks if you liked it better with or without them -- took that information into account along with data on comment voting patterns he has presumably collected since the change and made a decision.
What would pg's advice to a startup be in this case? Ignore your user's feedback and push the removal of feature (or push to leave the new feature, based on your stance), or listen to your users? I think he'd say to follow what your users want-- so why is he preaching one thing and practicing the other?
Seeing comment scores can easily lead to disproportionate scores as the ones with high scores are read more and consequently voted up more.
Many people may skip over new comments in favour of getting straight to the top scoring ones. I assume this is the rationale here.
You're right that the results of the vote show a pretty clear 'winner'... I'm not sure that's the end of it, though.
If 1000 people wanted a specific feature in my app I would obviously take that seriously and consider it carefully. But it doesn't mean I have to do it, just that I should be paying attention.
I also think that the people who desperately want comment scores back are simply more vocal about it. I don't care all that much, and I don't remember if I voted in the poll or not (but I do remember seeing it on the frontpage). I think this should be taken into account, too. I'm sure 37signals gets thousands of emails asking them to add <pet feature>, and few asking them to leave that feature out. Just because your users want it doesn't mean you aren't still in charge.
And, regarding your main point - duh. What would the result of that poll have had to have been to make it a compelling (e.g., actionable) argument for comment points?
I'm not sure, because I don't have a clear view of all relevant data. For example, I might weigh obvservations of improved discussion against the poll results. I'm not sure how overwhelming the majority would have to be, but if there was one, it would probably at least sway me to to do some more investigation.
And pg's comment was:
I'm curious if there has been any drift toward a consensus.
A three-fifths majority isn't really consensus.
The way poll questions are worded could sway people to vote one way or the other.
The order of choices is always fixed, and this could also have some effect on the votes. I've often noticed that the first choice (at the top of the list of choices) tends to be the one that gets the most votes.
Finally and most damningly, votes on polls have the same problem that comments used to have: you can see how many people voted for each option before you vote!
This makes polls susceptible to the exact same thing that comments used to be susceptible to before pg took away visible comment scores: voting with the herd (ie. piling on to whatever option seems to be the most popular).
I wonder how different poll results would be were poll questions stated more neutrally, the order choices were displayed in was randomized for each person viewing the poll, and the vote totals were kept hidden until after you voted.