It's galling to try to mark a comment as stupid and instead end up giving it my seal of approval...and vice versa.
Car safety is one of the canonical examples of the "risk compensation" effect:
"In 1981 John Adams published a paper, The efficacy of seatbelt legislation: A comparative study of road accident fatality statistics from 18 countries, Dept of Geography University College, London 1981 - published in 1982 by the Society of Automotive Engineers. This showed that in the countries studied, which included states with and without seat belt laws, there was no correlation between the passing of seat belt legislation and reductions in injuries or fatalities.
This paper was published at a time when Britain was considering a seat belt law, so the Department of Transport commissioned a report into the issue. In the event the report's author, Isles, agreed with Adams' conclusions. The Isles Report was never published officially but a copy was leaked to the Press some years later. The law was duly passed and subsequent investigation showed some reduction in fatalities, the cause of which could not be conclusively stated, due to the simultaneous introduction of evidential breath testing.
Other research has taken groups of drivers, including those who did and did not habitually wear seat-belts, and measured the effect on driving style in the habitually unbelted. The drivers were found to drive faster and less carefully when belted."
Ultimately I think it just comes down to how minimalist your design philosophy runs, and I'd be quick to admit that mine is more minimalist than most.
The number of mistaken up votes is probably a very tiny percentage of total votes, and accidental votes from multiple users on the same comment or post are most certainly even more rare. So the odds any given 'noise' comment or post is accidentally upvoted up to a meaningful level are extermely low.
And group think is already a problem on any voting news site. Allowing people to change their vote after they can see how everybody else is voting would certainly not alleviate that problem.
So while allowing a change of vote would help out an individual on their rare case of accidental upvote or downvote, I think the overall community effect would be detrimental.