Edit: another thing I should have mentioned is that it seemed to me that only those comments made after I gained this ability were downvotable, the thread my last few comments were in was sprinkled with downvotable comments, which were newer, and non-downvotable comments which were there before mine. I might have been imagining this or be mistaken though.
EDIT: checking again at 507 and I now have the power to downvote. So there's our answer (plus my moderately shameless play for some upvotes worked).
I think you might be mistaken. You might not have noticed yet that the downvote option disappears on its own a set time (24 hours, I think) after the comment was made. Perhaps it was only the comments older than this threshold that were non-votable?
I think my karma accrued overnight, so when I noticed I'd exceeded the threshold a lot of the thread was perhaps a day old. The only thing I am relatively certain of is that some of the older comments in the thread were not downvotable, and the newer ones were.
That would be quite computationally intensive to implement.