I can downvote with only 1030. However, for this article, yours is the only comment that it will let me downvote. Also, a good number of articles on the Ask HN page won't let me downvote although every article I have checked on the front page does; I suspect it may have to do with the age of the comments.

I Am 500 now and I can downvote, but only comments not stories.

It is 500. I remember exactly when I crossed it and received the ability. That said, certain threads have downvoting disabled.