Your karma is in the top right. You need around 300 to downvote, 800 to flag. Your downvote by clicking the down arrow beside every comment and submission.
Down vote comment. Nope, only 1 arrow for up, and I am good with that.
Just refreshed, and now I can down vote some comments...
Refresh again... I cannot down vote posts, but can down vote even less comments... someone is watching, and laughing at this. :P
You cannot downvote direct replies to your own comments though if that helps understanding what might be happening here.
You can't downvote replies to your comments and in other circumstances I think.
I recently got to 300 and I saw no change in the UI.