You can't downvote stories. You can only flag them when they're inappropriate.