Ignore downvotes. They usually self-correct over the next few hours as people see unjustified downvotes and supply an upvote. And sometimes they're just fat finger errors.