Thanks, that kind of makes sense.

I definitely have a “vouch” option for submissions. I think it’s a newer addition than vouching comments.

That's true. Flags aren't downvotes. They're meant exclusively for stories that don't belong on the front page of HN, at all. Using them as a simple downvote is an abuse, and doing that routinely can get your flag power quietly revoked. Flagging helps the site much more than it hurts it, but, like everything else, it's deeply imperfect; I've had banal submissions flagged myself. The thing to do when that happens is to mail hn@yc and ask for the flags to be lifted.