If someone thinks a given submission is inappropriate for HN they can and will flag it. Flags cause a submission to plummet in the rankings, and just a few flags suffice to send your item down to the 500's or worse, even if you have 5 to 10 votes. Getting 5 to 10 votes quickly is enough to get you to the front page at certain times of day. Once there, more up-votes usually follow. A single flag is enough to make that moot and get you off the front page.

Further, enough flags and your item will be killed with no attention from moderators required.

If someone has decided they don't like you, or don't like the topic on which you regularly post, then it's plausible that they will simply flag everything you submit on reflex.

Having a few active flaggers who are uninterested in your topic of expertise is enough to explain the symptoms you describe. I've seen it happen.

Further, complaints about voting and flagging are the items most certain to get flagged by a specific minority.