The Problem of a Lack of Downvoting.

Recently, I've noticed that a lot of the articles reaching the frontpage of HN, and often the top thereof, were flamebait/troll blog posts, off-topic (e.g. politics), or whatever. But moreso, the majority of posters in the comments often agreed that the article was junk! As such, one wonders, how in the world did the article rise to the top?

The problem is that when you don't allow downvoting, while it prevents the problem on sites like Reddit and Digg of "organized downvotes" and hivemind downvoting of disagreeing views, it also means that something's score depends solely on the number of upvotes. So, if you post something extremely controversial that doesn't belong on HN, it can get more upvotes that most other posts. Why is this? Here's an example (let's say HN has 1000 readers, for simple math):

Let's say you post Article A, a politics piece railing against, say... Nancy Pelosi. Lots of people are going to look at it, since its controversial, and everyone has an opinion. Let's say 50% of HN readers look at it, and 20% of those upvote it, and 60% of those would have wanted to downvote it. With downvotes, the article would have been buried quickly. But this article gets 100 upvotes and quickly flies to the top.

Now, let's say you post a very interesting article on creative use of assembly, Article B. Not nearly as many people will look at it--let's say 10% of readers. And let's say 40% of those upvote it, and 10% of those would have downvoted it if they could. The article only gets 40 upvotes, despite being much better and more on-topic HN content.

It seems to me to be much like the classic "color of the bikeshed" issue; if its something everyone has an opinion about, it gets voted up even if the majority of people think the link is junk or offtopic.

Now, my question is--how can this problem be resolved without reintroducing the problem of downvotes?