My point of view is that I only care about the quality of articles on the front page. If a downmod button would increase it, I would agree with such proposal, but I'm not sure (as you said) that such would be the case, and anyway I think the flag button does already work in a way like a big down button.
What I do believe is that content can get better if people actively submit more articles (so spending a little more energy in the site), not only by voting the articles in the new queue.
Anyway I have the feeling that the quality of articles in the site isn't always getting worse... these days I'm quite happy about it.