- the quicker but less rigorous way: set things up like a forum like HackerNews, let the crowd-sourced upvote and comment system work its magic; make the karma/reputation points act as a proxy for the quality and relevance of the article, its authors, as well as the commenters.
- the slower way: let articles "bubble" up from smaller but more specialized workshops toward more generalized and prestigious conferences and journals. Have the technical committee recommend the best articles to those more reputable venues, along with whatever it is the authors need to work on to re-submit the extended version at that "better" venue. This is kind of already the case with workshops or conferences that have an associated Special Issue of a journal; I'm advocating for a generalization of the practice to all venues and more than a 2-level hierarchy. The reputation of the lower venues could be backpropagated by the number of papers they successfully "bubble" up, PageRank style, so that would be the incentive the committees would have in recommending their papers to the other venues.