This is a smart idea. A lot of the best startups are using recruiters (who can get $30k or more per hire). The noise level will be a LOT higher than a recruiter, but it might be worth it. Hopefully Rand will write a post-mortem on this.
That might go on our dev blog (rather than the main blog) - http://devblog.seomoz.org/
I don't claim to fully understand the challenges of a startup hiring the right folks, but recruiters just leave a bad taste in my mouth. Often they use questionable tactics and you know they are costing the startup a lot of money.
In a vacum ignorant of the realities of the real world, I wish there was a way to rate engineers and startups on all the key factors. For a startup: compensation, prestige, work environment, meeting/working with great people and for engineers: quality of work, ability to get things done, working well with others, etc. The problem of course is that getting a truthful and accurate value for each these factors is almost impossible.