This is why the "assignment" that I request takes less than two hours and is a greenfield project that is not going to be used by the company -- and I tell candidates this up-front. Think something like a blog, recipe manager or to-do list app. It is trivial, but you may be surprised at the range of outputs that you get.

What is interesting to me is how the resume isn't the best indicator for how well they will do on the coding project. Also, to avoid wasting people's time, I give the coding project as late in the interview process as possible, assuming the other filters have been passed.

If the coding exercise looks like something that is of this sort, then I guess consider whether you want to work with that employer. From an employer perspective, this is the worst source of getting something built.

A candidate should be researching the companies submits his resume/CV too anyway.

"If it becomes widespread, we'll just see phony developer put their assignments up on rent-a-coder and ask for a crib sheet to study so they can withstand a technical interview on what they've created."

If that were feasible, I'd suggest the employer would much rather hire the rent-a-coder instead of the one paying him. That's a win for the employer there too, since they'll be able to offer the rent-a-coder guy far more than the scratchings he gets as a rent-a-coder. The ability to predict in advance a sufficient range of questions as to forearm a candidate will border on the mystic.

Coding exercises aren't a mouse in a maze puzzle, they are a starting point for gauging web development ability.

The companies that use coding challenges as a way of creating production ready material are going to be amply supplied by the rent-a-coder approach you outline. And the technical interview part of the test is probably geared with a formulaic approach that can be guessed. In that case, this candidate and that employer are well matched.

But not every candidate needs a rent-a-coder to pass a technical test, and not every company uses those tests to create production ready websites and applications.

A good candidate can sniff a bad employer just as well as vice versa. If a company seems to be asking for production ready material, then walk away. If the test seems on the up, you'd better have a very good reason for refusing.

When I hear talk like this coming from devs in a given startup, it only confirms that their work culture is terrible. It doesn't make me think their devs are bad. It's more of the "rock star" and "ninja" wishlist stuff. No company-- not even Google, Apple, or Facebook-- is going to employ only developers who are all at the top of their field. If you're a startup, you must have a pathway to get your average programmers to greatness.

When I hear stuff like "hand-holding" it makes me think of companies that hire based on keyword matching. They don't want to put anything at all into their employees, they just want the best output right away. No wonder these firms are always complaining about not being able to hire talent. They could develop talent, but they don't-- it's either getting the creme of the crop engineers (which they're not going to be able to get anyway since they can't compete with the above-mentioned companies on salary and benefits) or nothing!

The really creative startups I'm seeing now are those that train. They know there's a war for talent, so they're willing to invest in somebody who isn't a rock star today, but has potential and very well may be tomorrow.

Like you said, a small project is a little bit of a red herring. No small test project done in a short amount of time (like a few days) is really going to be complex enough to test if a developer is actually proficient enough to work on a large project.