There's a major issue with your method. The best developers know they can get jobs wherever they want. What incentive do they have to bother with the extra work?
Good developers of a certain experience level generally realize that if they apply for a job somewhere and waltz through the interview, or worse, the interview is a basically a pleasant chat with the tech director, then the chances are the hiring standards really aren't that great .
Oh and c) - if a developer does feel they're too good for your screening test, or such a thing is enough to put them off your company, then weeding them out at such an early stage may actually be a positive thing.
I do think it's crass to send out a problem sheet without at least verbally talking to the candidate briefly though. We currently do -
5-15 minute chat with a recruiter or PM who is savvy enough to filter out the people who really aren't experienced enough.
3 hour timed problem (debugging, coding, math, domain knowledge)
Technical phone screen
In-person interview (6-7 hours, all expenses paid).
Before I had my own company, every place I worked whenever I would talk to clients or go to conferences I would get on the spot job offers. Principals would ask "Would you ever consider leaving XYZ?" I would say I'm not looking at the moment. They would always then hand me their cards and say "If you're ever looking to move, give me a call first."
In addition, I would get calls at home to my unlisted number because of articles I had written or patents I held.
When it was time to move on I would have dozens of open offers already to chose from. Why the hell would I or any other great developer bother dredging through monster hoping to find the 0.001% of ads that are both legitimate and for a place than isn't some web shop with slave hours and low wages? Answer is I wouldn't. Any company that only relies on job ads and resumes is not a place that great developers work for.
Recently a startup I was somewhat interested in required a 4-5 hour coding project that was similar to the labs I do in school. I'm already bogged down in work interviewing, doing normal classes, writing a senior thesis, and preparing my research for publication.
I decided to forget about the company and look elsewhere, for I already had offers from great places like Google and Facebook and could always fall back.
I'm not saying its bad to screen candidates with interesting puzzles. I'm saying that you risk filtering out the better candidates that just don't have time to deal with puzzles from a company they don't know.
Additionally, keep in mind that I come from a competitive university in the academic world (although I figure competition is pretty bad at most universities). Thus, with my current 100 hour work weeks, I would just rather not bother with solving tedious puzzles.
One of them asked that I solve some programming challenges. Sure, I like puzzles and I like being challenged. However, at the time I was already being challenged by my thesis work, problem sets, interviews... it's hard to agree to make a huge time commitment just to get the right to a first round interview.
Since other hot startups/ cool companies (Google, FB) don't require this type of activity, it makes sense to me to think that a lot of people simply won't have the time to apply because they're busy solving more interesting puzzles.
http://www.facebook.com/careers/puzzles.php
And Google are notorious for having a long drawn out interviewing process.
Google's are drawn out, but each interview takes 1 hour max so there is an upper bound on your time.
My response is that you can find out the competence of an individual without wasting 5 hours of their time on a pre-interview puzzles. If they pass your phone screen and interview, you should be fairly confident that they are a competent programmer. I'm hard pressed to believe that someone who passes famously difficult interviews (for example Palantir) would find the puzzle anything more than busy work.