"I have a female friend. She asks every guy she meets at a bar two questions:
1. Do you have a passport?
2. Do you have a Kindle/Nook/library card?
Why does she ask these questions? If the answer to 1 is no then he doesn't like to travel abroad and if the answer to 2 is no then he doesn't like to read."
In essence, she determined two quick filters that were the most important to her.
I had a manager who had a similar test for candidates. He would give them a 5 question tech test where the first question had a "trick" item. Specifically, sort multiple 5GB+ files on a box with 1GB of RAM but 100GB of disk space. If the candidate got the first question wrong, that was the end of the line for the candidate. Why? It showed that they:
a. Didn't have attention to detail
b. Didn't fully understand basic technical constraints e.g. in this case RAM is limited but disk is not.
One other point I always like to make is to screen candidates in parallel. For example, if your process is to bring someone in to the office and then give them a tech test, that's a serial process.
On the other hand, if you give the tech test to recruiters and say "Have candidates fill this out and then send it in", you just created a parallel process to screen candidates.
I've also heard people say "What if they look up the answers?" to which I reply: if you give someone a test with google-able answers and they still get it wrong, you just filtered out a pretty bad candidate. Win for you.