Literally the first objective I had when I took over recruiting was to reduce the time demands on Matasano's hiring process, and the work-sample process we came up with slashed time demands by more than half. Candidates we
passed on wrote us to say how much more they liked how we hired than other companies. That's because:
* We demanded the same or slightly less time in total than other conventional-interviewing companies did.
* The scheduling of our demands was totally flexible, unlike interviews, which are rigidly scheduled. If you're a morning person, and you have next Tuesday free, that's when you threw the 2-odd hours you needed at the challenge.
* Obviously, it's easier and less stressful to put effort in from your couch than in an alien office environment with people staring at you waiting for you to answer properly.
* The challenges themselves were fun. They were real work: we didn't have people literally slaying dragons (they were breaking a web app and a client/server app), but they were the distilled enjoyable essence of that work with most of the BS removed. In fact, if you didn't find the challenges fun, that was a huge signal that we were the wrong job for you.
* If you did decide we were wrong for you, you could stop at any time --- randomly, on a Tuesday night, with a slice of pizza in your mouth --- and have none of the weird social pressures that would make you sit through a string of pointless interviews. Which is an experience I have had, more than once.
Regarding certification: pure, chill-filtered hatred of certifications is what got me into the part of software security I wound up in. There is zero chance that we are going to build something with the market dynamics of a certification.