Comment by btilly | Hacker News Reader

Comment by btilly | Hacker News Reader

Parent Full thread

Here is a simple example of how to automate this for developers.

Everyone who applies is sent a link. That link contains a problem description, and a stub program. You're supposed to write the program and send it back. When you send it back, they compile your program and run a standard set of unit tests on it. If it passes the unit tests, then a human looks at it. Generally the code will be "good enough" to bring the person in for an interview.

In the interview you will ask the person enough questions about their solution to give you confidence that the person you are interviewing actually wrote the code you are reviewing. This greatly shortens the interview process.

This is not a hypothetical approach. I know of more than one company that has actually implemented this.

You might think that it takes a lot of developer time to implement, but it doesn't. What you do is have a developer come up with a reasonable project that they think can be done in a reasonable amount of time. That developer then solves the problem. You take their solution and stub it out - that gives you the initial stubs people can download. The unit tests that they developed are the unit tests that you will use. And now you're off and running.

The problem is that the people you'd most like to hire probably have little interest in taking a programming test. So you're just filtering them out and left with the dregs. For this to work you need one of the following:

* Problems that are so interesting that people would hack on them just for the fun of it, even if they have no interest in the job.

* A company that's high profile enough (and in a good way) that they can have slightly abusive hiring practices and still get an abundance of good candidates.

* You're looking for inexperienced or desperate people.

The canonical example of using programming problems as a pre-filter was ITA software, who had both of the first points covered. There's some great discussion on this in the HN archives, e.g. http://news.ycombinator.com/item?id=3429231

>The problem is that the people you'd most like to hire probably have little interest in taking a programming test. So you're just filtering them out and left with the dregs. For this to work you need one of the following:

That may be the case if you're trying to headhunt experienced developers who are currently in good jobs.

But if you're an unemployed programmer, or an employed programmer actively looking for a different job, why wouldn't you take a programming test? Programming tests and technical interviews where you have to code are normal and expected for coding jobs, at least in my experience.

There's a big difference between a programming assignment used as a filter to decide whom to interview vs an interview question.

The hiring process is very expensive. But it's expensive in a fairly symmetric way. It's in the interest of both parties to stop the process if there isn't mutual interest.

Both the company and the candidate will spend a lot of time in interviews, so it's in everyone's best interests to make sure that time isn't wasted if there no (or almost no) chance of the person being hired / accepting a job.

There might be earlier steps in the process, but those are also symmetric. It'll take a few minutes of your time to send a resume to a company, but somebody there will be tossing away 80% of them after reading through them once. There might be a phone screen or two in advance, to decide whether there's any point in having someone come in for an interview, but that is likewise costly to both parties.

But a programming test as the very first step of an interview is totally different. Now the costs are asymmetric. The company has an up-front cost (potentially substantial) but spends little time on each candidate. The candidate spends the time up-front, and only later finds out whether the company is at all interested.

The candidate isn't invested in the process yet. So why would he spend an hour, when he has no idea of whether the company is interested? Maybe it's a really fun problem to solve. Or maybe he really, really wants to work at that particular company. Or maybe he is indeed unemployed like you suggested.

But someone currently employed is likely to have a different priorities from a student or someone who is unemployed, even if they are actively looking for a new job. How large a percentage of them are you willing to weed out completely just by making it harder for them to even enter the hiring funnel? 90%? 50%?

When there are 630 resumes in the pile, the read/filter will be done by a junior HR person who probably couldn't tell XML from SQL.

I'd rather be filtered by a programming test, where my programming abilities put me ahead of the pack!

With that said, most of the programming tests I've been given in the past have been fairly interesting so perhaps that biases my opinion. I'm certainly no fan of jobs where they send you a huge MS Word application form.

And I'd rather not work at a place where my resume alone doesn't warrant at least a phone interview :-) Either I really wasn't a good fit for the job, or they have a broken hiring process. If it's the latter, chances are that a company with that bad a HR department is also dysfunctional other ways.

Programming tests can indeed be interesting. Anyone feeling bored could do worse than to attempt solving something from ITA's hiring puzzle archives. But unless you're extremely certain that your tests hit the sweet spot, it seems really dangerous to make them a mandatory first filter. A test once both parties are at least a little bit invested in the process, sure. Or if resume screening really is a problem, allow someone to "skip the queue" by sending a test solution along with the application.

Normally if I'm sending off a resume, I know it's probably going to be automatically filtered:

    if not resumeText.lowercase().find("postgresql"):
        reject()

(Too bad I only bothered to put "postgres" on there.)

If there is an automated code test, I know exactly how the filtering is going to work. I write code, it passes unit tests, no auto-reject.

Sounds like a win for me. Of course, unlike many applicants, I can actually write code that works.

I disagree.

One of my top concerns if I'm interviewing for a company is, "Are these people going to be sufficiently competent to be fun to work with?" If the interview doesn't provide a good enough filter for incompetence, then I am going to not want to interview there.

If my first experience demonstrates that only competent people will get through, then that is going to improve the company in my eyes.

That said, it will slow hiring down. If you want to hire a bunch of people, you can't put a restrictive filter like that up. But if you just need a few, it can work out in your favor.

This is great solution and the impetus behind Interiew Street.

And while HN is a very programmer centric culture, most jobs can't be automated in this way. Look at the OP's job description, how do you automate that? And how does someone like the OP automate that process? Believe me, I would love to live in a world where every position in every company is able to command a great developer to build a customer filtering solution, but most companies in the valley (much less out of it) won't invest this time/infrastructure. Hence the rise of BS keyword filtering systems. I'm personally very interested in solutions that can make this process better, but it's a highly non-trivial problem in most subject areas.

Reply on news.ycombinator.com