I should interview potential new candidate for the team I work in, and based on the many opinions on "fixing the job interview" here on HN I figured I will solicit some advice.
My situation looks like this:
* Candidate has been pre-screened by HR already
* I have read the CV
* We have 1.5 hour video-call scheduled
* Predominantly a dialogue
Any tips how to best lead this dialogue?
Now to the actual advice. Make it a technical conversation. Talk about systems design. Turn the video-call into a screen-share and review some code together. A 1.5 hour open-ended conversation sounds like hell on earth and doesn't let you evaluate much about the candidate other than how much of a conversationalist they are. Dialogues are fine, but at least make it a technical dialogue through which the candidate can showcase their expertise and skills.
You probably have time for some of the behavioral stuff other commenters are suggesting, but beware--behavioral interviewing is more complicated than people are making it out to be. Are you hiring a programmer or a storyteller? You have to figure out which behaviors you want to evaluate people on and lead them to give you meaningful information about whether or not their behavior fits the desired pattern. To complicate matters, this is an inherently adversarial endeavor, since the candidate isn't going to want to tell you anything that will make you not want to hire them. Behavioral interviewing is great, but it's going to take a lot more than asking random people on Hacker News how to do it before you get good at it.
Don't waste your time. This isn't going to work. Even the most careful, thoughtful interview processes proposed here are essentially suggesting you hire by gut feel. Make sure you eat a full lunch before the interview! That way you'll be more open-minded and accepting! Wait, no, starve yourself for 24 hours before the interview. That way you'll be more rigorous!
There are two very simple suggestions I can offer for really improving interviews:
1. Replace as much of them as possible with work-sample tests: small, carefully scoped projects that closely resemble actual work you do in your day-to-day, given identically to every candidate. Note the word "replace". Work sample tests do not work unless they offset actual interviews. Lots of people give homework. They're still hiring by gut feel. Don't waste the candidate's time unless you have an exercise that will count as a vote in the final hiring decision.
2. Standardize your interview. Every question, written down. Before you give the interview, write a rubric: I expect answer X for question 1, answer Y for question 2. You don't have to be completely straitjacketed by the rubric, but it should be solid enough that any of your interviewers could give the interview, create the candidate measurement, and hand it back to you on paper, and then you could make a hiring decision from it without knowing who the interviewer was.
I did both of these things at Matasano. They worked wonders. I have since talked to several other shops that had similar processes. All of them swore by them.
The dirty open secret of tech hiring is that it is overwhelmingly optimized to hire people who are good at interviewing. Everybody in this industry knows a bunch of very "successful" people who have bounced from job to better job on the strength of their persuasiveness, calm under fire, likeability, and vocabulary. The reason we remember those people is we've had to work with them, in the real world, where likeability and calm under fire take a back seat to "actually being able to solve problems day in and day out", and the experience hasn't been pleasant. Try not to add to that problem.
I like the idea of the work-sample tests, but I don't know how to work them into the process yet, and I don't think I would manage to create good enough samples for this particular interview.
Standardizing the interview sounds like the best approach to try currently. Thanks for your suggestions!
Isn't it basically home work for them?
How many are interested in spending their time doing a project for you?
Or rather, my question is, how do you remove those perceptions about it?
Start with this script - "I've read your CV, know you've met with HR -- what questions do you have for me? what areas would you like to discuss?"
The quality of their questions tells you a great deal about their intellection, motivation, and business acumen.
Incidentally, George Bradt offers excellent advice on evaluating candidates> http://www.forbes.com/sites/georgebradt/2011/04/27/top-execu...
- Chiefly, you get to find out how interested and motivated the candidate is.
- It's disarming but in a good way: it removes the "grilling" and "test" mentality and starts a conversation.
- The quality of the questions that the candidat asks will answer the problem of knowing how well the résumé is subtantiated.
I'm going to have to experiment with this one. Thanks for the tip!
I use the behavioral interview process.
I spend 10 minutes reviewing their resume before the interview. I introduce myself to the person at the start of the interview, about 5 minutes. Then I explain my interview process to them so that they know what to expect though the interview, it helps put people at ease to understand what is coming.
Then I dig through their history, asking who they've worked with (yes, get names, people are inherently more honest when they use real names). Ask anything you want here. For technology I find it useful to dig into the areas they find most exciting, and discover how well they actually know those areas. This should last about 20-30 minutes. I try to focus on the thing that are important to the job being interviewed for, but this is also a great place to see how personable they are.
After this I turn the floor over to them, allowing them to ask any questions until the end of the interview, generally 10 minutes.
In total this is structured for an hour, I think more than that is too much, personally.
10 minutes sounds like a really short time for candidate driven discussion, IMO. Personally, when I've been the candidate, I treat the interview as a two-way discussion. I'm I suited to the job, are they an employer that I feel comfortable with.
If there is no time reserved for myself leading the discussion, or I feel I would have nothing to discuss anyhow, is an indication to me that no matter the technical requirements, the position is not a terrific match.
The best interview sessions I've had have evolved into a detailed discussion about technical details, business angles and organizational skills, lasting well over an hour. But those interviews were in a situation where the candidate pool was very limited.
I could see this working better for a more senior positions :)
* What's the biggest project you've worked on so far?
* What's the worst outage you've had to deal with?
* What do you think about [framework] (vs. [framework])?
An organic conversation with someone on the right topic(s) will give you far more insight into their suitability for a role, and their own personal career goals than any list of questions. Communication is one of the biggest factors when it comes to making a team work well.
I agree that the communication is a big factor. On the other hand, we had few previous candidates whose performance was really poor after we hired them.
Stealing the conversation starters. Thanks.
On the other hand, I wouldn't mind short-circuiting the interview.
1. Can this person do things with software?
2. Do we want to work with this person?
We screen for 1 in a short tech screen. We screen for 1&2 by having onsite pairing sessions, typically a day.
Mysteriously, having people work on real work with real potential co-workers in the way we actually work is a really good indicator of whether people can work on real work with real potential co-workers in the way we actually work.
With "padding your resume", do you mean when candidate has "Proficient in Javascipt" in his resume but means "Can slap a jquery on an html button", or alike?
Give them tasks which are as realistic as possible.
This is my process:
1) Start with a easy (but real) task to get the candidate over their initial period of nerves.
2) Provide positive reinforcement.
3) Gradually ramp up the difficulty of the task, providing positive reinforcement and/or guiding the candidate where you think they've gone wrong.
4) Towards the end put 'traps' in the tasks which you have encountered in real life - i.e. problems which surprised and stumped you. This is an opportunity to spot exceptional candidates who can see the traps.
I'd skip literally all of the conversation and video aspect and communicate solely with text based chat and screen sharing. Unless you have a real need to see the color of the candidate's skin before you make a decision, I suppose.
I don't know anything about your specific business, but as a senior level engineer, I need to feel I have an ownership of my role where I work. This includes understanding the business environment, the organizational dynamics, what is expected of the position and so on. I don't want to work at a position where I would feel like a replaceable drone. This all is highly personal, and context sensitive, of course, but I would not use this script to hire senior personnel.
If the role of senior management is actually more about bullshitting then by all means just make the interview all about having a chat.
But, like I said, contexts are different and those things might not matter.
I think you are wrong. An exceptional programmer who cannot communicate is basically useless.
I think the importance of developers' communication skills is context dependent. It's more vital in some roles than in others.
What is your role in the interview process? Are you technical and is the company depending on you to properly vet the candidate for technical skills? Then you need 90 minutes of concrete coding evaluation, etc. If you're just in the loop to make sure the candidate "fits" the culture, I suppose you could have a dialogue about anything... e.g. "soft" questions like "name a time a you and a coworker didn't get along and how did you handle it?"
If it's a technical programming candidate, is it web front end work? CRUD LOB? back end engineering and scaling? custom algorithms work?
The way you've posed your question is really wide open.
Many devs don't want to tell their life stories (introverts eh, go figure). And the devs you want don't want to fluff for 1.5 hours to satisfy your arbitrary time allocation or inane questions.
Focus. You're looking for:
1. Does the candidate have the skills I need. - Ask about their projects and test their skills.
2. Does the candidate fit with the team/culture. - Interview them with some or all the team, take them out to lunch with the team. See if they gel.
Look at the candidates work history that would be valuable to your company. Work down a checklist of things you need. I have a PM who sends the list to candidates, they appreciate the contact and it has led to hiring some talent we didn't deserve at that salary range.
Not sure how to divine if candidate is capable to learn writing code in language we need in a month :)
Use this process and you're most likely going to wind up hiring the candidate with the best conversation and storytelling skills. There are a lot of people who can make themselves look good on paper and in conversation that don't actually have a lot of skill.
If you're determined to go through with it anyway, my suggestion would be to probe deeply into technical details and the engineering decisions they made in the past projects they worked on and what the results were and also to probe for their decision making process when solving a hypothetical or real engineering problem that you have. Note that asking people for deep details on past projects has its own problems; even when the candidate is attempting to answer with full honesty, memory is fallible and highly variable between people.
Like the strategy of probing "deeply into technical details and the engineering decisions of the past" as a mitigation, thanks!
I don't actually think memory would play that big of a role, if candidate solved interesting problems in the past "How would you solve it now?" might be more interesting question than "Do you accurately remember your past solution?" :)
If all you're going to do in that dialogue is have a back and forth at a high level instead of looking at and writing some code, you're not going to be selecting well on a technical basis and you're really getting a randomized sampling of a distribution of technical skill of people that are applying for the job.
That is actually what prodded my question, because out of the colleagues we hired this year, half of them are amazing and half of them left because they couldn't keep up, so I wanted to know how to improve the "90 minute dialogue and go by your gut" interview process.
1. describing the position early to not "waste time interviewing people who don't want the job."
2. ask a few trivial and easy questions first
Thanks!
One time, in response to a ten minute description of the product, I asked, "So what distinguishes your product from just another CRUD app?"
The response: "Ummm... Well, actually, nothing. That's pretty much just what it is."
"I withdraw my application. Thanks for your time. I would suggest that you revise your advertisement to target candidates with less work experience. Good luck."
Based on my model interview, that is an amalgam of all my interview anecdotes, that interview format probably saved me about 45 minutes of a technical grilling before I finally got the opportunity to poke through the more obvious lies in the job advertisement.
Make sure to give candidates time to ask questions before you run through your entire spiel.
I'm a YUUUUUGE Belieber (see what I did there?) in the power of stories. It breaks the ice, and gets both parties into a rhythm of exchange.
With the CV as a "script" I try to get people to tell me a story about how they came to be sitting in this interview with me. I have them start at the beginning of their career and quickly move forward.
I ask open-ended questions and try to get them off topic -- talk about the city/country they worked. Or the tech stack.
Also, get them to tell a story about a side project.
ANd share one of your own. Like the time I thought self-hosting DNS in out office was a good idea, and how that blew up in my face...
Or, maybe you both should crack open a beer, screenshare some youtube clips, and share some favorite vids for 5 mins.
MAKE IT AS HUMAN AS POSSIBLE.
I've definitely hired candidates who were less technical then I thought.
I've come to learn that , for me, a cultural fit is as important as skill.
If the guy I hire has awesome cultural fit but lacks the skill, then we work to bring him up. Good cultural fit means I'm more willing to invest in him, and he will want to push through the things holding him back.
I also like to look at code samples and commit messages if possible.
They have to be good communicators.
There has to be chemistry in the team.
I've worked on teams where one guy is a social dud. It drained the team to work with him. That hurts morale.
Especially using the CV as the script/primary guiding line of the interview. Thanks.
Not sure about the beer drinking, though we have almost found a potential candidate working a beer tap at a bar once :-)
Spend 1 hour pair programming (using a remote collaboration editor, something like collabnet) on a real world problem, perhaps a feature you've already built and shipped.
Spend 15 minutes on a STAR[1] formatted question, such as
"Tell me about a time you had to work closely with a developer, but it was a frustrating collaboration."
Spend 10 minutes letting them ask questions and try to leave them with a positive brand experience and feeling good about the interview.
Interviewing is super difficult and uses skills not often used as a coder.
[1] https://en.wikipedia.org/wiki/Situation,_Task,_Action,_Resul...
Thanks for the STAR mention, haven't heard the term before.
Once you know what you want to learn from the candidate you can come up with a list of questions. Think hard about them. Will they really give you the information you want? Rank your questions based on how likely you think they are to give you clear information and how important you think that information is. You'll probably want some redundancy in your questions in case you don't have a clear picture based on the first answer.
> Choosing personnel selection procedures could be so simple: Grab your copy of Schmidt and Hunter (1998) and read their Table 1 (again). This should remind you to use a general mental ability (GMA) test in combination with an integrity test, a structured interview, a work sample test, and/or a conscientiousness measure.
That's it.
Just have a conversation.
From rule of thumb this can go from 30 minutes to 3+ hours. So don't block off time ahead; if this person is valuable to you, give them all the time you and they need.
Edit: I do not buy in to multiple interview rounds. If secondary interview is needed, only for a significant role.
Multiple interview rounds indicate no one is taking responsibility.
If it's a hands on programming role, not seeing them code (or some of their existing code) early on in the process could make for a costly mistake.
I, on the other hand, try to tailor the interview and questions to what the candidate has stated they know or what they are proficient in to determine if they truly are knowledgable in areas they say they are. I'll also ask questions tailored to the publicly available job description since, by applying to the job, you implicitly said "Yeah, I can do that", but put less weight on these questions if it doesn't match up with their stated skills and knowledge (so I'm kinda just probing; you'd be surprised at what some candidates know, but don't put on their resume/CV). The prescreen should basically mean, if you pass this prescreen you are eligible for this job, but we still have to validate that what you said about yourself in the prescreen and resume/CV are true.
So for example I'll usually ask questions about object oriented design patterns. When I ask someone how many design patterns they know and to list them and they give me three (and Singleton and Factory are 2 of them) that's totally fine. I would've only been able to give around 3 a couple years ago before I studied hard for interviews. But if their resume says something like "Strong object oriented design skills" and they can only give me 3 design patterns, then that's a mark against them because if you claim to be strong in object oriented design I would expect you to know more than the standard rube (like me) that can only rattle off 3 or 4.
If their resume talks about doing distributed systems I'll ask questions about distributed systems: what's good design, what are common pitfalls, ways to avoid or mitigate those pitfalls, etc. C# is your number one language, then how are parameters passed to methods in C#? What does it mean that C# is a managed language? What sorts of things aren't managed for you in C#?
As much as possible I try to include a question about how they used a skill or knowledge in a real-world application disregarding whether the end result was good or bad. Then discuss why effort ended up good or why it was bad and how they would've done it differently given hindsight.
If the pre-screen didn't cover it then give them a programming problem to make sure you weed out the candidates that can't FizzBuzz their way out of a wet paper bag. But don't make the programming problem the entire focus of the interview. I'm currently scheduled for a Google interview which is going to be 5 hours of non-stop programming puzzles that you have to solve while the interviewer is looking over your shoulder critiquing everything you do and expects you to figure out the "trick", code it perfectly, and do this while simultaneously explaining everything that you are doing. Plus the problems they come up are purposely designed to be tricky and sometimes even confusing. The funny thing is I bet any of those engineers would crumble under the pressure and ambiguity I have to work with right now every day and yet they'll be judging me on how I deal with tricky problems under pressure. You can see I'm not a fan of the traditional coding interview.
TLDR: Tailor the interview to the candidate's purported strengths. Ask for real world experiences and discuss the positives and negatives of the experiences. Probe for knowledge and skills stated as required on the job description, but don't put as much weight in them if the candidate doesn't list them as a strength. Check for basic coding capability, but don't make that the focus.
Let it flow from there, either through asking more probing questions or just listening.
I don't believe that all of this is necessarily applicable to a one and a half hour interview.
As far as programming ability, I think that if your company does any pair programming then this is a good exercise during the interview. I interviewed with a company where I was asked to use a computer without internet connection and solve the problem. The problem the script was vague, and my pair was required to be my research arm.
Regardless of if you are doing pairs programming. If you ask a scenario question and you are looking for a specific algorithm, data structure, or solution, this is okay. You are however strongly suggested to stop them before they waste your entire shared time and lead them down the correct path. With obvious hints. Requiring your candidate to discover that you're looking for a breadth first search when you ask them to implement a text based adventure game, doesn't make sense.
I believe that soft topics have become undervalued. Most the time a soft topics interview is done by a manager or an HR person that you're likely not to work with. My best soft topics interview was done with a panel ranging from developer through HR. They asked me about instances of conflict resolution and non conflict resolution that I had experienced. Having the panel meant that many people, caught phrases that I used and asked me in Greater detail that I think would not have happened normally.
It's also important to know the limits of somebody's expertise. There are a lot of topics that your candidit just won't know. But this information is mostly represented by what is missing from the resume. You can ask ask for anything that is missing from the resume, just expect the answer no. For topics they do list on the resume, have somebody who understands that topic discuss it with them. Either the interviewer should stump the candidate, or the other way around. This helps you understand how the interviewer handles somebody else's knowledge gaps, or their own. You can also accomplish this by asking them to give a presentation on the topic of their choice with open invitation to your company. This however can be very time expensive.
I had a company recently asked me do I know how to use an advanced IDE, or terminal IDE. Although my answers were yes, I feel that I miss understood their goals. What they were more concerned about, was whether I knew how to set up my IDE for a large , existing, code base. I think that this is an important skill to have, and I think that the question can be made more clear.
If you want good candidates - don't have HR screen anybody, don't spend 1.5 hours, that's way too long, and if you don't know what questions to ask, you're not fit to be doing this.
In other words - if you have to ask, you're doing it wrong. The company should hire a senior developer who already knows.
First, pedantically, they're the same words you used initially so they're not really "other words."
Second, someone at OP's org thought they should be included in the process and OP is asking for tips on how they can maximize the value they generate. This is exactly the situation I'd love to see if I were OP's manager.