How I build and run behavioral interviews
benkuhn.net
benkuhn.net
How about, instead of asking "Tell me about a time you...", and you presuming to understand the situation so well that you can make judgments about nuance based on the off-the-cuff example you asked for, and trying to cut them short from "rambling"...
You instead ask them "Say you have a situation like X; how would you approach it?" And you can change the situation by adding information, "What if the report responds Y?"
Then both people are operating from closer to similar information about the situation. (Though it's still possible that the interviewee understands something about these situations in general that the interviewer doesn't.)
This also avoids dredging up past unpleasant situations (someone with more experience will have handled more unpleasant situations, but that doesn't mean it doesn't invoke a somber mood, if they're not acting or oblivious).
It also means they don't have to also think about how much they can say under NDA and being discreet about personnel matters (while a poor interviewer might take hesitation or choosing words carefully as interviewee trying to put themselves in the best light or keep a fabricated story straight).
An expected objection to this approach of spinning a hypothetical situation is that candidates might just say what they think are the correct answers. But knowing the correct answers is at least half the problem. And what do you think many candidates are doing in the interview anyway, if they are the kind to know the correct answers, but not follow them in real life.
should add that the more you dig, the better you can identify where truth/embellishments lay. a reaaaaally good liar is rare and you can somewhat tell across differing kinds of quetsions where the red flags are. just need to create a robust system of questioning. this all sounds dystopian as i write it, but unfortunately that is the nature of interviewing
So many candidates torpedo their own interview by doing that "Word Salad" dump when you ask them a question. It's like they have a bunch of unstructured responses prepared for any topic, and if you mention one of those topics in your question, they just unload the dump truck full of words and ramble until you tell them to stop talking. Always looking for techniques to steer candidates away from this behavior which sadly is happening more and more.
And if they don't know how to interview people (which apparently many don't), they should err on the side of being less aggressive and overconfident about it.
In job interviews, I guess part of the problem could be that we've institutionalized "interview prep" rituals now, and people train for the rituals, including things like the "correct" answers to behavioral questions, and even the right keywords to hit.
So, people not being confident of the answer they're trying to get correct might kinda spam the Word Salad of keywords, either as panicked flailing when they think they should be talking, or because it's a conscious last-resort tactic (like the standardized test prep classes that teach you what to do when you don't know the answer but want to maximize your score anyway).
One time I remember going kinda Word Salad myself, was in a non-interview meeting, when I was escalating a serious issue through official channels, a long time ago. The official in the meeting was (unbeknownst to me, when I was going to a ton of work to get this meeting) secretly misaligned with their ostensible role, visibly very hostile to me from the start, and misunderstanding something. There was insufficient bandwidth to keep up with correcting them, and I realized that the situation had just taken a very bad turn... so I started desperately using rapid vague hand-waving generality high-level summaries. Maybe this Word Salad-ish instance has some overlap with some job interview Word Salads -- it's not necessarily always that the interviewee doesn't know the answer, but maybe the interviewer isn't getting something, or is asking a poor question, or seems negative towards the interviewee?
They are looking for a nice story where I had conflict with my manager I used STAR to solve it and everybody was happy.
So interviewees have to prepare such stories, remember the details and adjust things to make them look good. Because saying "I don't really remember anything like that happening recently to me" or "We sat in stunned silence for a minute and went on with the meeting" doesn't cut it during interviews.
Maybe managers ask these "personal conflict" questions so much because that is a bigger part of their job. But for ICs (mostly anyway) they are pretty uncommon.
This is the opposite to a behavioral question, as given by the context.
everyone has a plan until they get punched in the face
that’s why you ask what happened historically vs. what you would do
if you ask me what will i do if you put a cookie in front of me when im on a diet, id say id turn it down
historically, ive never turned down a cookie
A bigger problem is that you have a lot of people getting into fights without even being able to think of a credible plan in theory.
Tell me what your ideal self would do, and if you have great answers, you're ahead of maybe most people. Even if you don't know for certain that you wouldn't freeze the next time you're punched in the face.
Also, the fighter who can relate multiple times they got punched in the face and were stunned might still be a better fighter than the one who cherry-picks a time they didn't get stunned and can spin a story of the plan working out well. (ObSeinfeld: karate class. "https://www.youtube.com/watch?v=2nuOtsgHjdY&t=3m17s")
This is illustrated by my single favorite interview question was someone being interviewed for a senior devops role. The question was, "What was the worst disaster that you've personally caused?" The purpose of the question is, "When the shit hits the fan, will you be in CYA mode?" The only wrong answer was no answer. Anyone who has worked with production for a long time, has made mistakes that cause a disaster. You should have a good example, explain how you screwed up, and what you learned from it.
If you ask people whether they will behave defensively. Everyone will say that they don't. Ask them to come up with a hypothetical where they don't behave defensively, and they'll give you a great answer. But ask them to remember and talk about a situation which it is natural to feel defensive about, and you'll get an honest answer.|
That particular question extreme. If you haven't worked with production systems for a long time, you probably don't have any major disasters. But the same idea applies. And the best tool interviewing I know of to get there is the STAR method. In turn you ask for:
(S)ituation. What was the situation?
(T)ask. What task were you given in this situation
(A)ction. What did you do?
(R)esult. What was the outcome?
People have a really hard time faking it. Seriously try it. Interestingly, the converse is also true. If someone asks a question, a STAR response looks good. So much so that interviewees are coached to do it.
If the devops question is off limits, what specific questions can you ask? Well here is a real example. Amazon has a set of leadership principles, you can find them at https://www.amazon.jobs/content/en/our-workplace/leadership-..., by which they judge employees. If you interview with them, the recruiter will tell you to prepare examples for all of them. You'll literally get questions of the form, "Tell me about a time in the last 5 years that you demonstrated (principle X)." That then walk through the STAR method.
I have many complaints about different parts of Amazon. I have no complaints about the effectiveness of their interviewing system.
The Amazon interviewing system came across as uncaring big-corporate cattle pipeline. So it was effective in making me withdraw my application. (Maybe that was a good thing for both parties.)
> People have a really hard time faking it. Seriously try it.
I know multiple storytelling-skilled people who I suspect can fake this easily. Most of them seem to go to pains to only use their powers for good. Interestingly, those ones have a very negative reaction when they sense someone being dishonest/manipulative. Though, the one who hasn't always used their powers for good... is CEO for something you use. :)
I wonder whether some of the differences in how people think about interviews are due to varying strengths of the interviewer. For example, I wouldn't be surprised if those storytelling-skilled people are better than average at detecting when someone else is fabricating elements of a narrative.
I suspect that people who readily jump in with an answer - are the ones you claim to guard against, those who drilled and trained for giving responses to these interviews.
Since then I have a bunch written down that I review before I interview. It's like leetcode questions, if you are going to get them during the interview you study for them. You don't expect to naturally solve them without prep.
And a lot of interviewers (at least in my limited experience) are quite okay with you referencing your notes during the interview.
Essentially every single interview/recruiting process is selecting for someone who is good at going through the process, rather than selecting for someone who is actually good for the job
It’s pretty frustrating and incredibly time consuming
You could be the perfect person for a job, but if you haven’t practiced interviewing for a while, you’ll most likely not get the job
And the opposite is true. After enough interviews, you’ll get practice, gain confidence and will start looking like the better candidate, even if you are not the best for the job
>Vague platitudes: some people have a tendency to fall back on vague generalities in behavioral interviews. “In recruiting, it’s all about communication!” “No org structure is perfect!” If they don’t follow this up with a more specific, precise or nuanced claim, they may not be a strong first-principles thinker.”
I don’t think that conclusion follows. I think what is usually happening there is that the person is saying what they think you want to hear.
Ironically the phrase “first principles thinker” sounds like vague business speak to me.
From other hand if it is so hard to get some kind of behaviour from an interviewee then maybe it is really not their kind of behaviour? Not something they would do naturally without nudging?
Judging by my own experience, it is hard to me to hide my normal way of thinking, it needs some conscious effort, and if given an excuse to talk along my normal line of thought I would do it. And if I was nudged to do it and I didn't because I thought it is not the right time or place for it, then it would be a pathetic inability to read a situation I'm in.
Anyway it is all probabilistic judgements, you cannot get anything certain in a psychology. But if you got a bunch of probabilistic signals then you can decide probabilistically.
And you cannot get anything "objective" in psychology. Everything is subjective, even formalized tests. To this day people have no test that allows to decide if there is a human on the other side of a communication. The best we have is Turing Test which is laughable excuse for a test, not an objective measure. We cannot reliably measure a difference between human and non-human, how could we hope to measure reliably a difference between a good hire and a bad one?
We can’t. That’s my point.
Hubris in the sense that they really think that they are doing something right, so much so that they blog/post about it.
Yet, if I take this post as an example, I notice a lack of self-reflection about bias en why you would interpret behaviour a certain way and not another.
In the end, it's all subjective, and everybody is doing their best, but how valuable is it really?
Especially, if you don't know anything about the company/organization that this person is hiring for?
Also,
> To get a strong signal from a behavioral interview question I usually need around 15 minutes
I laughed out loud on that one. I also need about 15 minutes to get a strong signal from any interview question for any candidate, because I am biased and then I challenge my bias for the rest of the interview, because I don't think I'm smart enough to have cracked the 'hire the right people' code.
> doing this stuff worked better than not doing it
Better for what? Is there a follow-up on the performance of the hire based on doing vs not doing this? Or can it be rephrased to 'doing this stuff gave me a superior sense of confidence about how well I interview people'
For example: employees from large companies may claim credit for substantial initiatives but when you drill in they had only a tangential involvement. They don't intend to mislead, thats just the language people speak inside of large companies, and behavioral interviewing can help translate that into more objective information.
> making their reports feel psychologically safe
> ask how they thought their report felt after the tough convo
> bad answer = not sure, or saying things in a non-supportive / non-generous way
I wouldn’t feel “psychologically safe” with someone who expects me to read minds. How can you ever be sure about how others feel as a manager? I check in with my reports and their colleagues but I’m still never certain. Your reports don’t want to be fired and as such every interaction you have with them is flavored with coercion.
If I asked this question and they didn’t say something to the effect of I’m not sure, that would be an enormous red flag to me that this person doesn’t give others proper agency.
Where is this evidence in this post? What beneficial effects did your new behavioral interviewing technique produce?
I've missed out on hiring good people who were nervous or worried about doing a bad job in the interview. I'm sure I still do miss out on some but I've found a method that lets some people feel more comfortable without being overly difficult on me and can live with that.
This is exactly the feature that allows people to distinguish a genuine recording from a faked one in Iain Banks' sci-fi series "The Culture"---it's impossible to fake a recording taken by a (machine) Mind because there's too much detail to fabricate.
Making sure interviewers show genuine interest, and are open to different candidate background is one of the most difficult things to guarantee when you have many people interviewing.
To me, if you are using interviews to determine the best way to spend your $salary_money on a person to get $outcome_stuff, then you really can't be walking into it without first introspecting on the knowns, unknowns, and unknowables of a decision. I'd hate to estimate potential team dynamics from interview responses, but I can absolutely validate that the candidate knows facts they need to know. But, it would take the preparatory effort to understand those things myself.
The major thing I take away from this post is the conviction that hiring approaches really don't seem to have progressed.
Can behavioral interviews consistently identify those people, filter out their opposites, with minimal false-positives?
Anyone can write a blog post about what they think is a good interview but there's no empirical proof that their interview technique works.
Figure out what might be meant by "a good employee." This is hard. Maybe factor in tenure, promotions, projects delivered, peer evaluations, etc. Develop of set of measures for an employee's overall all-time job performance.
Now take each candidate's "interview binder" and map it to some set of ratings (coding proficiency, favorite ice cream flavor, etc.). Set thresholds for each rating initially to reach hiring targets and some vague notion of an acceptable employee profile.
Always allow some small percentage of candidates below the thresholds to receive an offer.
Correlate the interview rating dimensions with the set of employee performance measures as employees work through the job.
Adjust the interview thresholds and experiment with adding or removing dimensions as you learn which things in the interview are correlated with "good employees."
Maybe this is how it's done when it's done properly, but it's the "extend offers to a few people below the cut, and see if they fare worse" idea that I like.
However, in some places it seems to be too cavalier -- in the sense that it suggests we now have a magic prism with which we can "diagnose" the candidate's shallowness or lack of candor. When really they're just saying something basically innocuous. Cause it's like, been a long and mostly boring interview process so far, has it not, and they're trying to slog their way through the canned / pre-programmed (if arguably necessary) part of the process like you are.
For example, under "Things to watch out for":
High standards: if they say there’s nothing they wish they’d done differently, this may also be lack of embarrassing honesty, or not holding themselves to a high standard
Or they had a genuinely shitty experience (with high costs to finances, relationships and/or health) that was 80 percent out of the blue and beyond their control. And they'd really rather just move on. Or if they did tell you the real context and their real reasons for doing X -- you'd very likely ding them (quite likely fatally) for doing so.