Does stress impact technical interview performance? (2020) [pdf]
chrisparnin.me
chrisparnin.me
These whiteboarding interviews where the interviewers pose some random question does nothing for either party.
Let's say an interviewer reads your resume and is interested in a project. The interviewer comes to the interview prepared with some questions. They ask you to start with a high level summary of the project. Then, ask you - "What choices did you have to make during the project?". Then, ask you some specific questions about these choices. In each of these questions, the interviewer is not supposed to "evaluate your answer" but rather gain an understanding of your project and choices to be made. At the end of the interview, the interviewer would walk away with two things - (1) a clear picture of the project almost as good as the interviewee and (2) a clear picture of the design decisions the interviewee made (options they had and why they chose one over the other).
The above interview is a conversation and you are allowed to be silent. The process will be communicated to you upfront. In fact, silence is encouraged because the interviewer would much rather give you time to think over a spur of the moment answer.
It's astonishing how often you get nothing useful as an answer. Things like "my boss suggested it" as the reason for choices being made, or "not really" when asked about alternatives considered, or problems they ran into...
The hypothetical project whiteboard interview gives people who haven't done sufficiently interesting past projects and opportunity to show that they can think about them, at least...
That's not what they were talking about. The complex problem is supposed to be novel to the interviewee.
> In fact, silence is encouraged because the interviewer would much rather give you time to think over a spur of the moment answer.
Yes, they certainly say that. But for a normal person trying to figure out a novel problem, ignoring everything that's going on and sitting there silent for 30 seconds or longer while the person who single-handedly can determine whether you pass or fail - often depending on their mood - stares at you is incredibly difficult. And if their mood isn't good, they absolutely see the silence as not knowing the answer and push for you to move on. I had it happen to me on a Facebook interview. I personally can't concentrate enough to think through a difficult, multi-layered problem in that situation. It is in no way reflective of what I can do while working.
Online coding tests are more than sufficient. And a discussion afterwards about said tests is sufficient to make sure there was no cheating.
In particular, HR is expected to structure the hiring process in such a way as to avoid accusations of bias. In order to achieve this, they are expected to come up with precise criteria for hiring, which is at odds with anything vaguely qualitative or holistic, let alone ecological.
The result is that we strictly rank-order candidates using hard measures. In doing so we sacrifice the ecological validity of the test.
Luckily, I was able to remedy this a little bit by brushing up on common system architectures used by companies that I was interviewing with.
I usually preface it by saying that I will take a minute to write down a few ideas, so the silence isn't awkward. It is such a great way to engage your interviewer, while gaining a finer ability to choose the level of abstraction when you are working through a problem.
I love it also because it helps those who aren't native English speakers. The ability to visually represent ideas is equally distributed across language skills. It doesn't even need to be suited for a visual thinker. It can be math, venn diagrams, graphs or even straight up code.
Having reasonable social skills and the ability to not look like an idiot when you're on the spot are both super important to being a productive member of a business
Edit: someone has just downvoted every one of my comments in this thread and not left a response, why?
Difficult to impossible to reproduce results; practitioners with wildly different skills and biases; institutions with systemic biases; individual candidates who can vary wildly from day to day; cohorts that change over time and distance; a lay population with strong opinions.
We'd be better off selecting candidates randomly, but even this only kicks the can down the he road.
But it is most certainly not rare that you will be expected to reason about possible solutions to problem domains you barely understand using techniques you have varying levels of familiarity to an audience.
Maybe I'm wrong, maybe this is just my line of work, and if you're recruiting a person who pulls tickets and there's no expectations beyond that whatever. But I would be assed to hire someone who, presented with a question my team thought fair to ask in an interview, was unable to stand up and cohesively walk me thru their approach, reasonably attack some parts of that approach, and offer some insights to what pros and cons they see in their approach and others they might have passed up. How is that person ever going to help in a brainstorm if they are so socially anxious they cant speak to an audience? How are they ever going to help their teammates in a Code Review if they are unable to find their voice in front of a whiteboard-tier problem that every single other candidate is facing?
If your social anxiety is this debilitating, that you can't do your literal profession in front of a crowd who you purport to be on the same level as by virtue of interviewing with them, I think you have a problem yes.
Anyways that was a long winded way to say it is entirely possible for someone to be anxious and generally bad at interviewing while also being capable of being a productive team member.
And no, it's not your job to worry about other people's issues, but if you think that 'being stressed in an interview setting' == 'unable to collaborate' (this is what I read from your ridiculous strawmen), you're going to pass on a lot of good people.
No, it sounds like you didn't read the top of the thread, where we're specifically discussing Whiteboard Prompts. I agree interviews are stressful; I disagree with the argument that "Whiteboard Prompts" are so much higher stress than the rest of the interview that we can't conclude anything from them about the candidate.
I further argue that if one can't perform in a Whiteboard Prompt, wherein the entire goal is to have the type of conversation about a problem that engineers are paid to have about problems rather than coming to a code solution, than one will probably be difficult to work with in any situation which involves some pressure. Some classic examples of equally high pressure situations as talking to an interviewer you don't know but who is probably on your level include things I do each week:
* speak with stakeholders about progress
* opine to stakeholders about complexity of features, timelines and methodologies of their implementation, and possible blockers
* work with my team & extended team members to give Code Reviews, where often they are my direct report or someone else's direct report
So, I conclude that if one is able to excel technically, but completely unable to handle this socialization aspect of interviews, than in a large corporation where you frequently have rotating casts of external stakeholders, this implies you will not be fit for any of the work interfacing or collaborating with colleagues outside the team. This implies this person will be a low performer if hired into this team.
Call it a "ridiculous strawmen" if you want, but I don't see why we should completely discard testing of someone's ability to interact in new situations, otherwise surely I could argue in the ridiculously reductionist way you are interpreting my multiple, specific comments elucidating my stance that TripleByte has solved the interviewing problem.
The only point I am trying to make is that yes, interviews are very different than the other scenarios that you mentioned.
Here's one reason: in an whiteboard problem solving session, the other person has explictly prepared something that they know more about than you. On top of that, you are both aware of this fact. This almost never happens in a normal work-collaboration. A collabaration happens because both parties have something to offer. If not, they'd be explaining to you, and not the other way around.
No, not really.
I think people who have a weakness are most likely to attack, without evidence, things that expose that weakness as unfair. People in this thread are going so as to call the interview technique of whiteboarding a problem “practically inhumane”.
You are right that I think inability to talk thru ones train of thought on a new problem to them to be a signal of inability. I think whiteboarding exposes this kind of weakness. I also think it has many causes: extreme stress reaction, bad social skills, lack of domain knowledge, bad communication skills, and plain old stupidity.
So to read that “whiteboarding” should be disallowed from interviewing, because it is so much more high stress than every other part of the process, despite those parts also having equal importance in the overall stakes, really needs some elucidation. I accept anecdotes from workers who claim they are only low performers in the interview because of stress or anxiety, but I strongly doubt unmitigated social anxiety presents as a detriment to ones apparent job performance in interviews but not jobs at a high enough rate to give any credence to these claims without some sort of data driven backing.
Job interviews aren't perfect and everybody wants something different from them. But maybe the employer actually is looking for something you don't have and wouldn't it be better to find out before joining?
"The scariest part of all this is that people think this is a good way to filter candidates. It shows you that the company is populated by stress tolerant, herd mentality thinkers. I see them as a cross between army grunts and high school nerds. Simple fast logic, drawn to rules and systems, prone to being swayed by appeals to authority. Low sensitivity nervous systems meaning high stress tolerance and low ability to perceive, synthesise and abstract. Working with these guys is like wearing a straight jacket anyway - avoid like the plague. Unless you happen to fit that model, in which case go right ahead, you do you, but please be aware that there are other kinds of human out there who can add value outside of the cookie cutter."
But we have a problem where we are recruiting all kinds of personalities into a market and then after significant investment in education telling them that they aren't suited for the role.
Fundamentally what is going on here is a battle between different kinds of personalities and it's annoying that I'm in the minority. But there is an argument here to say that this isn't a very economically productive situation. You need all kinds of people in an organisation.
In the same way the minds of people are tuned to different levels of stimulation. People with low sensitivity are better suited to noisy information rich environments. People with high sensitivity work better in quieter less information dense ones. You might want a highly sensitive person to conduct a one on one interview, because they can better pickup the subtleties of the interaction. But you don't want that for a lawyer who has to work in a busy courtroom.
That covers perception. The synthesis comes both from the lower perception floor and from the tendency for those things that are perceived to perpetuate within the nervous system. A reactive nervous system carries waves upon its surface more readily. It reacts even to itself more strongly. The inter-neuron gain is higher. Consequently a highly sensitive nervous system (i.e. introversion) is associated with withdrawal from high stimulation environments and the perception and processing of subtlety.
The herd mentality comes from being less able to pick up the subtleties that let you know the group think is not an accurate picture of reality. If you cannot perceive the discrepancies then you cannot doubt the status quo.
So when you do public whiteboard interviews you are selecting for low sensitivity individuals who are on average less able to discern subtle patterns in perceptions coming in at low volume. Does that sound like the kind of person who excels at coding?
Although it might sound like someone who can work the least sub optimally in a noisy open office.
Additionally, if you find a whiteboard question to be high stress to the point of not being able to answer, while other candidates are smoothly answering it, demonstrating their chops both as a technical knowledge worker but also as a collaborative and communicative engineer, you bet your ass that's a giant red flag. Why should it not be?
Also, omg I cant believe you just used the word 'chops'. It's like a fighter pilot from ww2 just teleported into the chat.
I asked for a source that people find whiteboarding significantly more stressful than other forms of interviewing people are not complaining about here, and you gave some bullshit opinion that denigrates a massive amount of people simply for having stress management techniques / more advanced self-mastery.
Here's my opinion: technical interviews catch people who are not technically qualified, and whiteboard & other social aspects of interviewing catch people who are not socially qualified. I think that teams work together better with frequent and high-quality discourse. Do you?
I spent, what, 35 years. Maybe. Working on myself because I bought into the line that you can change your personality, that if you are not living up to what the status quo expects of you, then you need to work on yourself. And then I had kids. And I saw that they could not change. That their nature was not some malleable thing that listens to the whims of what the world expects from them. There are parts that can and do change, but the bedrock properties do not change, they are who and what they are and that is good.
You cannot step outside of yourself and admit that others may live in a different reality that finds things you find easy, difficult. And finds things you find difficult, easy. There are high stress tolerance people and they are great and good and useful. And you have low stress tolerance people and they are great and good and useful. But they are not the same, they have different abilities and to try to deny that is wrong headed.
If no, then no, it's not.
If yes, perhaps. Interview settings are still quite different than what they're going to face on-the-job, but you do have to infer from something, so I'm not sure there's any real alternative.
How do you collaborate with your teammates? I do it directly by approaching a problem in a shared coding or whiteboarding session, where we try to break it down to its barest bones and then implement logic to animate these bones. Once that is done, we begin to look at what types of higher-fidelity information we lost, and iterate.
This requires a good ability to communicate, write code in real-time, understand perfect is the enemy of good, and in general be able to simultaneously think deeply and support conversation / collaboration.
Of course, there are frequent async steps in such a workplace, but if you are the type of engineer who doesn't participate in these stages and instead can only pull tickets and write them to a spec, you are the type of engineer I don't want to hire. So I consider the signal:noise ratio quite high wrt watching someone melt under the same pressure as every other interviewee faced.
In a real life situation, both you and your partner will be generally familiar with the problem at hand; if not there's an expectation that the person with more knowledge will be explaining not just the problem, but also the solutions that are already under consideration, so that everyone is on the same page.
In an interview situation, the solution is obviously withheld because the goal is to make the candidate derive it, so any back-and-forth is artificial - the interviewee is the one who has to do analysis while being scrutinized while the interviewer has already done the work.
And of course, normally your partner doesn't have control over whether you get hired or fired. If you were dropped into a room with the CTO and told that you had to solve a new-to-you problem they had already solved or you would be fired, you would probably find yourself performing much worse than if your team-mate asked you to weigh in on the problem that had been bouncing around on Slack for the past few days.
Still stress free?
Weak strawman aside, I don't know what your point is. You can reference external things in most interviews I've executed. "Getting it wrong" is much likelier to apply to portions of the interview outside the Whiteboarding, like for example the coding portions where you find out if they can code. The whiteboarding is a loosely structured conversation hoping to understand how the candidate would approach the problem. Even not coming up with a perfect answer seems like it'd be far more stressful in the coding part than the whiteboarding part.
Indeed, every single thing you've listed is actually probably directly worse in other portions of the interview. Yet everyone only rails against whiteboarding in this thread, with no one answering my questions like "why is whiteboarding so much more stressful than each other piece" and "why do you think social anxiety would present cleanly as a detriment to your interview performance but never your job performance?
Too many strawmans, not enough substance.
It sounds to me like you view the technical interview as being split into a high-level whiteboarding phase followed by a separate coding test that is not done on the whiteboard.
In every interview I've had that involved coding and a whiteboard, I was expected to do everything on the whiteboard, from drawing diagrams and working through test cases to writing the actual code under evaluation, all in front of the interviewer.
Under this (very common) interview system, there is no "other piece" for whiteboarding to be more or less stressful than - it's embedded into the entire interview! (Yes, there are other interviews throughout the day that might only involve block diagrams or ask behavioral questions with no coding whatsoever, but I don't think that's what most people have in mind when they're told to prepare for an algorithmic whiteboard interview).
I think the interview process would be a lot better if it started with a no-coding-required whiteboard collaboration phase, followed by coding solo, and then ending with a mini code-review on the whiteboard. This is similar to the private technical interview procedure described in the paper.
As to your other question, I'm not sure why you seem so dead-set against the idea that performing analytical tasks while being evaluated in real-time would be harder than doing so under normal conditions. That's what this paper set out to investigate.
I have never experienced such an interview, both when I am the interviewer and the interviewee, and I've been thru the entire process at multiple FAANGs. What you describe - literally coding on a whiteboard - is a far cry from whiteboarding a problem in my mind, this is a fair point
> I'm not sure why you seem so dead-set against the idea that performing analytical tasks while being evaluated in real-time would be harder than doing so under normal conditions
The entire interview has this elevated difficulty. You are constantly being judged and evaluated over the course of the interview. I have consistently asked for people to clarify why the Whiteboard part is elevated over any other technical portion in how stressful it is and how much that impacts performance, and I mostly have been attacked with strawmen. I appreciate your good-faith argument here, and certainly agree with you that conducting an entire interview including coding with a pen instead of a keyboard is a braindead practice. I will be more explicit of my assumptions next time as I would not have anticipated entire interviews to be conducted on whiteboards when you are trying to also find people who can code. I would expect what I've seen, which is that you use multiple different evaluation techniques across different scenarios and levels of interactivity and then you evaluate the tradespace, in which the whiteboard & collaborative time gives valuable data on how personality interop would work with your team.
I've been through the all-day on-site interview at two FAANGs and every coding interview involved me standing at the whiteboard for the entire interview. As the saying goes, data is not the plural form of anecdote, but it's certainly common enough as a format that it has to be considered when discussing whiteboard interviews.
> The entire interview has this elevated difficulty. You are constantly being judged and evaluated over the course of the interview. I have consistently asked for people to clarify why the Whiteboard part is elevated over any other technical portion in how stressful it is and how much that impacts performance, and I mostly have been attacked with strawmen.
I agree that the entire interview has elevated difficulty, but which aspects are intrinsic / essential and which parts could we remove with minimal impact to the effectiveness of the interview?
For example, we both seem to be in agreement that writing code with a pen makes the whole interview harder without facilitating what we usually want to measure - the candidate's ability to reason about complex problems and write good code to solve those problems.
From the findings in the paper, just having a proctor in the room is correlated with the candidates performing worse on the programming task (as measured by writing code that passes the test cases). I'm sure there are benefits to being able to observe the candidate directly, but those benefits should be weighed against the drawbacks and evaluated in light of whether or not it's really a net positive, regardless of whether the interview process as a whole will still be stressful in other ways.
Only earlier today I told one of the new starters: can you please not stand behind me while I work. They asked "why?", my response: "because it fucking irritates me".
My work isn't a stage show. If we're actively involved in a skill sharing session together, or conveying meaningful information, that's different entirely.
But I think it's perfectly normal to not want to be the centre of anyone's attention while your doing knowledge / technical work.
You are indeed involved in a skill-sharing - well, demonstrating - session where you are meant to convey meaningful information about your abilities as a colleague to the audience. Your analogy has therefore directly fallen apart.
Also, I am not really sure what you are getting at with this language, but in general, if I was to have an interviewee who was to state the interview "fucking irritates" them, it would most certainly not be a successful interview.
Taking a good faith chomp at what you've said, representing an interview as "performance art" when it is probably trying to suss out your skills again seems like the kind of lack-of-social-skills that I would love to suss out in an interview. It's not like you are going to be asked to understand and implement a distributed algorithm for the first time in your life, they will take topics that should be at-worst adjacent to your knowledge base and ask you about them.
In an interview, I expect the interviewee, as someone claiming they have the capability to be my colleague, to then demonstrate that we can discuss, reason about, and explore in real-time solutions to problems. If you consider the mere act of collaborating with a colleague "performance art", it most certainly is a bad sign for how well your knowledge base will filter to the team thru any means except them reading your code.
If you were the manager at an architectural firm, would you have a candidate come in and invite them to design a bridge on a whiteboard? Interestingly, at the age of about 8, I could do this by remembering what the structure looked like; simple memorization. At 12, I competed in a bridge building competition, using balsa wood; I could draw a bridge that used good basic design principles. At 16, I could spend hours drawing a realistic looking design, annotating lengths and angles, design features. I know NOTHING about bridge building. Never have, beyond the basic low hanging fruit that anyone who's taken a basic physics class will be able to derive.
There simply isn't time during an interview to demonstrate a full professional design pipeline in ANY meaningful way. It's simply bullshit. This is why handing out problems to people as they arrive at interviews like some fucked up high school pop quiz makes you and your company look like a fucked up high school, full of pedantic pedagogy.
> Also, I am not really sure what you are getting at with this language, but in general, if I was to have an interviewee who was to state the interview "fucking irritates" them, it would most certainly not be a successful interview.
What an odd response. I would ask them what about the interview "fucking irritated" them, because there might be something terribly stupid about our interview process, and bettering the interview process is extraordinarily important to a business.
> Taking a good faith chomp at what you've said, representing an interview as "performance art" when it is probably trying to suss out your skills again seems like the kind of lack-of-social-skills that I would love to suss out in an interview.
Social skills are a skillset highly valued for a social role at a company, like HR, Sales, Business Development, etc. Engineering skills are highly valued for engineering roles, such as Lead Engineer, Architect, R&D Lead, etc. You don't interview your sales people and hand them a paper asking them to derive the quadratic formula from first principles; that would be idiotic. Some might be able to do it, but then why are you even asking them to do BS that is unimportant for a newly hired sales person.
> to then demonstrate that we can discuss, reason about, and explore in real-time solutions to problems.
This is simply not how most fields in science and engineering work. There are times when it's appropriate and needed to discuss, plan, and share information, but that is a VERY small slice of the pie. Forcing people to discuss, realtime, how they are going about solving a problem of any complexity, speaks to an awful culture saturated with bureaucracy and groupthink. A sit-down chat or planning meeting on occasion is really important, but should gazing play-by-play? Disgusting.
Let's say I were.
> I know NOTHING about bridge building. Never have, beyond the basic low hanging fruit that anyone who's taken a basic physics class will be able to derive.
These would be exactly the kind of things the SOCIAL aspect of that whiteboard interview would dig into. Indeed, same with software engineering. I am sure that many people would be able to solve the whiteboard problem I give; that's quite directly not the point. Instead, the point is to see if they can demonstrate their ability to say things like:
* why did you use these lengths and angles? If I were to make this modification, would it have structural or design implications?
* I like the structure overall, but you have "<N>" of some substructure. Why did you pick that quantity? What changes would you make if the client requested more or less to keep a similar feel?
The person would then be able to demonstrate that they don't know NOTHING about bridge building and simply recreated a thing they luckily knew the shape of, but instead dig into first principles, emergent properties, tradespaces that emerged from their design decisions, and why they navigated said tradespace how they did.
> What an odd response
Right, I'm talking about how social skills are an important part of being in a workplace - you do, after all, interact with your coworkers up to 40h a week - and then this fellow tells a story about how he as a senior engineer told a new hire they were "fucking irritating" him as a demonstration of his superior social skills, and now you're trying to pull this? Unbelievable.
> It's simply bullshit
> makes you and your company look like a fucked up hiigh school, full of pedantic pedgagogy
> You don't have sales people do quadratic formulas
> Disgusting
You're quite unpleasant, and arguing directly in bad faith
> Engineering skills are highly valued for engineering roles
And the ability to communicate is a critical Engineering skill. How on earth everyone in this thread thinks asking an Engineer to talk for 45 minutes about a problem is analogous to asking a sales crew to derive a quadratic formula would've been beyond me, but I've seen about 100 strawmans in this thread.
Design work is not a VERY small slice of the pie unless you're entrenched in some org with humungous momentum and years of technical planning accomplished for you. In that case, you can probably afford to hire anti-social fucks who will just sit in their cave and code by themselves, never collaborating, failing to share critical business knowledge because talking to someone else isn't in their job description.
In my engineers? No. In my sales people? Yes.
I want my newly hired engineers (including of the software variety) to be focused on engineering, not doing half-assed "problem solving" presentations without any preparation or analysis. That's not how problem solving works, and I want my engineers to take their time, model problems, explore different approaches, and find solutions that are well explored, documented, and proved.
Conversely, being able to think/bullshit your way through live presentations without being given a chance to prepare anything is EXACTLY what I want in sales people; that skill is part of the skillset of being charismatic, and that is the skillset of people who do sales.
I wouldn't turn down an engineer who turns out to be good with charisma, and the ability to come up with decent answers on the spot is something I expect senior engineers to grow into after a few years of familiarity with our problem spaces. But I certainly wouldn't select for it, much as I wouldn't select a car mechanic based on how nice their haircut looks.
Thank you for this framing, really help me clarify my thoughts around this issue.
> I want my engineers to take their time, model problems, explore different approaches, and find solutions
So, you represent "whiteboard interviews" in bad faith as 'half-assed faux problem solving', and then you state all the things that I personally believe a whiteboard interview shows about an engineer's approach.
Listen, I am sure some roles don't require someone to ever problem solve, and that's fine. But if you think a white-board interview where you get to see someone approach a new problem, vocalize their common approaches, speak about the trade-offs of their approaches, and discuss how they'd implement it is closer to "half-assed" than "modeling problems and exploring different approaches", that's a you problem and not a me problem.
Your haircut strawman is par for the course in this thread.
I would say that being able to understand other people's perspective is more important than comfort in high stress situations with strangers. So if someone didn't have anxiety himself, I would expect him to be able to understand how it affects others when it is explained to him. If he refuses to listen and insists it just lack of social skills on the part of those with anxiety, I would conclude he lacks the social skills necessary to work with a diverse group of people.
I expect anxiety to impact some people, and I expect stupidity to impact a whole lot more. Should I drop the technical interviews, because the stupid people find them extremely stressful? Should I take the assertion of every person with social anxiety that in a job interview they can't perform, but in every other situation a job presents them with they'll be just fine?
I'm sorry, but candidates being able to stand up, think thru a problem, discuss the merits of their solution, and explore the tradespace of other solutions is not a skill just for "sales" people. It's basic collaboration skills.
PS: I used to have heavy social anxiety and impostor syndrome. Those things were terribly unfun, but I openly recognize I frequently caused awkward situations, often failed to speak up when I should've, and was less open to sharing of ideas and code for fear of emotional harm. I spent extensive time to work on my social skills & technical ability, and now I don't feel those things anymore. I am also much more socially capable, open about sharing, willing to speak up in any given situation, and obviously technically stronger from studying.
Please fuck off with speaking down to me. "live coding is very different from the typical day to day interactions". So is the whole fucking interview, which is why I'm waiting for someone to explain to me how whiteboarding is somehow so much more unfairly stressful than all the other parts.
No we should recognize that some parts of the interview give us false signals. We should improve the process to get better signals.
> Should I take the assertion of every person with social anxiety that in a job interview they can't perform, but in every other situation a job presents them with they'll be just fine?
No. Give me the problem, leave the room. Come back in 45 minutes and let me tell you about how I solved it. Don't make me solve it with you staring over me.
> Please fuck off with speaking down to me.
My first instinct was to say something rude in return. Instead I will ask why you think I am speaking down to you? I didn't downvote you. I did say I don't think you are good at understanding other people's perspectives.
>PS...
I am glad for you. That is awesome. I have worked on my anxiety for a long time. I used to get panic attacks when I had to say "here" during attendance roll call. I am much better than that now. But I still have anxiety and will likely always have it.
>I'm waiting for someone to explain to me how whiteboarding is somehow so much more unfairly stressful than all the other parts.
I can't tell you why it is more stressful, only that it is. So give me the problem, let me solve it in peace, and then let's discuss. You will get a stronger signal that way.
PS: I had very good SAT scores. That is about as an objective technical skill test as you can get. I also tutored teens in Math and Comp Sci. I have worked in teams with fellow engineers, designers, managers, and non technical business people. I know how to communicate with a wide variety of different people. But make me do a whiteboard interview and I will get a panic attack about a third of the time. You will then conclude I am an idiot who can't communicate.
Those give me a world of anxiety. I know I can practice leetcode for 3 months full time and ace them, but that feels so wrong in spirit. Also, that's a tall ask for someone with a fulltime job. I will fully admit that I have never tried to remember syntax exactly for any language I use because coding speed was never a bottleneck for my job. Thus, I can almost never finish the darned problems in 45 minutes. My palms get sweaty and my head blanks out.
Weirdly enough. I can literally write the exact code on a whiteboard no-problem. The brain is weird.
I guarantee you will find people who do the actual work. Companies don’t need massive amounts of coders that can minmax an arbitrary book algorithm. If you want someone to be “clever” in exactly that way then you can push them in that direction while solving a closer to real looking problem.
I had a miserable interview experience once were I gave a tree processing solution in a functional, tail recursive way. They spent the interview sliding me back into a mutable, iterative approach, and it really through me off. Additionally, I don't think they even understood the functional solution. Half the time, it felt like I was needing to guess where they'd like me to go.
And we all know working and software development is nothing like that. At no point in that particular interview was I asked to describe projects I had worked on. When I left, they had no idea what I had even worked on previously because they didn't ask. They just asked a tree question, which someone could look up over an evening or weekend. So pointless.
How would you evaluate a candidate's problem solving skills? "Trust me, I can do it, I just can't show you" doesn't seem particularly compelling if I'm looking to hire someone.
Some of what you are saying is accurate, but it's dwarfed by the noise that the different kinds of brains that you interview are producing. One of the most fundamental properties of a brain is it's sensitivity to stimulation. This is how 'activated' it gets by sensory stimulation. A loud annoying sound for one person is barely noticed by another.
People who have reactive nervous systems are introverted, and more easily stressed. It's not a symptom of their upbringing (necessarily, although it can be due to early life trauma, early loss of the mother etc), it's not necessarily about how they think about the ease or difficulty of the interview process - it can merely be from the large amount of information they are perceiving about the emotional and cognitive state of the interviewer.
This sensitivity of the nervous system may not be what you are looking for and that is the nature of the world. But do not be surprised when those you exclude because of a fundamental aspect of their nature react to the exclusion negatively.
Sensitive people have a lot to give. Sensitivity correlates well with openness and openness brings in fresh ideas and new approaches. Openness also correlates with IQ.
The reasoning is that - aside from the studies that you can look up and read - that the sensitive nervous system is more alert to its environment and therefore more creative - as creativity seems to be, in part, the ability to combine environmental perceptions fluidly. In order to be creative you have to perceive a lot and let it sit. This kind of personality also tends to process events more deeply, thoughts spread further and wider. This is bad for straightforward logic, because it creates background noise, but it's great for getting the big picture.
High sensitivity is quite prevalent within the gifted population. You also find a significantly greater proportion of gifted people afflicted with anxiety, which is not surprisingly connected with reactivity - sensitivity. There is a theory of giftedness which describes the phenomena entirely in terms of excitabilities. By excluding the easily excitable from your organisation you are literally excluding the gifted. I am - if the raven matrices you can find inline are to be believed at the bottom end of gifted. And yet I can appear to be an idiot if run through the standard white board process. I deal with it using pregabalin which deadens the nervous system enough to remain functional during an interview. But I do contract work these days and dont have to jump through hoops anymore.
So by only including individuals like yourself you are excluding sources of creativity, you are limiting yourself to a particular kind of person and by extension limiting the kinds of thoughts and approaches your organisation can have.
It's similar and indeed perhaps intertwined with the exclusion that minorities of all kinds find within the workplace dominated by the kind of culture that sees whiteboard interviews as a good thing.
On the other hand, the questions where I can talk through a problem are by far my favorite. Every problem in our field can be talked about for days. There's just so much to discuss:
- different approaches
- performance considerations
- perceived complexity
- and a hundred tangents I could go on to
In my opinion, it is the interviewer's job to get the candidate to talk. If the candidate can't speak a word out of their mouth, the interviewer failed. When I interview I just want to get you to talk...tell me stories, tell me tangents, tell me the stupid stuff, tell me the smart stuff. Tell me what kind of solutions you prefer and why?
Reading your comment made me think that maybe this isn't the best approach. Maybe some people want me to shut up and just do the hard math problem on their own.
Maybe interviewers could offer maybe say five methods or key topics and ask the candidate to choose at least two or three.
Conversely, how they treat me as I go through the process is a great way to judge the engineering culture of the org. If the interviewer feels license to cut me off, jump in, correct me on syntax as I'm still writing, or just generally sprays frustration, it's a high-confidence signal that I'll get the same treatment as a new coworker, and I have ended interviews accordingly.
But rarely does someone get posed a difficult technical design question on the job and on the spot where they've had little or no opportunity to think about it ahead of time. Usually there is a ton of contextual knowledge leading up to those group discussions vs. being asked a question in an interview and then be expected to perform at that same level.
And even for the non-duress situation, there are plenty of situations during normal work where a senior engineer will have to jump in and contribute to a problem they've never seen before.
I prefer myself and my coworkers to pause and think for a few minutes if needed and come up with a coherent idea.
Gibberish doesn’t really trigger any insight to me, if not distracting. Well reasoned thoughts and coherent sentences do.
I am very mindful of this when I interview and I think it’s my job as the interviewer to strain myself to try to understand the interviewee, not the other way around. I quietly try to understand my interviewee and don’t interrupt unless there’s a real reason since I want her to think freely and not feel stressed.
1. First interview, the candidate interviews us. What market we serve, what our development processes are, what our technology stack is. If they express interest by being prepared and asking good questions, we send them
2. a programming task. Choose 1 of 5 tasks. The tasks are not abstract problems or puzzles, but come out of the designs we've implemented. We ask for 100 lines of code and not more than 2 hours. They take as much calendar time in days as they need, most have full time jobs, and let us know when you're done. The candidate then comes back, typically in 2 to 7 days and hosts a code review in front of our team. I give strict instructions before hand: The purpose of the review is to learn how they thought through the problem and how they solved it, not to criticize their style and approach.
3. If we decide we like them to this point, the third interview is with managers from other functions. How well does the candidate communicate, come across to non-developers, express interest in the company and role, etc. It's a check point to look for concerning weaknesses, as well as get buy in from the broader organization.
We like this approach because it allows the candidate to code in a much more natural environment. They can take the time they need and comfortably solve. No one spends their professional career coding on a white board. This approach so far has yielded excellent results. We ask developers how much time they took in the programming task and why they chose the one they solved. The programming task is telling, not in the quality of their code and how long they took, but how much did they get into it? We have had the range from candidates who did not complete it at all and opted out, to candidates who stumped 40 year veterans with elegant code. In every case, we learn how well they can express their thoughts, and importantly, their level of love for the discipline. This is as important to me as any other attribute.
So far, our 5 minute test has failed spectacularly. Knowing what a good answer is requires too much interpretation of the subtleties of the answer, so it's not very useful for the managers who may not be the greatest at coding.
I think we'll tweak the question a bit, but we're kinda stuck.
How do others handle pre-screening?
We give candidates all the time they need, as in a week or more. We explain the process when we ask them to interview, and if they prefer a white board interview instead of a take home, we can do that. So far, no one has taken that option.
You're right that some people object to being asked to do programming (though none that we've asked to come in have objected) and some people think they should be compensated, which we don't do. I point out that the code review, for example, takes my team of 4-5 engineers 1-2 hours of their time, so we are matching the candidate's time spent with my team's time.
In any case, it's certainly not for everyone, and we're flexible.
The standard whiteboard hazing trivia is bad enough, and 4+ hour long marathon on-sites that require weeks of study and taking vacation days just to interview are ridiculous. But your method is not much better. You’re still perpetuating this myth that somehow, unlike virtually every other profession on the planet, software engineering requires intense examination of on-the-spot demonstrated skill, rather than simple conversation probing past experience and domain knowledge.
We just do interviewing completely ass backwards in software. Part to assert gatekeeper superiority, part to judge fealty. It’s just bananas.
BTW, I will consider the approach of simple conversation probing past experience and domain knowledge. I infer that would be in lieu of a coding demonstration. If that has worked for you, any detail that you can fill in would be appreciated.
> The trier social stress test involves having a subject prepare and then deliver an interview-style presentation and perform mental arithmetic, all in front of an audience.
> … rather than avoiding unwanted stress, technical interviews may be inadvertently designed with the sole purpose of inducing it.
We've at least abolished the brain teaser part in most cases.
To give just one example, I don't think I'd have ever solved the "find the largest rectangle" in a bitmap. Certainly not in a 45 minute interview. The brute force solution is O(N^3 * M^3). Me, I couldn't think of any spatial structures that would make the problem all that much smaller. For any structural solution it seems to be the degenerate case still ends up using too much space and too much time. But, there's an optimal "magic" solution is O(N*M). If you happen to know it or have seen then then you bang it out. If you haven't it's arguably just luck that you'll have the insight during your interview.
I've seen several others of the same vane. Given 4-24 hours and a debugger to help my thoughts and directly my mistaking assumptions I might solve some of them but in 45 minutes on a whiteboard, it's just a lucky role of the dice that I've seen something similar before.
The Jan 2021 google foobar was similar. I didn't figure it out nor did a friend of mine. We both spent 3-4 days. I bought it up with another friend. His first solution was wrong. When I pointed out why he was like "Oh yea, this the XYZ-algo" (meaning he didn't solve it, he just had read the solution before). The wikipedia page for the arguably obscure algo basically showed the exact same example as the foobar one.
As someone with clinically diagnosed and treated anxiety the whiteboard interview is an absolute personal hell. I have extremely good performance ratings everywhere I’ve landed a job even in very high stakes actual work environments, but my opinion on the average interview process is it’s extremely discriminatory against people who deal with practically any mental health issue (which, let’s face it is probably quite prevalent in this industry).
Anyway, other than that interview where I was super stressed and tanked it, all my others have been nothing like it. Just goes to show how stress and a loss of confidence can wreck your chances.
After berating one of my responses, I told him this wasn’t going to work out, that this was easily the most off putting interview I’d ever experienced, but thanks for the consideration.
The dude was stunned, apologized profusely, and we somehow salvaged the last 20 minutes which we spent nerding out on tech and hobbies.
I ended up taking my name out of consideration, but it was a cool experience realizing I can (respectfully but assertively) tell people to fuck off if I need to.
I have friends who, before upcoming situations they know will be stressful - interviews for engineers, performances for musicians, surgery for doctors - have obtained prescriptions for beta-blockers, which mitigate the physical symptoms of the fight-or-flight response for a few hours at a time.
I appreciated the following tidbit, which I didn't know (since I figured prep for this particular test was now widespread there, by convention, but not officially stated):
> students at Stanford take courses for passing technical interviews, CS9: Problem-Solving for the CS Technical Interview.
And because of things like that, the current interview convention seem even less useful as a predictor, for purposes other than "Did you go to Stanford, or are of similar socioeconomic status, to have been coached on beating the current metric, specifically?"
My current role’s take home was suppose to take “a few hours”, took the entire weekend. But hey, it worked out.
When I have done take homes, I've tried to keep the time requirements minimal and the project itself simple and hopefully even a bit fun. It might even be as simple as "here's a REST endpoint, pick the JS SPA framework of your choice and do something interesting with that endpoint." Even just a simple form with clean code that builds will be enough to get you hired if you're otherwise a good candidate. Because in the real world, most of of us are just building CRUD apps.
All that said, I only bother with this if the candidate is incredibly green. For anyone with decent experience I generally assume they're not lying on their resume, and an hour long chat about their projects and technology interests will tell you more than any technical assessment.
It just seems so crazy that we pressure technical candidates the way we do, and expect decent results.
"Ok cool. Before we get started, let me understand some non functional requirements. How many concurrent users do we expect? Is brief downtime acceptable?"
Then you listen to the answer.
Then you say "ok cool. So given what we just estsblished, it's making me lean towards X type of architecture. Here are trade-offs A,B, and C that are involved. Does this make sense to you so far?"
You listen to their answer. They will 100% of the time indicate to you if you are on the right track or missing something.
Then you go "ok cool. So on the high level we are going to have (for example) a client, a server, a proxy, a database, etc . Given what we discussed, I think the most complex part of the system will be X so let me begin by thinking through that and then we can pop back up to other parts. Does that sound good to you?"
Etc.
I am not looking to make you feel bad but being able to have this conversation in this way is a baseline skill of a senior engineer. Compare that to what you actually did and ask yourself if it's you or the "bitch" that has room to improve here.
Judging by the way you talk, I seriously doubt that.
I've had interview instances where my face felt flushed and my mind completely blank on a question I knew how to do but struggled on. And then 30 minutes after the interview finished, the answer is clear as day in my head because it was indeed something I knew and the anxiety was gone.
I feel there is a fair amount of discrimination against those of us who experience anxiety like this, but in a way, I don't care all that much because places that interview certain ways aren't places I want to work anyway.
So I guess water is not wet.
I will say that I am comfortable with people not wanting to work with me, because I don't fit their "cultural criteria."
Take-home tests are a good idea, but you still have the issue that so many folks here have mentioned:
They don't just want you to solve the problem. They want you to solve it the way they want. Orthogonal approaches are generally not met with support.
I once had a take-home, where I was asked to produce an iOS app that used a certain dependency (that I had never used), and was asked to do so, using MVVM.
In four hours, I had it done. It was a localized, ship-ready app. I didn't use MVVM, because UIKit is designed specifically for MVC, and using other models actually makes the app larger and more prone to problems. It was a tiny app, and I didn't want to add any code that wasn't 100% necessary (basic Quality 101). I also encapsulated the dependency (Quality 101, again).
Did I mention that I had never even looked at the dependency API, before starting that project?
During the process, my home had an electrical emergency, and almost burned down, so I guess there was a bit of stress.
I could smell the burning rubber, as they peeled out (although it could have been the electrical fire). They never asked me why I made the choices I did. I strongly suspect that I was never in consideration anyway. I think they were checking off a "interviewed old guy, and he sucked" checkbox on an EEOC form.
I handle stress quite well, indeed. A quick glance at my CV tells you that I worked for one of the top Japanese firms in the world, for 27 years.
They refine and mass-produce stress in Tokyo. If you last 27 years at a joint like that, you can handle stress.
The folks that actually get around to working with me, seem to enjoy the experience. Funny how that happens.
Does Stress Impact Technical Interview Performance? - https://news.ycombinator.com/item?id=23844367 - July 2020 (26 comments)
I remember leaving as quickly as I could and thinking that I had to find a place to sit down as soon as possible. I made it to Washington Square Park and just sat at a bench for about 30 minutes feeling like I was slowly coming out of a tunnel. Never happened before nor since.
Even if they don't make you anxious (that started to subside after the first hundred times or so) they're still not a good time. I've had senior staff running interviews at big companies do things like harass me over data structure choices, insist that I was lying when I described a textbook algorithm, accuse me of industrial espionage, and all other sorts of absurd or unpleasant things. In some cases even after that I still got an offer, because the interviewers weren't out to give someone a hard time... they were just bad at running interviews. One of my all-time worst interviews involved the design director (?) at a billion-dollar game studio calling me (unscheduled!) at 7am and demanding that I come up with 50 unique uses for a paperclip. Really.
The gap between an average or bad interview and a truly good one is enormous, and the number of well-conducted interviews I've had is incredibly small. I can only imagine how much worse it is for someone with additional obstacles, like how is someone in a wheelchair supposed to stand up and write C++ on a whiteboard for 30 minutes? What does the whiteboard interview look like for a blind woman?
[0] https://news.ncsu.edu/2020/07/tech-job-interviews-anxiety/
Recent example: I was asked to do a density map with about ~20 minutes remaining in an interview. I could immediately see the performance issue of the naive solution (re-summing the same positions), but I needed some time to think through an efficient solution. But because of the time limit and the fact that I had someone watching my every step (i.e. there was more performance than problem solving going on), we agreed that I would take the naive approach and simply sum all of the positions as quickly as I could. When I wasn't asked back for a second round, it was obviously in part because I had resorted to such an ugly and inefficient solution.
I don't blame the interviewer—he was actually one of the best I've ever interacted with—and I actually think the problems selected fit the role very well. But at the end of the day I don't feel that format gave a very good snapshot of my abilities. (Case in point, I looked up solutions afterwards and an intuitive approach I had thought of was very close to the optimal solution, but I didn't feel I had time to go down that route.)
I don't think there is a perfect interview format. They all have downsides. But this is definitely a tradeoff you're making with timed algos.
Are you hiring for a fire department? The Navy Seals? Engineering section on a nuclear submarine?
Maybe, just maybe, things would improve if you had a few people who said 'no way' when someone tries to crank up the drama on your team instead of just grinning and bearing it.
Capitalists don't reward the quiet workers. They exploit them. Billionaires and wannabees do it remorselessly.