Tech sector job interviews assess anxiety, not software skills: study

All: please don't miss that there are multiple pages of comments. The top few subthreads have become so large that they fill out this first page entirely. You have to click 'More' at the bottom to see the rest (there are over 700 at this point).

https://news.ycombinator.com/item?id=23848039&p=2

https://news.ycombinator.com/item?id=23848039&p=3

We're working on performance improvements that will hopefully allow us to go back to HN's original style of one big page per thread (not infinite scroll, don't worry). In the meantime please look for those 'More' links when the total number of comments is over 250 or so.

I conducted a couple hundred interviews for my first FAANG employer, and I was constantly amazed at the percentage of candidates with years of Microsoft or Facebook experience on the resumes who apparently did not know how to program. I always thought, 'huh, guess I know why they quit after 3 years, amazing that they all lasted this long."

Then I interviewed for another company and utterly bombed. It became suddenly clear to me that I had been an idiot. Of course nearly all of those candidates were perfectly good programmers. They had had shit interviewing days, probably mostly due to nerves, but they probably would have mostly been perfectly good employees. How frickin' arrogant I had been for concluding that people who couldn't solve incredibly high stakes algorithm riddles on a whiteboard in 45 minutes with enough speed and flair were somehow not qualified to be my coworkers.

People bomb, including good people. That is sadly part of the system.

Once I had this candidate. Damn, I know my questions are deceptively simple on purpose, but he literally couldn't do anything. Not even a trivial brute force. Not even any related simple knowledge questions. Couldn't tell an average from a median. The only good thing I could write in the feedback was "seems to know some basic syntax".

It was quite a learning moment to read that everyone else was praising how this was the most brilliant candidate they've seen in years. Well, mine was the interview just before lunch, in a schedule that was atypically later than usual. I guess starting an interview after most people start their lunch break was asking for it. He also got hired - apparently the committee agreed that "can't think when hungry" is not really that important a flaw.

It seems pretty obvious to me that this kind of interviews are simply there to assess how much you want to work for a certain company. As in: would you be willing to study for months, go through mock interviews, read books, test your skills etc. to have a shot at working for us? Even though the majority of that stuff will likely have zero impact on your day to day work?

That's all there is, really. I've had no problems in the past implementing complex algorithms, if I needed it to get the job done. Today I'd barely be able to tell you their names. But to get to the point of knowing most of them by heart and being able to implement them with no help whatsoever, would take a lot more work and dedication, knowing that you'll probably end up not retaining most of it anyway.

The same system is used in competitive sports and academic tests. They mostly measure how much you really want to be part of that league/team/school, rather than whether you'll do well or not. If there was a perfect correlation between pre and post entrance test performance there wouldn't be PIPs, performance based layoffs etc.

It was made the clearest to me when I interviewed a colleague who had left to another company and two years later was interviewing to return. I knew he knew everything and had the skills for the job because, well, I know him, I’d worked with him before. He’d done this very job previously. Yet he still performed terribly in the interview and (rightly or wrongly) I had to base my evaluation on my knowledge of him rather than his performance in the interview. Interviews suck so much for evaluating people. If you want a job and good career progression the best advice has to be prioritize making friends with lots of people.

Interesting. I usually do pretty well in interviews and I've always attributed that to my four years on the debate team in high school. This tells me it might be even more true than I previously thought. Even when I'm nervous as hell, when you put me in front of a judge (interviewer), I click into "performance mode" where I project confidence even if I know what I'm saying is BS, let alone just a bit unsure. This is how you win parliamentary debates when you've had five minutes to prepare for a topic you know little to nothing about—and I guess it helps in interviews too!

That and debate gave me the ability to just start a sentence without knowing where it will end and then talk my way to a conclusion that sounds like it makes sense (useful when you have 2 minutes to prepare a 3-5 minute speech on a topic you've never seen before). So when I get up in front of a whitebard, I just start talking confidently, and even if by the end haven't solved the problem quickly or efficiently or at all, it sounds like I've been thinking sensibly about it.

My takeaway, I guess—parents, make your kids do debate!

I'm a ex-FANG dev that started a recruiting company, so I am VERY familiar with the arbitrary tech interview process. I'm a huge proponent of contract-to-hire. Even started a company* to make it easier for companies to offer contract-to-hire.

Now before you dismiss the idea because you are "too good" to do C2H hear me out. Everyone knows that actually working with someone on the job is the only way to get an accurate evaluation of an engineer. Everything we currently do in technical interviews (whiteboard coding, pair programming, take home assignments, etc) are just attempts to recreate a working environment. If you are hiring a contractor first, you can do a light-weight interview process and then just bring them on and start working together.

As the dev being brought on you can prove your value before you negotiate your salary. You can also try out the company and manager before committing to an offer.

It's crazy, I know, but I think it would drive dev salaries up if it weren't so hard to change jobs because of the tech interview gauntlet we've created for ourselves.

[*] https://www.facet.net

Huh, the only successful interviews I have are the ones where I'm burned out after a long, fruitless, frustrating job search. And so I walk in assuming the interview will be a waste of time and just not caring anymore.

That's when I get an offer.

Here's the study: http://chrisparnin.me/pdf/stress_FSE_20.pdf

They asked students to find the "Longest Substring Without Repeating Characters".

For the formats: "We created two interview settings: public setting and private setting (see Figure 1). In the public setting, wearing an eye-tracker, participants solved the technical interview problem in the presence of an experimenter. The participant was instructed to talk-aloud while solving the task, which is a standard practice in technical interviews [28, 47]. The lab space contained a whiteboard at the front of the room. An experimenter was situated near the participant for the entire session without interrupting participants’ thought process. If a participant asked for clarifica- tion, the response was brief. For example, if a participant asked, “Does this look correct”, the experimenter was instructed to reply, “Complete to the best of your ability”.

In the private setting, participants solved the technical interview problem in isolation. Participant were provided a private room with a whiteboard. Participants were informed that the experimenter will step out the room and will not return while they are solving the problem. Thus, they had to make sure that there is no questions left for them. When the participant was ready to begin the task, they wear the eye-trackers, the door was closed, and the participant worked in privacy, until the task was completed within allotted time. After the experiment, we clarified with participants if they had any uncertainty about the problem they just solved."

They ran a randomized controlled trial on the effect of having another person in the room, which is interesting. It does not, however, show that these interviews fail to assess software skills.

This is also essentially the worst case for having another person in the room: they are completely superfluous, just watching you, not helping, not communicating, not collaborating. Not even saying things that might make you more at ease.

The best technical interview I've had:

- Take home: 2 days to read a newly released Deep Learning paper.

- On site: explain the paper to the hiring manager.

- Evaluation: (1) Did the candidate understand it? (2) Did the candidate go farther, and look into the cited papers and understand that? (3) Can they explain Deep Learning concepts and talk their way through related problems?

In 5 months of searching for a job, this felt like the most relaxed interview, because it was really a conversation between people in the field. It was also the most demanding and interesting job I encountered in these five months. I got the job, but this happened especially because the manager actually did the work of respecting my experience and talking to me, instead of going for the traditional and "safe" BS.

An opposite experience:

Technical Interviewer asks me whiteboard SQL question. I answer it, only to be corrected on a minor point. Turns out, when I tried it out on my own, I was actually correct. This person was to be my manager.

I did get an offer from them, but this and other red flags made me decline.

Takeaway for managers who actually care about hiring well:

- Don't force your candidate through one off tasks that don't represent their day to day. Is your Data Scientist really just running SQL queries and optimizing computational complexity of their algorithms? Isn't the main point that they can think critically?

- Hasn't their previous experience shown they can code? Their Github? Sure, give them a (quick) take home project if you really want to make sure. In the rare case they faked their way through the whole thing and you really couldn't tell after talking to them for a few hours, you will surely notice in the first few weeks.

The interesting thing is that Leetcode style problems encourage your code to go into the opposite direction that most production code should go. The game of Leetcode is to write the most consise, clever function that solves the problem in a tricky way. Thats exactly the kind of code I would flag in a review - sure it might be super efficient and elegant, but good luck maintaining it a few months later. I would way rather go for a solution thats less efficient but easy to maintain - I can always improve it later if it becomes a bottleneck (it usually doesn't).

I know the take home assignment is a divisive subject, but I think it's the perfect way to evaluate a person's coding ability. Did they use dependency injection or did they reference globals straight in the code? Did they break down the code into small testable chunks or write one giant file? Would their code pass code review? These things are way more important for most codebases than shaving an order of magnitude of big 0 complexity.

The only issue with the take homes is that they take a ton of time. Most companies say that it should only take a few hours but also ask for production grade code. Thats impossible - clean code takes time. Companies should at least start compensating for the time taken to do the assignment.

My favorite interview process involved a take home assignment which if you pass takes you directly to the final onsite interview. There the live coding sessions involved building more features onto the take home. As an experienced programmer who sucks at Leetcode I really hope the industry moves in this direction.

It's ironic that people who want the job will be nervous and might interview poorly but people that are only interviewing for experience or just don't care will do better.

This is likely a source of gender disparity in technology. Men tend to be more confident in math and engineering situations, even when they know as much or less than their female peers (the much-reported "gender confidence gap")[1][2][3]. We are disadvantaging equally capable women by using an interview format they are less likely to do well on.

[1] https://onlinelibrary.wiley.com/doi/full/10.1111/j.1949-8594...

[2] https://eric.ed.gov/?id=ED542116

[3] https://www.theatlantic.com/magazine/archive/2014/05/the-con...

I've been interviewing SRE candidates for a long time. In theory, being able to make technical decisions under high pressure is integral to the job.

But interview stress is very different than work stress. It doesn't seem to translate well. Some people do fine during the interview and then crack under real pressure, and some people it's the exact opposite.

I've changed my interviewing to involve very little coding. I might do FizzBuzz on the phone screen just to weed out the folks who really can't code anything on the fly, but that's about it.

After that, most of my interview is me explaining some real works tasks that they would have to perform, and then asking them how they would try finding the solution and if they have ever solved a similar problem in the past, and if so, how. I also ask them about difficult problems they've solved, how they solved it, how they found the problem, etc.

My goal is twofold -- try to figure out how they research problems they don't know the answer to, how they investigate issues; and give them lots of examples of what will be expected so that they can bow themselves out of the process if they don't think it's a good fit for them.

This obviously works much better with people who already have a job, and are more likely to take themselves out of the process, but so far it's worked pretty well.

That experiment assumes that SWE job is exclusively about solving technical problems, so we can just "lock the candidate in a private room and let them solve it".

In reality human to human interactions, sometimes stressful, are part of the job. Customer & PM requirements, technical debates, giving feedback, explaining your technical solutions, etc. are common.

Maybe part of the problem is the education that does not prepare people to reality of the job.

Note: I do realize that SWE interviews suck, and I'm not trying to defend them. Just making comment on the study.

The methodology:

> For this study, researchers conducted technical interviews of 48 computer science undergraduates and graduate students. Half of the study participants were given a conventional technical interview, with an interviewer looking on. The other half of the participants were asked to solve their problem on a whiteboard in a private room. The private interviews did not require study participants to explain their solutions aloud, and had no interviewers looking over their shoulders.

> Researchers measured each study participant’s interview performance by assessing the accuracy and efficiency of each solution.

[...]

> “People who took the traditional interview performed half as well as people that were able to interview in private,”

A company flew me out to San Francisco to interview once. On my way into the building in the morning I was mugged (almost right outside). I told my interviewer, they took me to the lunch room to sit down and compose myself. "Take as long as you like", they said. 2 minutes later they came to collect me for my interview.

I was a wreck. Worst interview of my life. Didn't get the job. I am pretty sure I would have if I wasn't falling apart.

>All of the women who took the public interview failed, while all of the women who took the private interview passed.

Companies that are serious about avoiding gender discrimination in their hiring practices need to carefully consider what they are really looking for when certain candidates seem to have "the right stuff" after a whiteboard interview. We need more research in this area.

The only good way to assess a candidate is to hire them. It's sad, but true. Some of the brightest and best will not even be able to code with someone else even paying the remotest attention to them. Interviews used to take account of this 15 years ago. Google came along and destroyed this.

I remember talking to a Google interviewer and I came up with a better solution than they expected. Since my solution was non-standard for online median finding-they wanted a unbalanced binary tree where I wanted a tree with the values at the leaves and using a fixed height, in this manner you could easily find successor and predecessor and I'd have actually found a better solution than their expected solution. They wanted the standard binary tree solution with predecessor and insertion, where I failed.

Everything will still jovial, until I mentioned that I consider the tools I use and how to operate them as part of my skill-set. Apparently this is false.

This was me interviewing at Google for a potential job using computer graphics or computer vision. I wouldn't even be working with problems anywhere close to this.

I've found that everyone is just copying Google's process because of all of Google's cash and success, but don't realize that most of Google's success is not due to any investment but rather just one solid team doing search well.

Anyways, I'm glad I didn't get hired, I have a better job (pay and work). Both of the projects I would have worked on have already failed or nearing failure already, it's been like 3 months.

As a hiring manager the test that I believe works best is a code reading exercise. This helps reduce anxiety or the effects of anxiety (fear has a very big negative effect on creative thinking)

I take a small fairly self contained function from the existing code base. Give them 10 minutes to read the function.

You can also add a few actual bugs into the function if you like to see if they are picked up.

I then generally ask them a couple of questions. First to step through what the function is doing. Then what the larger context of the function might be and finally if they have any thoughts on improving the function.

It's a quick test usually under 20 minutes you can even hand it over to non technical interviewers if you describe what kind of answers they should be looking for.

Interviews are a joke and are as useless as randomly selecting people.

I did 14 interviews at Google (two applications, first one after being contacted by them, for second one I contacted them to try again). As far as I could tell, I passed most of them, and for some of them I even provided new, unique solutions that the interviewers were amazed by. That being said, both times I got reject in the last step before the offer without being given any reason why. The only think they told me was that I should learn less from programming books and have more experience actually building things. I have never read a programming book, only learned by doing and had many projects to showcase. Some interviewers were not even paying attention to what I was saying, seemingly working on something else on their laptops. My recruiter was both times so certain that I would get the job (based on the very positive feedback she received from some of the interviewers) that she started telling me about the relocation process.

Today, 4 years later, I am not mad about it as I got to work in a very cool gaming startup for 2 years and now I run own company where I have created a product used by thousands of webmasters. I am honestly just confused about the process, as I know that I am a top-tier programmer and, based on what others said, a nice person to work with. I guess that ultimately it was their loss, but it's just a machine so no one really cares if X gets the job or not.

Multiple discussions about this on HN today, that's good. Maybe it will lead to the pendulum swinging back a bit.

Interviewers need to do their due diligence. Coding tests can have their place when interviewing for certain kinds of positions, especially for new grads and entry level positions. In the end nothing can replace checking references and looking into a person's work experience.

There is no algorithms and data structure whiteboarding test that can differentiate between a junior and a senior developer. Being senior is a product of acquired wisdom, not just skill. And in fact senior devs can easily bomb out a whiteboarding test because we're further from university CS experience.

Bad interviews make for bad hires. Bad hires can cost a company more than not hiring at all.

Whenever I've conducted interviews in the past, I've gone out of my way to help the candidates relax. Need to use Google? Go ahead, I do literally all the time. Need some advice, something explained? Just ask, that's why we have peers. Nobody writes code on a whiteboard, with no docs, with three people watching, silently taking notes.

Interviews should reflect the environment within which the job is actually done, and allow the candidate to demonstrate what they're good at, instead of treating them like a circus animal jumping through hoops, giving them performance anxiety.

For my most recent job, they circumvented my interview entirely and just did a few days of contract work instead. That way the company gets usable work, the candidate gets paid, and it filters out those who can interview really well but may not be a good company fit.

I hire for industrial IT, and being able to handle the anxiety of a high-pressure situation with grace (e.g. mill down, losing tens of thousands of dollars a minute) is an important factor. No one expects miracles, but you need to be able to keep your cool and work the problem, not waste time fighting with your own rising anxiety.

Seeing how they deal with a stressful situation is an integral part of the interview in my opinion, and I imagine it is a feature for any IT role where that is the case. Standard dev roles, or other roles where time-based stresses aren't as prevalent, may be different.

Only yesterday I had a candidate bow out 10 minutes into the whiteboarding exercise, which is probably best for both the individual and the company if that kind of situation is untenable for them.

This largely applies to people that have already produced working software in the real world, but...

Companies should trust the people applying based on their resume and ability to talk about what they have done in the past or how they would approach real world problems (organizational, interpersonal, technical). Maybe have them write some small bit of real world code using libraries and frameworks they are familiar with, while they are alone without anyone watching [key|pen]strokes.

Maybe I'm lucky but after 20 years in the industry shipping products to consumers I've never ran into a Software Developer or "Engineer" in the real world that couldn't get things done in a normal working environment in private industry (now in government, don't get me started). Some are slower, some are faster, some can handle bigger problems, but I have no idea how to reliably filter for that in an interview having tried many things.

The "OMG this senior person can't write FizzBuzz on a whiteboard or while videoconferencing and knowing someone is watching your every keystroke so they must be a horrible programmer" approach is just wrong.

I've only ever failed interviews where I had to solve problems or write code with someone watching (virtually or in person). Let me be alone and give me a product to build, though, and it'll get done. Customers will love it and it will be great. See: resume and products in the wild. The Math-oriented abstract tech interview is missing out on all the practitioners that care about the product and can build reliable maintainable systems in a team environment.

These days I'm primarily management, though, so I get to set the hiring guidelines. Yay!

This is why I stopped writing code. After I bombed a bunch of interviews I threw up my hands in disgust, it wasn't me it was the process.

One thing that stands out to me about the process is that everyone gets the same interview, doesn't matter if you are fresh out of school or have a lifetime of experience. Experience counts for nothing in these interviews. Can you design twitter in 40 minutes, thinking through every possible failure mode and issue along the way, or not? Apparently, I can't or not well enough. I think these companies really want folks with great public speaking and whiteboard skills - I'm kind of an introvert and that is a challenge for me where the day-to-day work is not.

I certainly know my clinical anxiety has caused far too many interviews to crash and burn spectacularly. It'll be like "wow, you're graduate educated, 16 years experience, you must be hot shit", and then I go up and just flounder. For me it's not so much writing on a whiteboard, it's having someone standing next to me, or sitting in a chair looking up at me, occasionally interrupting me. I'll try to think out loud and all of that stuff, but that seems to invite questions that sound judgmental to me. I'll start thinking "they don't believe in me, they don't like me, they think I'm a fraud". Unfortunately, I've had those fears confirmed some times too. I had an interview end with a guy saying, as he handed me off to the next person, "You really are the lead of the ... department?"

I experienced this personally as I bombed a whiteboard interview because I'd been studying scala to work with them and programming a side project in C, and literally blanked on simple ruby syntax and idioms when I started to panic. When we started our own consultancy made 2 simple rules:

* Give people a take-home test that mimics the real work we do.

* Pay them to do it.

It's amazing how many candidates we find that are great at the actual work we need to do, and happy to do it, and how many people it weeds out who wouldn't be happy doing our normal work.

My 2c. This is purely anecdotal, and based only on hiring about a dozen developers.

The best way to hire good developers is to take them out for coffee/lunch/beer and talk shop for an hour or two. That's it. Make it more of a discussion than an interview. Ask them about the their experience, what have they liked working on, what have they hated, but also tell them about what you've been working on and see what feedback/ideas/questions they have about it. I find fielding their questions and comments just as valuable as asking them my own. Joke about things that have gone spectacularly wrong, trade war stories, that kind of thing.

I think it's a great way to assess whether a dev "knows their stuff" or not. A couple of other notes: - If you're not a dev yourself, get someone who is and you trust to do it. Try and keep it "dev only" if possible, remember this is supposed to assess their dev ability, but is also a good indicator of communication and cultural fit within the dev team. - Try to do this one-on-one or at most 2-1, any more than that and it becomes a panel interview. - If someone is regurgitating theoretical or blog-post-scanned information without much understanding, it becomes more apparent the longer you chat with them. So an hour or two is a good length of time.

We usually also do a quick take home coding test too, but have found the chat/discussion to be better indicator of long term success. As many have already pointed out, I also find the coding tests (whether take home or in person) don't really assess "real world" ability and can often be gamed.

I can relate to this. There are two interviews that gave me really bad anxiety, but I think it's because I went in with anxiety as both required moving across the country in different directions. The first I was accepted, but I think only because the hiring manager worked with me before. I felt short of breath and like I wasn't listening to half the questions. The other was Facebook, which was probably the hardest interview I'd ever done. One particular question involved a lot of keeping things in your head and math (network/node design), and I just froze and got frustrated and in the end gave a crap answer with no thought. Given a half hour without someone staring at me waiting on an answer, I probably could have came up with something decent, I figure.

I'm not particularly upset at either case, I'm happily employed. But there was a nagging feeling in the back of my head that I was disappointed that I didn't get to convey 'the true me' to either company, whatever that means.

I went through the interview gauntlet last year. When my extended family would ask what it is like I would say:

Imagine you want to hire a comedy writer for a TV show. However, you conduct your interview by having your candidate perform stand-up improv comedy like "Who's Line is it Anyway". While you want to hire someone funny, you don't need to hire a performer.

In my experience, programming interviews are basically an improv programming performance. I didn't pass an interview until I had enough practice performing while others watched.

>. For example, in our study, all of the women who took the public interview failed, while all of the women who took the private interview passed

Talk about buying the lede. If there's any justification for abolishing this practice, that's it right there.

Having performed hundreds of tech interviews, I realized that my developed super-skill was to keep the candidates relaxed regardless of how well or bad they did.

That meant carefully controlling my emotions when their answers were bad, and having a smooth transition when the answer didn't come at all - a situation that typically causes tension. You have to essentially hand out the answer or transform the question into a trivial one so that the candidate can have a small win and relax. If you're not doing that, chances are you're screwing up the whole process by throwing the candidate off-rails after a simple non/bad answer.

That's why the interviewer has more control over the outcome (besides the scoring) as his / her bias will heavily reflect on the dynamic.

This is why internships are so important. Sometimes it’s still hard to tell after 10 weeks. It’s very naive to think we will know after a 30 minute interview.

One of the best hires I had made had a car crash interview. They could hardly speak they were so nervous. I could see from their CV that they had some good personal projects and thought lets give them a chance, that's what prohbation periods are for. On the job they were amazing. It was great to see them grown in confidence and they were a key figure at the company after a couple of years.

I want to push back on this a bit.

Writing working code well and fast is important.

But so is communicating with your coworkers!

Being able to talk about your code is not a weird thing you only do in interviews. It's an essential part of your everyday work, and the programmers who don't do this well are in a sense underperforming.

So just removing the talking part, as this study did, removes an important part of the interviewing. When I interview people, I value their ability to talk about the problem and how they think about it more than coming up with great code quickly. I need people who can talk to me, not just the compiler.

That said, talking through your whiteboard programming task might not be the ideal way to measure this skill either.

Our industry has created this mess thanks to that stupid crackin “book” and process designed by someone who never actually worked as a software engineer or released and maintained an actual product.

When I “cracked” my Google interview and got the job, I joined a team I never met. Of course we all could code , since we “cracked it”, but did I enjoy the team dynamics or team culture? Nice people but if they were a 10 person startup I wouldn’t choose them or would they have been successful.

None of the social dynamics which are important to teams was there or even fostered, acknowledged.

You just did a transfer and kept fishing.

Why, b/c Google just believe coding skills are all that mattered.

Yet they matter, but so do other things.

One of the best interviews that I ever had was a phone interview where I just so happened to have a good session of meditation beforehand. I never realized how much how well I could do in an interview until I did one while feeling relaxed.

When I interview a candidate, I'm explicitly and intentionally looking at how they respond to stress, how (if) they're able to say "I don't know", and how they communicate complicated ideas. I open with soft-balls to put them at ease and get them opened up and talking. Then dive deeper into technical things. All the questions are technical, but the main answer I get is how they communicate, how they relate to people, and how they deal with the unknown.

If you understand that a technical interview assesses anxiety, and you roll that into the plan, if you know what the game is, then you're 1/2 way to a win-win already.

Interviewing is a skill. I once had a good job where I was happy and didn't move for 10 years. Big mistake - after that I bombed at interviews. After doing 10 interviews I'm now up to speed. I kinda feel like I need to do some interviews every year just to remind me how to behave and how to keep market ready.

Traditional tech interviews (live coding challenges, algorithms, brain teasers, etc) have so little value and cause so much suffering in candidates, they are borderline sadistic IMO and in the end you are just as likely to make a bad hire, then not be organizationally prepared to deal with that so you end up with a broken team and bad outcomes. Why?

There are still challenges with contract to hire but overall I've had great success with it. First of all, we get a realistic deep look at candidate capabilities before we commit and they get a look at what it's like to work with us. Then we're very prepared to part ways because that is the entire point of the contract to hire. If feels 10x more successful in hiring great people than any other approach I've personally seen and, very critically, it is more humane and respectful.

There is a hypothetical concern that highly pursued candidates won't wait around for that but we've found exactly the opposite is true. The best developers are happy to see we have a more sound hiring process and are psyched because we have a credible case that our teams are really high quality and free from the random bad apple that spoils the work experience at most other companies. Great devs aren't worried about being hired today or in two weeks, they know they can get a job anytime they want. What makes great devs great is usually their passion for great craft, process, culture - if we can credibly promise that, they overwhelmingly are happy to do contract to hire.

I failed the third phase of the Toptal interview process (where you have to solve two coding challenges, each within 20 minutes, while screen-sharing with a human observer). The event was nerve wracking.

On the first challenge, I only completing coding and did a first run of my solution with 90 seconds to spare. It didn't give the correct answer, and I had only about one minute to debug and try again. In that case, I felt immense anxiety and nearly gave up. But somehow I managed to keep it together long enough to spot a misplaced array index [ ] reference and produce a correct result.

On the second challenge, on my first run at about 17 minutes in I discovered that my entire approach was not general enough to meet all the possible cases. As I recall, while the rules of the problem are stated, you are not able to see test cases until you run. With three minutes left and the understanding that my approach was fundamentally flawed, I just gave up.

The experience was demoralizing. On the plus side, when I did live coding challenges for job interviews after that, I found them all comparatively easy and relaxed.

What would be better, in my opinion, is an overnight problem to solve, followed by a detailed walk-through of the solution the next day. That presented walk-through can prove to the interviewer that the candidate did not simply copy a solution (without understanding it) from elsewhere on the internet. Of course, if the solution was copied by understood, that should be positively accepted in many cases. There's a reason we use trusted libraries instead of writing our own code where possible.

This can be said of all interviews, not just tech. It’s hard for great many people to put themselves out there. I mean interview itself is a performance, isn’t it?

Having said that communication and interpersonal skills are big part of most jobs even in tech. We hardly develop software in isolation these days.

I don’t know about other industries, but in Tech there are some really inexperienced interviewers out there, who’re impulsive and lacks discipline in conducting objective and meaningful assessment.

I see lots of posts critical of companies and interviewers. The problem is there are too many qualified candidates for some jobs and companies must weed out candidates somehow. If they have 100 qualified candidates for 1 position, how would you rather they pick? The real problem is something else. It's a lack of jobs, inequality, the cost of living, and I don't know what else. Many people are getting squeezed.

I think the more accurate headline is likely to be "anxiety can negatively affect performance in interviews."

I would love to see what a control group of non-tech interviews looks like. Not sure what the equivalent "whiteboard while out of the room" is, but it's hardly just software devs who get nervous and can panic in interview situations.

There is a lot of anecdotes in the comments about these interview processes.

The ones with the best data are the company themselves, especially those who interview thousands of candidates a year. These companies (e.g., all FANG) have enough data to assess which interviewer, which interview questions and which interview process have positive correlation (and even causation) with future employee performance. And they can optimize the pipeline to interview most candidates with the interviewers that can actually assess future employee performance for instance.

Either these companies have already done these optimizations and analyze the trove of data they have, and the process is what it is for a reason. Or they haven't yet and there is still much room to optimize the process, and applicants have a chance to see those processes updated for the better (although here better means better for the company, not necessarily better for the applicant).

Tech interviews are kind of like a lot of games in life: do something difficult and boring in order to survive, and make it look easy and enjoyable while you're at it.

This is bad news for URMs. I am hispanic and was an international student— looking back there's many "extra" things that added to my feeling anxious before interviews. The imposter syndrome was real and I worried about wether I'd fit in, for example. I luckily don't have much of a language barrier but there were definitely a cultural barriers.

In the back of my mind I was always thinking I had to do more than "enough" and be exceptional. You read studies about how two people with the same qualifications might not get the same results after an interview due to unconscious biases by the interviewer. I felt like I had to be good enough that'd I'd overcome any kind of bias the interviewer might have. It's stressful.

When I last did a serious job search a few years ago, I thought I was pretty hot stuff. Nonetheless, I spent a few months reviewing algorithm text books and practicing code interview problems.

Then my first phone interview came, and I totally bombed it. And the next one. And the one after that. All in all, I think I bombed about seven code interviews (didn't even make it to an onsite) before I started to get some traction. One of the interviewers outright told me that I wasn't a very good coder.

After that I managed to get a couple of decent offers, but the process was quite humbling. If I interviewed again today I'd probably be in the same boat. Whiteboard interviewing is definitely an acquired skill.

Which is why you get a lot of over-confident engineers who are good at memorizing data structure/algo questions and talking BS but are terrible at real day-to-day SE work. The modern tech interview passes over so many great potential engineers.

I have extreme performance anxiety when it comes to interviews. Recently I had a very good experience interviewing with Netflix, where the take-home assignment allowed me to show what I was capable of doing. All the technical interviews after that were talking about the take-home at a high-level. The experience felt more accurate to how actual work is (receive an issue ticket, do work, then a code-review).

I was rejected in the end, and I believe I either a) wasn't a culture fit or b) they found someone with more experience (I'm 6 years in at my first tech job). What was nice about it though was that I was very calm and collected throughout.

This is only true at FAANG, guys, stop working at FAANG its not doing the world very much good and there are tons of companies around looking to hire you that are. The future is starting now and FAANG engineers won't be a part of it.

I feel like jkingsbery's comment[1] is relevant here:

> One of the interviewer asked me to recite the algorithm for constructing a Convex Hull. > ... > we struggled together for an hour, with him frustrated that I couldn't remember the details of an algorithm I last had seen 6 years earlier and couldn't recite in 60 minutes what took our professor 180 minutes of lectures to cover, and me frustrated that would have taken me 30 seconds to look up.

[1] https://news.ycombinator.com/item?id=23847814

I just spent a huge amount of time interviewing for a new job. They put me through an absolute ringer — something like 10 total screens and interviews, over about a month.

At the end, I started feeling a really negative association with them. I would never sleep well the night before the inevitable next interview and it just got to become a really tough thing. And most of the interviews were totally no-nonsense, no pleasantries, just tough.

I ended up getting the job but I turned it down. The interview process told me everyone I needed to know about the company culture.

> rather than whether the candidate is competent at coding

As a really quick reaction it's worth noting that coding is probably less than 25% of my job as an engineer.

In others' experience is this also the case for you?

I just went through an job hunt. I've been in tech for nearly two decades, and tech hiring has gotten definitively worse. It was never good to begin with, but it has become more painful from a candidate's perspective.

Employers are so focused on the well shared "the worst hire is a bad hire!" that they implement obtuse and ineffective hoops candidates must jump through in attempt to weed out "bad" candidates. This results in hiring those who survive the process, not those that will do the best work.

It seems like loads of comments consider "someone good at coding" as the only benchmark of a good employee. Even the report implies that "people who took the test in private passed". Passed what?

I am looking for people who can handle pressure, who can communicate clearly, who can handle the unfamiliarness of the experience, who can handle time pressure even though most of the time they wouldn't face that at work. If an experienced engineer cannot split a string in less than a minute on a test then they don't figure as a valuable emplyee to me. Why? Because anyone can Google something or apply deep thought in a private room but good employees imho can work with noise around them, with distractions, with changing priorities etc. in other words the interview is a poor representation of a real environment.

Sure, it means you might lose out on someone who just needed a few hours to calm down but how are you supposed to know that? Maybe they needed another few hours, days, weeks? Maybe the test you gave them was hard but if you asked them about something else, they would have done better.

It's no different than someone turning up late. It very likely wasn't their fault but you use it as an indicator anyway because you don't know them and don't have much else to go on.

There are literally dozens of reasons that tech interviews suck. However, they all mostly boil down to the same root cause: there's very little correlation between the content of most interviews and the jobs people will perform. Some real life situations that I've been on one side or the other of:

* Companies doing a "centralized" process that ignores the candidate. The questions are always overly generic and it really boils down to, "You need to want to work here enough to basically take any job we'll throw at you." Sorry, I don't want to work any company enough to just take any random job.

* Coders asked to complete academic problems that will NEVER come up for them in their real work.

* Coders not being allowed to leverage external resources like they will literally do EVERY DAY in their real jobs.

* Managers being asked to be coders in interviews.

* Questions that have binary "right" and "wrong" answers.

* Asking questions that have a million ways to do it and then honing in on one specific disagreement the interviewer and candidate had.

* Interviews that are more about the interviewer stroking their own ego and proving that they are smarter than the candidate.

* Completely ignoring that the absolute best indicator of future performance is past performance; not, "what quiz could this person solve in 45 minutes in an interview."

The corollary may be that anyone with low social anxiety is likely to do better.

https://en.wikipedia.org/wiki/Dr._Fox_effect

I'm genuinely curious - do whiteboarding and code exercises mititgate this at all?

Clearly there has be some minimal standard of ability. But given two similar performers, do performance-interviews favour those who project confidence? Or is there no influence on outcomes?

I had my first coding test for an interview recently (second dev job but a decade of other experience), over a video call, shared screen in a pre built AWS in browser IDE and I assumed I tanked it because it didn’t go perfectly. In hindsight it went like a typical 40 minutes of coding, I started down a path, didn’t quite work, had to console log to see what was up, tried a new tact and then it worked. They were clearly looking to make sure I had some basics of Angular down but also that I could problem solve in the moment and see where I went wrong.

I enjoy interviews (I like meeting people and chatting about their projects and problems) and I think running a VC backed company for nearly a decade has set me up well for detaching my performance in the interview to my self worth/abilities (ie. I’ve grown a pretty thick skin).

Ultimately it comes down to “can I do the job?” and “is their process set up to accurately reflect that?”. If the answer to the first is no then that’s cool, I’d probably have a miserable time in the role and under perform even to my actual level. If the answer to number two is no then I probably don’t want to work for them anyway as they are unable to accurately measure and analyse prospects/opportunities beyond just hiring.

I'm pretty proud of the interview loop we use at DefenseStorm:

- First section is "pair programming". We try to replicate what coding on the job is actually like: you can whatever IDE/REPL/language you want and can use google to look up docs. Our problem is specifically designed to be a fiddly problem which leads to edge-case handling with no possible 'beautiful algorithm' answer. Once they get the first solution, they have to revise their code to support a new fiddly use case. One interviewer actually provides help: pointing out syntax errors and suggesting alternate approaches if they appear to be heading towards a dead end. Most applicants "succeed" - so we evaluate on how well they can keep their code organized and readable, how they respond to suggestions, and if they are easy to work with/good communicators.

- Second section is code review. We give them several snippets of not-so-great code and ask what they would suggest we change before putting it into production. Does a great job of showing us where the candidate has depth of knowledge, and is a good approximation of a portion of the duties we expect them to perform. I was surprised to learn that very few companies do this - we find it extremely valuable.

- Third section is a conversation about tool familiarity (can you linux/aws?), depth of knowledge in the various domains we use regularly and designing an architecture (indicative of developer seniority). This is probably the least remarkable section.

Every section is set up so there are no gotchas and there are always multiple correct answers. I think we do better than most at minimizing the anxiety inherent to the interview process.

Coding interviews are not about coding skills, they are about communication skills.

I thought I had discovered a clever hack when I was interviewing. You ask enough closed questions about the problem that the interviewer tells you the answer. Then you repeat the answer back to them, as code or aloud, and you pass.

Now that I am interviewing, I am desperate to find anybody who will ask me enough questions to have me reveal the answer, and then repeat it back to me.

Used to run interviews by putting candidates at a table with a laptop (typically their own). I'd give them a coding question and 45 minutes to solve it, and then spend 10 minutes walking through their solution together. I'd sit across the table from them and do my own work. They could stop me and ask questions all they wanted, just like we were teammates.

It worked great. It also gave me (the interviewer) some time back in the day.

The only thing that interviewing assess is how good at interviewing the candidate is.

I swear that for 90% of the typical software interview loops, someone with some minimal understanding of computer science that spent 2 month studying the typical Leetcode problems would probably have been fine and do way better than a software engineer coming out with 10 years of experience that didn't take the time to review his leetcode problems.

What are people doing as alternatives to these kinds of interviews? The research is quite clear that technical interviews are worse than useless (literally -- they will select for the wrong traits in your candidate pool), yet they still seem to be the dominant hiring practice.

My current company does a few rounds of low-key chatting, some about technical stuff, then a 1 to 2 month trial period with the understanding that there is no guarantee of a job at the end, it only becomes permanent if it's a good fit on both ends. Obviously this is time consuming and expensive, but it works because we're a small and very stable company, which I realize is very much _not_ the norm. This does work great, though. They've managed to build a team of excellent, autonomous, highly skilled developers who stick around for the long haul. Wow, that sounds a bit self-aggrandizing -- I'm really thinking of my co-workers, I hardly count myself in that category. Maybe someday.

Anyway, I'm genuinely curious what other models you all use that work well. By "work" I guess I mean result in high quality and long lasting additions to a team.

I think this is obvious, but look at the alternatives.

* Resumes can be exaggerated, falsified, or perhaps more often often leave out important details or relevant experience. Reviewers often also put unconscious emphasis on formatting or style (I had one colleague that I swear would accept anything in Computer Modern). * Take-home projects (if not carefully designed) can be a considerable time sink and reviewers may be biased toward particular coding styles or languages. * Prior employers are often not allowed to provide meaningful references, and even when able to, won't often have the time or motive to do so. * Referrals will be biased towards the friends of your current workforce, making it hard to improve the diversity of your workforce, and are a pool that will become exhausted as you scale or if your employees stick around. * Behavioral interview questions are biased towards candidates that know to prepare for an even more contrived corpus of questions with even less bearing on day-to-day job performance. * Candidates may not have prior code that their are able to share; certainly most employers make that difficult or impossible. Expecting side projects or open source contributions biases towards candidates that have means or desire to program outside of work.

I think it's ridiculous to think that a programmer should be hired without any part of the process evaluating their programming ability. The key is to do this while minimizing stress for the candidate. Explain the format of the interviews up front. Tell them about the people they are interviewing with. Let them have breaks. Help them through tricky parts if you have to. Avoid things that aren't realistic (whiteboard coding) but don't shy away from things that are (whiteboard a system design). If leetcode doesn't represent what you do at work, don't make them do it.

>”What’s more, the specific nature of the technical interview process means that many job candidates try to spend weeks or months training specifically for the technical interview, rather than for the actual job they’d be doing.”

Had to go through half the article before this showed up. You could be learning something new and useful rather than spend so many cycles trying to master trick problems.

To me, the bigger question is, why do these interviews require every candidate to prove their programming/design skills from scratch? Say I have worked at a respected tech company for N years, my record there should provide a lot more signal than a day’s worth of interviewing would. I think we need some sort of standardized metrics for different dimensions of tech skills, and those metrics update over years of experience. Then companies don’t need to assess tech skills - they just need to look at your record, knowing that it is objective. Certifications, open-source contributions are some examples. The problem is the work you do closed-door in a company cannot be objectified or standardized easily. If we can solve that, that would go a long way here. Interviews can then focus on soft and behavioral skills solely. Not to say that those interviews are any more useful or signal-deducing as they happen today.

Will anything change in the near term (~2 years), I would bet not. I sure hope the way interviews are conducted changes. After all, most of us aren't trying to spend many hours on sites like leetcode.com. I find those sort of problems to be fun and enjoyable on my free time, but they can be VERY challenging under pressure.

Have you ever read Coders at Work?

One of the things that terrified me when I read it was Ken Thompson's description of his interviewing process.

Paraphrasing: He would ask about a project the interviewee was proud of, then drill them with critical questions about it.

I actually like the general idea. But the social pressure of this situation seems to be quite intense.

I have basically been doing a second job sending out applications and interviewing hard core for about a year now. There is so much time I have accumulated in take home projects. Anxiety is a big issue for me interviews(Yes, I am a woman) and even in ones that go well I am still clearly minorly affected by it. There was one back in November that I got to the stage of being flown out to their office. At the end of the interview during small chit chat I was nervously talking about the stickers on my personal laptop.(My plane had been delayed to weather and I was basically interviewing with them a couple hours after everyone would normally stop working.)

My count on applications is well past fifty and I have done so many phone screens(HR and technical) that all of them seem like a blur to me. At this point I wish I could throw a portfolio of my work at a company and get a job. From a Reddit thread about "unrealistic car mods" shows my overall skill set in my ability to get the job done. https://www.reddit.com/r/cars/comments/hdrgv1/car_people_of_...

My current job I got because I found a major vulnerability in the company's authentication system and recovered the private key for the client to server encryption.(I didn't make much money at the time and didn't want to pay the subscription cost for the premium service.)

I have interviewed/hired nearly every person on my current team. One of them was almost denied by a senior manager because the candidate wore a backwards baseball hat during the interview.(Lack of professional attire, etcetera.) The rest of the interviewers over ruled the manager saying that was a dumb reason. Turns out this candidate was super conscious of a balding spot on his head and he wore hats to hide it. He turned out to be an amazing coder and great at solving problems that came up.

For me, it's almost impossible to think in any kind of group situation. I do ok sometimes on interviews. It really depends. If given time alone, even during an interview, with my own tools and without distractions or having to tell the idiot interviewer what I'm thinking (as if anyone can think and talk about two different things, the problem and what the interviewer wants to hear at the same time), I do amazingly well. With another person watching, in-person or remote, I can't even think of how to start. I get caught in loops of anxiety and I can't relax enough to think things through. I also am afraid of trying wrong things so sometimes get paralyzed a bit. Mainly, I try to avoid such interviews and go for ones that will let me solve the problems on my own, with my own tools, without some idiot watching over me.

In one of my recent interviews at a mid size silicon valley company, I was given 4 hours to make a tetris like game.

One of the most enjoyable interviews ever with no whiteboard and no one around me to check on my progress. I was blank for first half an hour or so but made great progress by the end. Eventually got the offer.

20 years startup/internet engineering here. I have "failed" roughly 50% of my interviews usually as the result of failing the coding rounds. Almost all of fails were conducted by someone with nearly half my years of experience who were interviewing for something from the CS academic curriculum. When I am asked to conduct an interview, I don't do anything much more complicated than a glorified fizzbuzz. I prefer doing architecture interviews.

My biggest problem with algo interviews is most of them are trying to determine if you happen to be familiar with one particular algorithmic solution to something - if you don't happen to know that one algorithm, you fail. In the real world, you have to learn as you go. You never know the solution to every problem in advance.

The problem is we don't like the process, but not many people propose reasonable solutions. One extreme is just hard LeetCode questions, and the other is, "they should just talk to me about my experience and projects." We need something in the middle. Some ideas I've been thinking of:

- Pair programming with a problem that neither the interviewer nor the candidate has seen. Doesn't matter if they can't find the optimal solution. It would show how good of a communicator and how proactive the candidate is.

- Make the candidate implement a solution to a problem that's already solved. The interviewer shows the problem, explains the solution, and the candidate needs to code it. It would reveal the coding fluidity, understanding of specs, how they ask questions, etc.

Thoughts?

This is why I really go out of my way to try and dissipate the anxiety as much possible. And because I feel anxiety myself when I interview people, so I remember that we may share that perspective and we'd both have a better conversation if we could relax a bit.

My interviews therefore end up being conversational, and definitely non-confrontational. I want to hire someone who the team will get along with, who knows what they are doing, but in the end I have to remember that if we hire them I'd prefer we didn't start off our relationship with an over-the-top technical interview.

So far I've been successful at finding good candidates this way. Maybe it's just luck. But I don't think leetcode interviewing is the only way to get well-qualified employees.

As the interview process seems so crap and few people have any suggestions, here is mine.

I got asked to bring in my laptop with some code of my own we could discuss. I had a side project at the time so wasn't a problem. I got asked some questions, asked to add a simple feature. It was pleasant. I knew the code, so I was relaxed and didn't need to look up stuff. I think the interviewer learned some stuff as well.

The only problem people might say is that you used someone else's code. Even if that was the case, if someone else can talk confidently about code that they didn't write and jump in and add a feature easily, you have probably found someone good.

Can we try and push this as the new way of doing interviews. Low stress and respects peoples own time.

There's one genetic factor known for the performance variance against the stress, which is also called "warrior vs worrier": https://www.huffpost.com/entry/stress-management_b_2671591

IMO, there's Moneyball-like opportunity for companies who can utilize (or, in another word, exploit) this aspect. A tech company may conduct tech interviews with a private environment & a traditional environment, and if one's performance is much better in the private setting, they might be able to offer a job with a reduce salary than the candidate's capacity shown in the private setting.

I get massive anxiety interviewing for tech companies, and I've been doing it for many years. The worst interviews are the "trivia quiz" questions that are either correct (according to them) or incorrect. There's no discussion of how things work and no frame of reference or anything. For example, when I interviewed at Amazon coming from Microsoft, they asked me lots of Javascript 'trick' questions even though I told them I was using C# day-to-day. I knew most of the questions, but got caught up on a couple of JS 'tricks' that I just couldn't recall at the time.

Sadly, it's all a game, but the good thing is that interviewing is a skill and can be improved with continued practice.

First thing I'll say is that I'm not an advocate for how industry often interviews today, whiteboard and puzzle problems and all, and their efficacy is worthy of discussion. It's a hard, hard problem and there are major deficiencies. But this academic "study" seems highly suspect.

48 subjects and we can make a generalization? People's stress reactions seem intimately tied to enormous number of contextual factors. Looking at the abstract it seems the "goal" of this study is "to make problem-solving assessment more equitable and inclusive". A noble and worthy goal but this does not sound like academics or science. It sounds like its conclusion was sought for.

The current "tech" job interview process is engineered to select for people that are both competent and secretly insecure about their skills, and to do this with little or no false positives. Competent employees are, well, competent and secretly insecure employees tend to have a need to impress. Both of these traits works to employers benefit.

A large amount of false negatives, like candidates that would otherwise be great employees but maybe just suck at whiteboard coding, is considered acceptable by most employers because research shows the cost of bad hires is much much higher than missing out on many good hires. Employers know the current process is miserable and leaves a lot of good people behind.

I can attest to this. A number of years ago, after fifteen years as an engineer getting jobs through my reputation, I went into my first white board coding interview, with one of the FAANGs. It was a Boggle like problem to find possible words. While at the board my back was killing me, and I had a horrible time thinking due to anxiety. I knew that I wanted a trie, but I blanked on how to implement it properly. A couple hours later at home I remembered all of it.

That said, it did filter out people who had trouble handling the new stress of a whiteboarding interview and people who did not practice whiteboarding. If I ever interview for a FAANG again I'll make sure to do a lot more preparation.

I think for senior level positions especially it is more important to see how a candidate approaches a problem beyond just memorizing algorithms or LC solutions. I wish there was way more emphasis on "case study" style approach where you could discuss at high level how to build X. Even pulling the candidate into a real technical discussion where you're trying to optimize something and see if the candidate has any input. Building up on experience is what we should be after.

To me, asking LC or the same algo and data structures questions (implement DFS .. wow genius) is just gatekeeping via a measure that most people can pass by just memorizing stuff for a month.

I think there are a lot of hidden biases in hiring that no one will overtly acknowledge e.g. personality, gender, attribute of race (yeah I know, but still there are people unaware of their racial biases), ability to take orders, if they are too good and might threaten their job or make them work harder because they will output more completed work, etc.

usually this line of thinking is summed up as the follow question: Can I see myself working with this person for 40+ hours a week?

then they bring down the axe with the following: not a good 'culture' fit and move on to the next candidate and the broken cycle continues.

This is all in addition to the broken whiteboarding challenges.

I'm susceptible to stuttering and/or rambling when I'm extremely anxious, sometimes without even being fully aware of it. I made it to final interviews for tech jobs twice, and I got the offer both times.

In retrospect, I was extremely anxious the first time, but the company was about to enter a high-growth phase and didn't seem to have fully-developed hiring practices (I got a tentative, very informal verbal offer on-the-spot, and a phone call to confirm I had the job about 24 hours later).

The second time, I was interviewing with a big US bank so I assume they had a better eye for details. Maybe I'm getting better at not showing my anxiety.

The root problem isn't the interviews themselves, but the belief that--despite a stunning lack of evidence--interviews are somehow useful measures of future job performance.

And I believe this applies to most job interviews, not just in tech.

Hmm...but if everybody is stressed out, then it equalizes and you're testing for skills again, just skills under pressure.

And having skills retained under pressure is actually a valuable trait in this industry. And at least at desirable companies, the process is designed to minimize false positive, it doesn't care that much about false negatives. This is by design.

My guess would be that they filter out orders of magnitude more people with poor resume-writing skills or that didn't go to the right school.

Although I do agree that interviews that you can/must practice for are silly and there's a lot oof that around.

At Google I'm convinced our interviewing system only works due to a combination of luck, large numbers, and generally good intentions. None of the individual steps is a particularly good way to tell if someone will succeed at Google, but anyone who passes all of these filters is unlikely to be a terrible engineer.

When you avoid terrible engineers and hire by the tens of thousands, you get plenty of good engineers and a few great ones. You also miss a ton of qualified candidates, but Google has more of those than it needs. It can afford to miss them in a way that small companies can't.

I have found that what best predicts interview success for me is the ability to confidently give answers, even if they are nonsense. One of the problems is that interviewers often aren't sure of the answer either.

I work in an interesting tech career that does need to assess this: Pre-sales Engineering.

At my current job, we have our candidates do a short take-home assignment to assess coding skill, but we also have them do a live presentation to assess their pre-sales skills and ability to present under pressure.

We try to separate the two tests to get a better read on the two skill sets we're looking for. So far we've hired strong candidates. We prioritize avoiding false positives over false negatives; in other words, we prioritize turning candidates away over hiring someone we're unsure about.

I leave tech interviews feeling like I had just been through a hazing ritual.

I don't think it helps that what works for some people doesn't work for others. I coded very very little before I graduated college but got my first tech-ish job because I knew a guy and I could explain my thought process well despite having no idea what grep was.

A friend of mine would have fallen on their face during that sort of interview, but they're more book smart than me and write quite well.

If we were the two candidates there, I would have got the job, but they would have done it better. I don't know what the solution is, but it isn't more of the same

Engineers tend to over-focus on correctness. I’ve run hiring process in the past where over a year we interviewed around 50 engineers. Feedback from our own engineers involved in the process would often be like “Seems like a nice guy. But his solution to the challenge - one he’d never seen before - had a bug in it! And the solution was just O(2^n) and failed to notice the O(n^2) solution. Even though I glared at him a lot and kept distracting him with hints” ... I’m exaggerating for effect of course but some form of that would happen often

The best interviewers can feel you from certain job experience questions that you’ve done in the past. It’s looking for a series of related answers that a candidate just cannot make up on the spot.

So what are we going to do about it? Nothing. Absolutely nothing, that's what.

This happens all the time. People complain and complain, something comes out proving them right (or at least confirming their opinion) and all that happens is people relate. Politics, jobs, healthcare, interviewing practices, work conditions...

If programmers actually joined a union and organized themselves to make a db of companies with shitty interviewing techniques, bad business practices, etc. maybe something would change, but hey, complaining is easier, right?

At one company where I was the interviewer the company used a few simple java questions for each screening (not great I know). It was basically just a way to make sure they did have experience with Java at at least a junior level. This became an issue when we started getting candidates from recruiters. One candidate bombed on the first question. Then when we asked the second question they answered the 3rd. Clearly the recruiter was trying to coach the candidates based on past questions.

The strangest thing about jumping through big tech corp take home projects and multiple interviews is they then place you in their internal training boot camp and you realize why didn't you just stick me in here in the first place then rank my results when we finished training instead of wasting money for a week shopping me around departments so highly paid senior employees could all stop what they were doing just to have a go at me with the whiteboard and multiple on site lunches.

This makes sense if you’re at a FAANG organization you shouldn’t need to make breakneck code changes with people breathing down your neck.

At a lean startup or smaller team though this isn’t unheard of. I’ve had to debug and code up multiple things while everything was on fire around me.

Mistakes are guaranteed to happen no matter how many precautions are in place. You can reduce the probability of a mistake happening but when it does whoever is fixing it needs to understand how to stay calm and work well under pressure.

Most interviews I've done in my software career were done on more than 24 hours without sleep.

Not by choice, but because I cannot sleep the night before an interview. I have difficulty sleeping in general, and the anxiety of an interview drives it through the roof.

Whiteboarding is also terrible, pretty sure it is used intentionally to skew hiring results toward certain types.

If someone is looking over my shoulder while coding I can't think, I think best when there are no external inputs(no noise/eyes closed).

Idea:

Assign a short coding challenge and rank candidates on their solutions.

Then, one at a time, hire each candidate as a contractor for 20 hours of work. Let them fix bugs and do code reviews. See how effectively they communicate through code and in code reviews. Invite them participate in the weekly standup meeting.

Finally, have team members who worked with the person write confidential feedback with a hire/no-hire rating and justification. Hire the first candidate who gets all 'hire' ratings.

When you're under time-pressure, you actually spend more time thinking about how much time you have left, rather than thinking about how to solve the problem.

It's like that parable.

2 students approached the master, wanting to become swordsmen.

'I want to reach your level in 10 days', said a student. Master replied 'It will take you 10 months'. The other student said 'I don't care if it takes 10 years, Master'. The Master said 'and you will learn in 10 days'.

I find it interesting that academia often points out that skills interviews have a soft-skills bias, but rarely do I see them consider that this may be a feature and not a bug.

> “Our study suggests that a lot of well-qualified job candidates are being eliminated because they’re not used to working on a whiteboard in front of an audience.”

This is a common daily task at any company I've ever written software at. People who can communicate their ideas effectively are better at their job.

This problem is perfectly clear to people who have had senior level careers in any other professional industry. Software does not know how to assess people because it lacks ethics and standards.

It’s a problem that most people seem to not want fixed until they are themselves negatively impacted. Since the very concept of ethics is so utterly foreign its need and application cannot be effectively communicated without resulting in some manner of embarrassment or hostility.

As suffering from anxiety disorder, It totally hits home. I have been a top notch employee at every place I worked, but I totally bombed most of my interviews.

> In short, the findings suggest that companies are missing out on really good programmers because those programmers aren’t good at writing on a whiteboard and explaining their work out loud while coding.

In most of my programming jobs, explaining your work was most of the job and writing code was less important.

The whiteboard environment is a bit unnatural but I don’t want to hire someone who gets the right answer if they can’t explain it to me.

I sincerely appreciated that the technical portion of the interview for my current employer consisted of pair programming with their developers to improve on the pre-screening "take home" assignment I completed. It actually gave me a chance to explain my thought process and work through problems in a similar way to how it happens on the job. It was infinite preferable to white board algorithm questions.

Nothing will change.

I think the problem often lies with the interviewers’s ego that results in this strange paradox where they’re trying to demonstrate their competence to the interviewee by way of technically “outwitting” them.

That way, in the event the interviewee is personable enough to get hired, the working dynamic is that much closer to being formed between the two, and chances are reduced that the interviewer is hiring a potential rival.

I shared this article with my company. It was amazing how fast our leadership (particularly those involved in interviewing) came back with critiques about sample size or if they properly accounted for student performance in the study. Perhaps fair critiques, but it's not like this effect is unknown by software engineers (unless your head is really stuck in the head). Ego defense at its finest.

A lot of it relates to how much the interviewer can create rapport and comfort. The interviewer having the upper hand in the power balance, they should go the extra mile to set the stage correctly. Many good engineers are terrible at interviewing. In the wake of the interview they boldly set forth their assessment while being largely ignorant of their own negative impact on what just happened.

How many years took them to realize this?... As someone who spend like 6 months job hunting and doing more than 20 interviews(including Google, Microsoft, etc), this was pretty evident. Yeah, I forgot to code that weird problem which implies the use of dynamic programming in a white board, but hey... what about all those years of experience and evident code skills. It is ridiculous.

Occasionally I work as a speaking examiner, assessing people's foreign language level.

The test is rigorous and well designed. Few minutes of questions are designed to be so easy any can answer them. "How do you spell your name?" etc. This is by design as people can be extremely nervous when taking a foreign language exam.

If you are interviewing people, do what you can to put them at ease at first.

Totally true that people with no self-doubt/impostor syndrome overperform their ability level in whiteboard interviews compared to people with more self-doubt, who are often the better engineers. I am personally a beneficiary of this. It also contributes to all of the terrible dick-swinging culture in a lot of tech companies since these are the people who pass the interview.

Its really funny. I was interviewing at a large company recently and the way they do interviews is weird. Each team that is interested in you does their own on-site. The first one I bombed b/c I was so anxious but after I experienced it once, I aced all the other teams I interviewed with. Nerves definitely play a role, its cool to see that it is being quantified.

I always care about explaining to the candidate how the interview will look like, and what are the most important elements I care about. When you skip this part, the candidate has no idea if for example during live coding, you care about the results, the code quality, IDE shortcuts fluency, etc. The feedback after the interviews proves it is a game-changer.

I used to be super anxious and get brain freeze.

This doesnt' happen to me after I grinded leetcode for a couple of months.

you are anxious if you feel unprepared.

I do think this is true - but I also think that the ability to control anxiety (an increased tolerance for uncertainty) is a characteristic that is learned with experience and can be a marker of an experienced engineer. Someone who panics in an interview might also be inclined to panic when a manager frantically reports a bug in master, for example.

I've posted this before on HN: Tech interviews are designed to find people who are happy in a role where their level of responsibility begins and ends at: "Here's your coding assignment. Go do it". The interviews are designed to eliminate people looking to get in the door, then quickly move into or up to a non-coding position.

This should hit home with everyone. What I would add is one thing that adds to the anxiety is whether the interviewer is looking for reasons to _reject_ a candidate (i.e. don't let the bad ones through) vs reasons to _hire_ a candidate (i.e. let them shine on aspects that are hard to communicate on a resume or when checking references.

Anxiety management is absolutely foremost in these interviews. I did not successfully get through one of these interviews until I started experimenting with taking anxiolytics beforehand.

I eventually found that a moderate does of phenibut 5 hours prior was an ideal balance of social inhibition vs the types of impairments that come with these drugs.

As someone with bad anxiety and who gets panic attacks in roughly one third of panels, it does feel fairly discriminatory.

The best interview I had was when they asked me a whiteboarding question, gave me a pencil and paper, and left the room. I solved it, they came back, and we talked about my solution. They still got to hear my thought process.

I remember interviewing at Google and being completely unable to think or relax. Instead of making any effort to learn about me and my skills, the interviewers became more aggressive and nitpicky.

Looking back, my life is dramatically better working remotely for the same level of income with friendly helpful coworkers in a non toxic environment.

Last year, I went to both Amazon, FB on-site interview at Seattle and Vancouver office, and completely bombed it (not quite, I solved 50% of the programming questions). I felt like a fraud when try to answer the LP questions. Anyway, is there any companies other than FAANG that can support relocation (to US or Canada)

But did they see if being neurotic is a net positive or negative for doing the job. Skill isn't the only variable in being good at a job.

If you skilled but you neuroticism makes you less productive even compared to someone less skilled who cares about skill level.

You could end up with a skilled team that folder under low amounts of stress.

I have totally blown interviews and also done very well. To me, the interview is a distinct skill set. You have to treat it as it's own thing. What matters is practice, practice. Not a little bit. Say 6 weeks. Then you are ready to slay. When I haven't done that I've been rattled and done poorly.

Suppose you're BigTechCo, and your interview process passes up this huge pool of talented developers who don't interview well. Then your competitor ScrappyStartup, who gives some of these folks a chance, ought to be eating your lunch, right?

Something to think about, especially if you're involved in a startup...

To a first approximation, that conflation is fine; software engineering is stressful work where the tasks to be solved often involve known and unknown unknowns, and it's important to observe how a candidate performs under pressure because the job will include pressure.

Looking at the article, it seems all they looked at was whether the code worked or not.

interviews also assess communication and thought process. and working code isn’t necessarily good code. also wonder if the people in the private room used their phones to research answers

To be fair, even these interviews are mostly garbage, they don’t have a better way to gauge you with just 2-3 meetings. Which really sucks. What they should do is to check your pass job experience, verify from the people you work with. That’s not easy.

I feel like sometimes big tech companies give technical interviews when they're not actually hiring, that it's a sort of formality. Can't really confirm it, but thinking this way makes me feel better about not passing to the next stage.

I didn't really want to speak up on this but yeah this has been a major problem for me. I have a disability and diagnoses with anxiety disorder. These intensify the symptoms even more and sometimes I blank out.

I failed every single interview when I had to do programming task on the spot, however people who hired me based on my resume and how I talk about solving problems, had great business with me for many years.

This was interesting:

For example, in our study, all of the women who took the public interview failed, while all of the women who took the private interview passed.

That is an amazing statistic! From 100% failure to 100% success!

I haven’t interviewed for a job in nearly 19 years. I think job interviewing has changed a lot in the intervening decades and I’m a bit worried about how I’d do if I ever need to interview again.

Technical interviews do not resemble any kind of professional software engineering environment that exists in the real world. That is their biggest problem. Everything else is secondary.

Get a whiteboard at home - even a small one. Try to write an algo on it. You will be amazed at how stressful it is, WITHOUT anyone looking over your shoulder.

But with practice, that stress goes away.

Direct link to the paper: http://chrisparnin.me/pdf/stress_FSE_20.pdf

Agreed! Cut throat interviews in which you are expected to perform under heavy scrutiny can heavily affect performance. I prefer the case study approach instead!

I fear there's a movement to get rid of leetcode interviews and I'm against it. What's the alternative? We go back to "you have to look the part", nepotism, credentialism, and other such BS in other industries. A data structures and algorithms test is fair and objective. You can objectively get all the challenges correct and then there's little reason to not hire you. Compare that to interviewing for, say, an investment banking position where you're judged on stuff like what frat you were a member of and how charismatic you are.

I think we need to come up with effective alternative to whiteboard interviews. Who’d be interested in forming a working group to come up with something?

Ok, so what can I as an interviewer do to reduce stress for the candidate?

Let's make good use of this discussion and compile a list of practical suggestions.

Can I pose a question to people here.

Would you have moved companies earlier or moved more often if a coding interview was not standard practice in hiring?

Managing your own emotions is a part of EQ. Why should we hire people with low EQ to a job as collaborative as software engineering?

Unfortunately corps understand this and assessing anxiety is the purpose. Experience is more indicative of technical capability.

Just curious does anyone knows or has created a list of companies that gives you take home tests I would love to find out

Fizz buzz/leetcode != Real software engineering

I think we should come up with new software engineer interviews techniques

I always had the feeling this was the case, so my strategy has been to gaslight the person interviewing me.

I love how so many are passionate about this subject, always comments into the hundreds.

I've written about this before.

I've become pretty much convinced that many of the leetcode tests are "young-pass filters."

I recently saw an ad for a two-month, intensive course on how to pass interview questions.

The heck with that. I want to learn relevant stuff.

If people don't want me working for them, then I'm not particularly interested in working for them. Too bad. We could have made beautiful music together. I'm no slouch.

I usually just imagine visiting some friends that I haven’t seen in a while.

Well that sounds good to me. Sofware work can be pretty anxiety inducing.

The tech industry, if it can manage to do it, would benefit immensely from enacting interviewing methodologies that aren’t ultimately rooted in an underlying need to haze others. It’s as if the CS nerds forgot that when the frats hazed others we were disgusted by it.

What causes anxiety is wasting numerous weeks of your life that you could have used to pay your rent or spend with your friends, family or do something meaningful with your time. That's how much this people think you're worth!

Tinder generation, swipe next.

Social engineering like it's 1000 AD pre technology era.

No matter how difficult a coding interview was, the job afterwards was always harder and more challenging - taking that into account I would rather for even harder job interviews to reflect the actual difficulty of jobs.

I stopped applying to FAANG because of the exact reason.

The comments in these threads must give tptacek PTSD.

again, this demonstrates that interviews in tech sector are all wrong.

You don't ask plumbers to show how they do the job before hiring them.

I think this is known for years. Is't it?

Uh, maybe at times, but I assure you -- I've interviewed some VERY confident, VERY incompetent software dev candidates.

Tech interviews are an evergreen source of delight /s. I have been the interviewer and interviewee plenty. A few reflections.

- I try hard, and I believe with a certain success, to follow a sound approach to interviewing. I read the resume, I see where their contribution to the team I am hiring for can come from. Then, false positives and negatives never go to zero, and such is life (although some people are much better than others at observing and judging). I mean, people get married and divorced in 6 months, and it follows that there are inevitably bad assessments also in tech. But I try to ask questions that can allow me to gauge the skills of the person in front of me for the job that they are hired for. A conversation, more than a test. Things that can get googled are of no interest to me, for example. Or theoretical and never-to-be-used-IRL knowledge. Can you imagine do a try-out for a hoops team, you are a guard, and they try you out as a play? Or you do a try out for the Patriots as a quarterback and they test you on punting? Which is what happens in tech more times than expected by chance.

- Among the most nightmarish experiences I had as an interviewee, I remember: a hiring manager yawning for the whole interview; another time, I successfully (brilliantly?) completed a coding test in a few minutes and the interviewers said it was all great, I have no other questions. But then the recruiter told me that no, the interviewer felt that coding was so and so (what?). Another time I was dismissed after the first of five planned onsite interviews. No idea why.

- What about homework, you may ask? I took a few (2 or 3) homework for some of the most popular companies in SV. Well, one time I made this model and I don't know, maybe I will die an Achilles death for my hubris, but I believe I can fit a tree-based classification model after 20 years of work. The recruiter said that both graders like my model, but I did not do hyperparameter optimisation. I told them that the optimisation was both in the documentation of the homework and the associated Scala code. They said they were asking the graders again (it was just one, the other copied what the other wrote) and that they would have been back with an answer in a week. After a gentle nudge, they said that no, the graders did not like it anyway (they never took a second look and I wasted my time: for what, exactly?). I am not gonna take any other homework from any company, if not for reasons of desperation or destitution and I hope it won’t ever happen.

- Most companies rarely if ever do any business or technical analyses of their interviews. Google pollinated the tech world with their interview strategy and the other companies said: if Google interviews this way, why should we do any different? I work for a big company now, there is no post-mortem, and neither pre- nor middle-. We hired more than 100 people this way for a new team, it could have been better, worse, who cares? Nobody checks anyway. Because, and this is the main insight I’d like to present, especially for big companies (for start-ups it is different, an exceptional IC can “make” the company), it does not matter. There is a big area of 20+-D skill-space that is “flat” with respect to the employer’s contribution to the company. You don’t hire that uber-incompetent and you are doing fine because the market, overall economy, luck, people skill of the C-suite are much more important than ICs, managers, or directors’ skills (beyond a certain threshold, of course). Google prefers to hire talented people and then it does not allow them to work on anything crucial because the most important thing is not letting them work for other companies? It does not matter. And if the question is: but if the interview process is inefficient, are we not wasting thousands and thousands of employee-hours? I have got the same answer for you: it does not matter, because there is often no need (in big companies) for more than 20% of the time of the employee, people get hired for all sort of reasons, but rarely for (true) technical or business reasons.

does it mean that take home interviews as the best form of interviews ?

One observation of mine is that interviewing is often more about finding the most feasible and consistent process rather than the most effective one. Whiteboard interviews make for consistent evaluation and it's easy to train up new people on a set of questions. Scoring based on how far the interview got, and the level of optimization of the approach used is relatively unambiguous. So it's logistically easy and less subject to bias - that's why it's popular even though it has little resemblance to day to day work.

A more effective interview would involve tasks that more closely resemble day to day work. However, the examples of this that I've seen so far make for much more ambiguous evaluation or are much more difficult to schedule:

One example interview process I've seen is having people submit PRs to a codebase to fix bugs or implement features, and then review a PR submitted by the interviewer. This more closely resembles day to day work, but has the disadvantage of spreading the interview process across multiple days. Additionally, you might tell the candidate that they shouldn't take more than 2 hours to implement the PR but who knows if they spend 10+ hours on it.

Another alternative interview format is a 2-hour long one that starts with the interviewer asking "how would you build a text editor?" There's no right response. If the candidate gives responses like ropes and gap buffers then the interview might go in a direction focused on data structures and systems. Some candidates ask if it's a WYSIWYG editor like word, or an ASCII/unicode editor like Vim. In that case the interview might test the candidate's abilities to think through how to build an interface to decouple the UI and underlying data structures. The candidate might start off with a simple array, and work through why it becomes infeasible at larger lengths and think through ways to mitigate that. This interview was flexible to test technical knowledge and reasoning for candidates at any level, and could go in a variety of directions. But on the other hand that makes consistent evaluation difficult and training people up on this interview similarly difficult. This was at a small company with maybe 30-40 engineers, where each team basically had total latitude on how it carried out its own hiring. That's not how a lot of larger companies' interviews work, which often emphasize consistent evaluation and multiple interviewers.

The only interview which does more closely resemble real world tasks, and I suggest that more companies employ is debugging interviews. It requires the candidate bring a laptop and that the company build a couple repository templates, but once that's done it's a very easy interview to conduct. Just observe the candidate debug and make note of how many bugs are fixed and whether the fixes do bad things like breach layers of abstraction.

I wish I had an agreeable way out of this field. I'm not going to critize the current interview process, it has its advantages and disadvantages and theres a reason it's used, but if its shown me anything it's that I chose the wrong career.

> the findings suggest that companies are missing out on really good programmers because those programmers aren’t good at writing on a whiteboard and explaining their work out loud while coding

Sure, no love for whiteboards from me. However, this does suggest an arbitrage opportunity for an employer who is willing to look differently at these candidates, and then profit handsomely, Moneyball-style.

There's a bunch of money just being left on the table and nobody is picking it up, really?

My two cents is that interviewing candidates can give a whole lot more insight into how to perform in an interview. Interview performance is a skill that is separate from the actual job, and gauging that performance is sub-optimal. But everything in this space is sub-optimal, it's all pick your poison.

Maybe some people (how many?) fail because of anxiety, but on the other hand, anybody who doesn't fail certainly has proven some minimal coding skills?

I usually took it as a positive if companies did some basic coding skill test, showing they at least cared that people can code.

Also, the people who fail because of anxiety, how will they fare in the normal job situation when they are supposed to discuss stuff on the whiteboard with colleagues?

Maybe the results could be taken as "give people who fail the coding interview another chance", but I wouldn't take it as "don't do coding interviews".

"Technical interviews are feared and hated in the industry, and it turns out that these interview techniques may also be hurting the industry’s ability to find and hire skilled software engineers."

It doesn't hurt "the industry", there's a competitive edge to hiring otherwise superior engineers that are passed up by poor hiring practices at certain companies.

Not everybody needs to work at FAANG, or worse, companies that act like they're FAANG. Less money, less bullshit.

If the results of this experiment are true part of winning the game may be taking some anti-anxiety drugs before the interview... I heard high blood pressure medicine works.

Anybody have any first hand accounts on using drugs to calm anxiety before the interview?

Very true. I suffer from this myself, and I have interviewed for Google twice. When I interviewed the second time I thought I bombed, but I was hired. With the benefit of hindsight, I can rationally tell myself that me getting into some pretty high flying places is not a coincidence, but I'm 1/3rd of a programmer in an interview compared to when I'm not under stress.

Well they are auditions rather than interviews. So it seem to be accurate.

Current SWE interviews seem extraordinarily effective. They've produced corporations that dominate our lives, with systems that have unimaginable capabilities. Their profits currently dwarf other companies such that the S&P500 is basically a tech company index. More companies will adopt hiring practices like these as the applicants become more competitive. SpaceX has grueling all-day multiround tech-like interviews like this, whereas Boeing just does 1 hour with 2 managers. The effect on employee quality is clear.