Hiring Without Whiteboards
github.com
github.com
2. a programming task. Choose 1 of 5 tasks. The tasks are not abstract problems or puzzles, but come out of the designs we've implemented. We ask for 100 lines of code and not more than 2 hours. They take as much calendar time in days as they need, most have full time jobs, and let us know when you're done. The candidate then comes back, typically in 2 to 7 days and hosts a code review in front of our team. I give strict instructions before hand: The purpose of the review is to learn how they thought through the problem and how they solved it, not to criticize their style and approach.
3. If we decide we like them to this point, the third interview is with managers from other functions. How well does the candidate communicate, come across to non-developers, express interest in the company and role, etc. It's a check point to look for concerning weaknesses, as well as get buy in from the broader organization.
We like this approach because it allows the candidate to code in a much more natural environment. They can take the time they need and comfortably solve. No one spends their professional career coding on a white board. This approach so far has yielded excellent results. We ask developers how much time they took in the programming task and why they chose the one they solved. The programming task is telling, not in the quality of their code and how long they took, but how much did they get into it? We have had the range from candidates who did not complete it at all and opted out, to candidates who stumped 40 year veterans with elegant code. In every case, we learn how well they can express their thoughts, and importantly, their level of love for the discipline. This is as important to me as any other attribute.
Plus, if your company gives someone what they might think are ridiculous questions or they realize it's just not what they're good at, they can easily reject proceeding any farther with the interview. If someone walks into a crappy whiteboard interview, they feel kind of stuck. Seems to be better for both sides. Less pressure.
But any time take-home tasks come up on HN, there are always people here complaining about it and even demanding to be paid for their time.
I get that it doesn't suit everyone, but nothing does. Coming up with an approach that works for the company and candidates is really hard - as evidenced by the many, many threads on this subject here on HN.
Perhaps the best compromise would perhaps be to permit the candidate to perform the task there and then or take it home, but this may require more of a commitment from the interviewer, if they then need to additionally devise tasks to be completed in a shorter time.
I somehow feel this rewards opinionated developers.
Also right now I'm looking at a couple places because their recruiters called me up and said they were willing to meet my requirements which are basically pay - I don't care about most of these things, if they want me to do stuff and are willing to pay my rate and if they, after looking over my cv believe I can handle their stack, that's it. I don't really have any questions for them.
So I guess you don't have people looking for people for you - you only have people coming in that think your company is special enough that they really really want to be there and are invested about finding out everything just to confirm that commitment on their part is good?
on edit: this of course supposes that 90% of the companies out there are very similar and do not have any problems that are so interesting that I should feel prompted to ask. If I ever applied to the Internet Archive I'm sure I would have things I would like to ask.
If I hire a developer "to build X," I would rather work with the developer who explains that X is the wrong thing and we should actually build Y, than the one who shrugs, asks no more questions, and builds a really excellent X - that later turns out to be the wrong thing to have built.
I don't want code that stumps anyone. I want code that delights 40-year veterans, for sure. But the signal of genuine elegance is "oh, that's a lovely way to do it" much more often than it's "how the hell does this work?".
That’s not to say I don’t think this process has promise, but I’d need to be convinced that the above two issues can be fairly dealt with first.
It sucks, but you only need to be better perceived than the next best competitor. Most folks don't have 40 hours to spend on a 2 hour max coding challenge. Hopefully the ones that do are already employed and not currently competing with you, or are so inexperienced that 40 hours produces horrific over-engineered abominations instead of clean efficient readable code.
If I want this job, I would spend this extra time to submit this better solution because obviously that will improve my chances of getting the job. But that means that these take home assignments might not be such a good indicator of efficient proficiency. You might be overvaluing people spending more time on these tasks and undervaluing people spending the allotted time on these tasks. For example, a better developer who did write a solid solution in under 2 hours might get passed over because her solution seems not as good as mine where I did spend another hour to rewrite my initial messy solution.
The goal is as much to learn how they think, solve problems and communicate as it is whether they know how to write software.
I'll give one example: One task is to figure out how long it takes to grab the current time from the operating system. We use a rather obscure OS. A candidate chose this and solved it simply (though it's got a couple of points to mind), but then he got curious and wanted to know whether a virtual machine had a different timing and then what the performance on a different operating system was. He told us that he spent about 8 hours altogether. I felt a bit bad on one hand, but on the other, it's exactly what we like to see: Solid programmers who love programming technology for its own sake.
I think it's common for an on-site interview to include a reasonable chunk of programming, and using a continuation of the take-home to do that is good because it means the candidate isn't starting from scratch. Coming in warmed up should minimise noise from nerves, unfamiliarity with tools or context, etc.
I recently had a humiliating experience. Just was approached on LinkedIn and at first they sent me a take home. I learned the front-end tech which was required for it, and then coded it. They just wanted me to fill out a doc but I put it all up on stackblitz. Another portion was back-end which I also put up on another online env so they could see not only the code but how everything was laid out.
Get to the over the screen share interview - just nervous and both the interviewers were impatient, and just the atmosphere was so nerve-wracking - they did nothing to make it more relaxed. Just seemed like the senior guy was stressed out and I was wasting his time. My brain kind of shut off and the seemingly simple problem at that time just seemed insurmountable because I felt myself second guessing my every thought.
Been developing for a number of years for a few companies, and I guess I haven't prepped for the tech interview - I thought my existing skills are what they need and should be enough to evaluate but boy was I wrong.
Needless to say they didn't go with me. It's not like I needed the job, or probably wouldn't have taken it anyway since what their maximum range was lower than what I make, but the startup seemed quite interesting. Just such a bad experience.
If they're not placating you at all than I doubt that'd be a great environment to work in. Or they'd already selected someone.
I think if I was interviewing using your process I would submit code because I would guaranteed a chance to talk about it with other developers.
Temporarily only
I guess the tasks are something like "take this scaffolding and implement thing x". What would you think if the candidate responded saying the scaffolding does not allow to implement X properly without sacrificing things y and z, and instead of turning in code for review would show up for a discussion about the compromises and deficiencies in the scaffolding?
Responded how, when? (At what point in the process)
> > instead of turning in code for review
I.e. something like instead of turning in code example they turn in a motivated analysis of why the scaffolding provided cannot support "proper" (and then analysis of so called proper) implementation of the thing asked, provide analysis for changes required in the scaffolding.
Basically, the candidate would turn "provide and review code" process into "let's talk about architecture, implementation and tradeoffs". Would you discard such a candidate for not following the process because you expect seniority or would higher level thinking be a plus?
Whilst the challenges were not particularly complex, you also had to ensure that the person who was going to run your code could stand up your environment, so part of the second challenge was "thoroughly document how to run your code, the environment needed, etc, etc."
Life is too short to deal with this kind of stonewalling.
When I set it up the goal was to avoid a lot of the issues I've personally had or heard about with coding challenges: they take way too long, they're too abstract, or you feel like the company might just be using you to get a couple of days of free work on their actual codebase. For this reason I feel fine about not paying people, as it's just a small part of the application process and not free work for us.
I think it's worked pretty well so far. It happens after the initial phone screen and before the first technical interview. We send people the challenge, they return it whenever they want, and if we like it we set up a technical interview. Since the challenge uses our tech stack and is similar to the work we actually do, a large part of the first technical interview is discussing their solution. Why they chose certain patterns, why they added a certain library, how they'd consider testing it, why a certain function might be slow with 10k entities, and so on.
You don't normally pay candidates for the time they spend doing the whiteboard interview, do you? You can view the take-home task as a replacement of the whiteboarding session, equally with no pecuniary obligations for the candidate.
Then what? Would it feel good to pay
He asked me vague question, like what is architecture, I started to reply with what design tradeoffs I made on the app I was working on. He literally laughed at me, and said that is not architecture. I was shocked that someone would laugh at a candidate but said ok, what does architecture mean to you? He responded, I ask the questions here not you. I immediately ended the interview. Worst interview I have ever had. Every time I see that company brought up, I think back to that experience.
At some point the interviewer asks which I would choose in a project, C or C++ and reasons why. Basically answered that it depends on the project, but that both languages had their pros and cons, and it all depends. I told him I did not that many projects under my belt at the time.
Interviewer got somewhat pissed off and kept asking me about specific languages constructs which I was not yet familiar with. In a way felt like he was trying to ambush me and make me feel incompetent. I mentioned to him that I was not yet experienced enough to answer these questions, but was eager to learn. Well he did NOT like that either and ended the interview promptly.
It seems like the guy really did not like me (which is fine), but also wanted to turn this interview into some type of power trip ....
Had this been today, I would have politely declined and left. But I was a young and eager cub. Live and learn!
Sounds like a good judgement thing to reply :)
Too many times we read someone's experience, which even if only half true, would be horrifying, yet no indication of who the company was.
Similar response.
Nobody has ever asked me how often we need to build calendaring software, because I explain right up front that while this is a “toy” question, our core product functionality schedules people, and nearly every feature from a calendar app has some analogue to things we either do, or are asked to do but haven’t prioritized yet.
I think it’s ok to ask “toy” questions, but I also think that there should be a ready answer to the question “Does this have anything at all to do with the job?”
p.s. We don’t ask a question directly about scheduling, for a simple reason: Almost everybody understands the basic idea of a calendar, so it’s a more “level playing field” for candidates to think about calendars than schedules.
Again, thanks so much for naming the guilty party (NoRedInk) in this case. So that we can firmly add them to our list of companies not to bother engaging with, if this is the kind of behavior we can expect from their interviewing team.
The above anecdotes speaks volumes about the mentality that drives companies like these -- and the "culture fit" they're apparently looking for.
I actually had the pleasure of interviewing at one of these companies before. They had a take-home project, which was to choose and implement and couple enhancements to a toy app. In the subsequent conversations, we discussed how I approached the problem, details of my design, technical tradeoffs, etc -- all the sorts of things you would expect a professional sw engineer to be able to do on the job. It was challenging but fair, and I was struck by how reasonable the process was. It left a wonderful impression on me.
Take-home projects are not a perfect solution to the interview problem. A major issue is that they require candidates to have free time outside of work to complete them. Personally, though, I'd much rather spend a few hours per application working on projects than countless hours prepping for stressful whiteboarding puzzles. As a developer who prefers to digest problems slowly, I look forward to the demise of on-the-spot whiteboarding interviews.
If you work at a company that subjects candidates to such high-stress algorithmic riddles, and you have the ability to make your process more sane, please consider following the lead of cos on this list.
Also many companies will insist on a take home project in addition to everything else, including a whiteboard interview. So you can pour time and effort into this project only to be eliminated further down the line (or just ghosted).
It's like they assume applicants are applying to one job at a time and have no other work or life commitments.
I dunno, it feels like the tech industry has forgotten how to interview candidates, as in "just chat with someone like a normal human being to see if they are a good fit for what you need", and like many other aspects of the profession have fallen for dumb processes, fads, cargo-culting, outsourcing to third parties, and dubious technical solutions in lieu of experience and social interaction. I learned this early in my career, and found I can pretty much size up someone's technical chops - assuming they're in my wheelhouse - and get a good idea of their professionalism - with a short, half hour phone/Zoom interview. Yes, I can see how this might lend itself to bias and prejudice, but it seems the solution there is to train interviewers to overcome biases and have more diversity (in all senses of the word) in your interviewing team.
I think the best approach would be giving the candidate a broken app and have they debug it live or in a take-home debug-and-fix or debug-and-improve exercise. Could also have some refactor assignment.
Has any company tried something like this before?
They lower the cost for the company to zero. So they can widen their pool, and take a chance on more candidates, with no extra cost incurred for them. If you take this to the extreme – every company asking every candidate to complete a take home – then the result is far worse for candidates.
There is no feedback mechanism for candidates during the process. I can say "you shouldn't spend more than two hours on this" until I'm blue in the face, but candidates think they need to polish far beyond what is reasonable. On occasions where we had a system for capturing progress, I found that most candidates far exceeded the time recommended to them. Some were spending up to six hours. The frustrating thing was the extra time never made a difference to the outcome. I could tell from their initial hour whether they were going to pass.
As a hiring manager I've used them, but work hard to be considerate to the candidate. I'd prefer to design a better in person interview than take the easy route that is disrespectful of candidates' time.
Its a one-hour hands-on programing lab via screenshare - a pretty straightforward task of consuming some data from an API, processing it a little, and displaying it. We use Ruby for our work, and ask the same of our candidates in the lab. No handwritten syntax, exotic/tricky algorithms, or gotchas. Just a task to see how a candidate approaches a problem, deals with requirements, and implements them in our language of choice.
Happy to go into more detail if anyone is trying to implement something similar in their own hiring processes.
I really enjoyed this session, but I can see how it could be stressful to others.
I had a company do this and it went really poorly for me. It was a python app and at the time I wasn't very expert in python. They told me to use my laptop and all I had for python was a vim environment. I knew enough python to do white-boarding problems. But debugging a complex app exercises a whole different set of skills I just hadn't developed in the language. I was also learning vim on the side. I looked like an incompetent amateur.
So there's a language and tooling issue. The bigger problem is that debugging an unfamiliar code base under time pressure is almost as niche as doing algorithm problems under time pressure. I've been on enough company wide on calls that I actually think I'm pretty good at debugging unfamiliar code under time pressure. But if I have someone who understands the code looking over my shoulder, they're usually at least trying to help.
It's still time pressure. It's still a somewhat contrived problem and code base. The interviewer is still comfortably omniscient. And it introduces some more complexity due to unfamiliarity of the language, tooling, code base or problem space.
Ultimately, I'm not sure it's any better than white-boarding.
A lot of these problems are solved with a take home question but then all the problems with take homes rise in their place.
Obviously no one is going to submit an app that doesn't accomplish the primary task. I feel like you get judged more-so on how you decided to get the initial app up and running, which likely isn't something you'll need to do working there.
Our test is actually quite simple and everyone that did any kind of development should have done these basic things.
But it tells me a log of the current state of the candidate. Most code samples on internet are not production ready code. Security and error handling are mostly not included. So this is where a candidate can shine... but I'm happy if you can string it together working even if it is by googling.
https://www.quora.com/What-is-it-like-having-a-practical-int...
Last year, I did take home test for 5 different companies and all 5 of them invited me to the next round. I yet to have a successful whiteboarding interview.
Just like onsite interviews. Any interview process will require candidates to spend time outside of work. The only way to fix that would be a true, universally recognized certification/diploma that would prove that you can write software, but I don't see that happening any time soon.
And unlike an onsite interview, I don't have to take time off during working hours to do them.
My mental model is that I'm willing to spend N hours in total for the job application process. I'm happy to spend some of those hours working on a take home problem as long as that is instead of and not in addition to spending those hours doing multiple rounds of interviews.
A good middle ground would be to have the company make a donation to the candidate's choice of nonprofit.
I think this is a key piece why this kind of project is useful, because that's exactly what they'd do when they start in their coding role: jump into an unfamiliar code base and fix a bug, implement something tiny, make things better along the way.
When hiring for a React role, I went to find an open-source React app and simply asked the candidate to mod it a bit. We then discussed the why and how, but also how they liked the app and why.
We only gave this task to the final candidates, and the ones that had React experience were done in very little time, because typing wasn't the issue here, but reading unfamiliar code and understanding the mechanics of the app.
Writing a hello world app from scratch or coding fizz buzz doesn't tell me enough.
I mostly understand why some companies do the x hours routine, and when I've done interviews this way I've historically performed well, but it just felt like it added unnecessary stress. For instance, anyone on the job doesn't have to worry about issues like, "What if I couldn't get the app up and running because I had the wrong version of X installed?" longer than maybe their first day on the job. And if you plan on hiring an engineer for a few years, one day is irrelevant. At best this felt like it favored contractors who were more experienced at jumping into new projects frequently.
Take homes are not very respectful of the candidate's time and, especially if they are not time bound, often leads to a candidate wasting a significant amount of time on these assignments. Plus, there's always the possibility that desperate candidates would cheat and have the assignment done by someone else.
I would only use take homes and HackerRank style quiz for applicants that are from institutions I don't fully trust with quality or applications that are very spammy in nature, i.e from a college/bootcamp that has an abysmal application-to-hire ratio [0].
For candidates from serious institutions I favor a two-step approach: a screening interview with a light coding question and a full whiteboard interview. The screening's purpose is really only to answer questions about the hiring process and check that the candidate can write very simple code by himself when given a trivial problem [1].
Now, a lot of folks do whiteboard interviews wrong. They often expect to get the exact implementation of an algorithm they found in a textbook and for code on the board to compile. This isn't the point of whiteboarding. Doing this only promotes rote memorization. A good whiteboard interview should be a toy problem that can be solved in several different ways by using different strategies or data-structures. The idea is to see how the candidate will break down the problem. Is the candidate able to formulate test cases, write a simple implementation, verify his code and correct the implementation should it fail a test? On the more meta side of things is the candidate able to take feedback and explain why a certain strategy was chosen? Of course it's not representative of real world engineering but it's a good way to peek at someone's ability to debug and reason about programs; these abilities translate well into debugging and design. Especially at the college level, I really can’t make any assumptions on what the candidates know. I’m not judging their knowledge of the standard library of X programming language or the framework-du-jour but their ability to learn it fast.
[0] https://economictimes.indiatimes.com/tech/ites/95-engineers-...
[1] https://blog.codinghorror.com/why-cant-programmers-program/
> I'm actually a little scared to leave my current FAANG gig for that exact reason tbh. I'm fairly certain I wouldn't make it back in the door without more leetcode grinding + repeated loops than I'm willing to do at this point in my career.
>> I'm in a similar position - a tech lead role at a Big N - and have recently been doing some interviews for senior roles at FAANG. The most frustrating thing is knowing I can't really leverage anything I've learned over the past 7 years of my career in the interview, at least at the early stages.
>>> I feel extremely little of what I'm doing in my real, actual, job, helps me in advancing in my career. Unless maybe if I choose to stay at my current company until retirement lol. Otherwise, why bother doing anything more than the bare minimum to get by at work? It would be a far better investment of time and effort to grind leetcode and practice for interviews, instead of going above and beyond to excel at my job. At least until I get into an "endgame company" where I feel it's worth staying long term.
https://news.ycombinator.com/item?id=23848717
Even as we debate the validity of the metrics measured by technical interviews, it's also worth considering the metrics used to measure engineer worth in the industry at large. This existentialist "fear and loathing in FAANG" discussion was quite illuminating, and a little sad.
Makes you question if this is the precise point of it.
Take home project? There goes 4-12 hours per company that gives you one and sometimes more. Companies have no incentive to cut it down or not give it to even marginal candidates so you'll get a lot more of them than full in-person interviews.
Pair programming? That's basically white boarding with a better white board or, in the remote interview world, just regular white boarding. Sure the problem is more real world, in theory, but then you get issues of knowing the same frameworks, etc. So to do well you need to spend a lot of time beforehand studying their specific frameworks and code bases if they ask you to do an issue on their open source project.
It is very easy to completely forget all about graph algorithms in, say, 3 - 5 years during which time you've been a productive, valuable member of a software development team that...you know...actually builds things customers care about.
Being a productive, valuable member of a software development team is teaching you things that will set you up for success in higher level roles. You won’t be taken for those roles on Leetcode interview performance alone.
I mean, maybe hard or esoteric easy questions but I don't think that's true for basic things like 'find the first duplicate' or '-traverse- this graph'. When people rail on whiteboard coding, I think they mean obscure algorithmic questions and not fundamental data structure questions.
I don't expect someone to be able to do Dijkstra's algorithm (or even spell Dijkstra, did I do it right?) but everyone should be comfortable enough with a bit of help doing BFS or working with linked lists, hashmaps, no? Am I crazy here?
Eventually I changed the strategy and offered candidates the choice of the take-home problem or an on-site whiteboard style interview where they solved a subset of the problem in pair-programming fashion with us. Zero people chose the on-site interview. Everyone chose the take-home problem.
No one complained any more after we reframed the take-home problem as the candidate's choice rather than the company's.
Ironically, you passed on the smartest applicant of the bunch.
> reframed
Aren't meaningless shenanigans generally associated with meaningless things?
Do you have any evidence that either strategy really matters? Does it boil down to, "We choose an interviewing strategy that selects for people who look like programmers."
Are you meaning they wanted your company to pay for the time used to solve the take-home problem?
If so, isn't it normally done that way in order to prevent companies potentially exploiting free labour?
Do you offer candidates a remote pair-programming interview?
E.g. screen-sharing session over Skype/Zoom?
If i'm really serious about joining a company and the many months/years to come with them, I can probably spare a day to show it.
It's always surprising to read people complaining about spending even 8 hours on a take-home problem in the comfort of their home, with no one looking over their shoulder, finished at their leisure. Meanwhile, anyone serious about applying to the same company wouldn't hesitate to take PTO and leave work to go on-site for a similar amount of time for an interview.
I personally when I interview candidates prefer to get to know them and how they think before I even try to understand their ability to problem solve. Knowing the language or algorithms are all easy to learn things. Knowing how to naturally be curious and solve problems is a far more difficult skill to learn and demonstrate, but it is easy to evaluate indirectly.
That's what leetcode used to test and then people optimized for it and now it tests memorization. At scale things work differently than in isolated cases. I've also found that most problem solving interviews really test how well I've seen the problem before and how well I can lie about seeing it before. Interview enough candidates and the good problem solvers will look dumb compared to the ones who simply saw it before and can act like they didn't.
We place the candidate in a room with a large whiteboard for 15 minutes alone with the lights off. I then walk in with a 6 piece dresser [1] from IKEA (obviously keeping the lights out). I explain to the prospect that this particular dresser is missing one piece - BUT we need the dresser assembled then disassembled and placed back into the box [without anyone seeing]. More than HALF the candidates become violent and demand to see my github account - some even suggest I have a mental issue. I am often able to calm the candidate down - this way we can get into the REAL debate: KOPPANG vs MALMs. Any candidate caught assembling the IKEA dresser is quickly dismissed out of spite. This technique is from Boz at facebook - it is how all* senior principle engineers are screened at FB and the reason everyone codes PHP (but swears they do not).
HR has complained that my techinques are "ridiculous" or borderline illegal - but so far my successful candidates perform, tend to have a solid grasp of typescript and quickly move into senior management.
https://www.ikea.com/us/en/p/koppang-6-drawer-dresser-black-...
People want hiring processes to be somewhat consistent, and in that world it's really kind of a non-starter to replace tech screens with GitHub profile reviews.
for that matter, GitHub is useless if the projects you contribute to are on BitBucket or GitLab. that contributions graph has a lot of assumptions baked in.
Maybe the solution is for companies to hire more thorough managers who are capable of vetting a github profile to see if the candidates skills are relevant to the position, not turn the interview process into a performance.
So anything on github that I might want you to look at? I’d have to be on a three year replacement cycle, permanently, which is a hell of a lot more work than you might estimate that a reasonable github presence would entail.
As an introverted and generally very socially anxious developer, this sounds like a dream compared to a standard interview, which has always felt to me more like an interrogation and generally very aggressive and hostile. I'm confident in my skills and I can communicate well in other situations, but the typical interview environment is cripplingly stressful to me. I would love to just get through a talkative lunch instead.
Going through my GitHub account however isn't good. It's full of quick one shot things, experiments and so on, very little production ready code and not representative of anything.
That kind of interview mitigates the amount of trivia and preparation-gaming while also ensuring that the applicant can solve problems as a team and is a competent programmer. It is cheatable in that one can memorize implementations of some features, though it seems somewhat difficult and not more cheatable than just memorizing a bunch of CS trivia.
I think part of the problem is not everyone is good at giving interviews. But most software engineers can talk about their work once you get them going.
FWIW, I think white boarding interviews are pretty fun.
Ask questions about the code:
- why did you write this in this way
- explain how this works
- if you needed to add X into this, how would you do it
- are there other ways to achieve what you did here, what are the pros/cons of this?
- what would you do differently if you were to re-write this?
- are there any bugs or common issues, how do you go about diagnosing and debugging them? Could you write the code differently to be more resiliant to those issues?
Anyone who wrote that code (assuming it was relatively recently) is going to be able to talk about it sensibly, and be able to dig into the code and show how certain things are done.
What’s your threat model here? People making up elaborate lies about code they’ve written?
It obviously depends on your definition of "better". And it's yet to be seen whether whiteboarding is "better" than pulling candidates out of a hat. You'll note that lots of companies that do strict whiteboard-style interviews are also extremely comfortable firing employees. Would you be "better" just hiring anyone whose resume seemed applicable, that could talk like a normal person, and then firing them if it didn't work out?
Do you have any statistically significant data to back this claim?
That whiteboard interviews are actually testing how you perform under stress rather than how good you are at your job.
Additionally, in my opinion, it measures how good the interviewer is. If you get a bad interviewer or even just one in a bad mood, just pack your bags because you aren't getting it.
Whiteboards are incredibly arbitrary in my opinion, if taken alone. If it is part of a complete assessment, ehhhh.
if properly trained, interviewers ask questions that build without throwing the kitchen sink at you.
whiteboard interviews (when done holistically) assess for signal, not binary correctness
- do you have a solid framework to build a solution?
- are you considering multiple approaches / data structures / complexities
- referencing similar problems, vocabulary, situations to illustrate breadth of knowledge
sure, the stress component isn't ideal, but there are multiple relevant capabilities being assessed in a very short amount of time...
in other words it's a much better test of tacit knowledge than alternatives
WB interviews should be about problem solving not regurgitation of rote learned algorithms
I won't ever fault a candidate for failing to close a paren or being unsure of whether the size of a container is .length() or .size(), but if virtually every line a candidate writes is of questionable value, I'd say that means something.
One of my earlier jobs dealing with code involved a very brief interview (no whiteboard involved) followed by a 6 month contract offer. The deal was that I basically had 6 months to prove myself, and if everyone felt like it was working out (myself included) this would be bumped into full-time with benefits. The crazy thing is, I was actually the one who decided I did not want to continue after the 6 months was out while they wanted me to stay very badly. I think this can be a really useful checkpoint tool for both sides.
Anyone can misrepresent themselves in an interview and dupe a room full of people for a day or 2. No one can keep up an act for 3-6 months continuously when working with complex code tasks. You will get found out before the term is up, assuming everyone is using this process as an evaluation tool and following up on it regularly.
It also puts a candidate at a disadvantage, because looking for a job is heavily one-sided burden.
On a personal level, as both (occasionally) a candidate and a senior hiring manager, I always wonder about white boarding - i believe it works when the interviewer is looking for your thought process, and not code specifically. For example, something as simple as saying “I don’t expect this code to compile” at beginning of interview immediately puts candidate at ease and let’s you understand their logic and not the ability to recall specific function signatures on the fly.
In the UK, we sort of have this in the form of probation - in most permanent jobs, you don't gain full employment rights until you've done a probationary period, which is usually six months. You get full salary and benefits during that period, but you can be fired with no notice at any time (or something like that).
In practice, though, the culture is not to use the probationary period as an extended interview, but more of a backstop against really screwing up hiring. You'd only let someone go after probation if they turned out to be absolutely incompetent or unsuitable, not if they were just disappointing.
It would be interesting to try using probation differently, though. Make the interview more of a screen, employ twice as many people as you need, only retain half of them at the end of probation, and be explicit throughout that this is the plan. I suspect you could implement and communicate this such that it wasn't absolutely horrible for probationers - active and transparent review during probation, one or two months' golden parachute, what else?
Even considering the flaws, I think this approach is better than immediately hiring someone full time. I've seen some candidates who seemed perfect on paper and fine in interviews and then turned out to be utterly toxic to actually work with. For the interviewee, the chances of encountering that situation are even greater, since there is one interviewee but typically many future coworkers.
Some personality traits can be very difficult to surface in an interview environment. IMO interviews are worthless for getting that kind of information.
But these positions are overwhelmingly for people breaking into the field. They have much to gain, and very little to lose by working for a company for three months with an uncertain expectation of a full-time job.
Now consider hiring an experienced worker who is out of a job. If Alice offers them a contract with the company having an option to turn it into full-time, while Bob offers them full-time work with the understanding that a major screw-up would lead to dismissal during a probationary period, Bob’s offer is much lower risk.
Finally, consider an experienced worker with a job. For a non-hypothetical example, they’re working at a famous name, with options, and a YC startup that has grown to 200 employEes recruits them.
It’s going to be very difficult to get them to walk away from guaranteed employment that feeds their children, for a temporary gig with no equity participation And the understanding that it’s an extended job interview.
—-
The above focuses on the candidate’s risk/reward calculation. What about the company?
Well, the least experienced person has the greatest uncertainty. They might be very smart, but unable to work well with people. How would we know without giving them temporary work with people?
The most experienced person has the least overall uncertainty. They “brought the receipts.” Without arguing about how they should be interviewed/tested, if they aren’t a completely toxic person, if they have a non-fatal weakness that pops up, it can probably be managed to be fixed, or accommodated.
The employer’s risk goes down as the candidate’s experience goes up. Sure, it would be a bed of roses if the company could hire senior people on a one-year contract and see how all the intangibles work out, but that usually isn’t necessary, and it’s not going to happen often enough to operationalize that as part of the process.
—-
TL;DR:
There is no one-size-fits-all approach that fits all levels of experience, and also scales to large numbers of hires, and also is repeatable.
But the kernel of what you suggest works well for the sweet-spot of high-uncertainty and low risk to the candidate.
Unfortunately, most companies fail to honestly estimate the cost of the tasks they assign for these processes. Many of them because they just don't take into account numerous small tasks that the candidate will need to deal with.
I've seen a lot of these tasks ask for setup instructions, good unit tests, explaining any decisions you make, etc. In case there's any visual user interface (e.g. some web component) they also ask for it to be nice, as in "we don't want a designer but we need you to make it somewhat nice". And then they'll say it should take just a couple of hours.
I've also seen a fair number of others casually say their task should only take about 8 hours. And I can do that on my free time on five consecutive days, after having worked already 8 hours each day. Sure thing.
After that I just decided I'm not going to put the time and effort in that sh*t anymore. I'm not even going to go out of my way to accommodate interviews in my working hours.
Next time it happened I replied back that I'm not interested. For the record, they wanted a silly automation in AWS and kubernetes. Most of the time would go to setup everything up as my future-employer assumed that I had an AWS account and that AWS is the only cloud that matters. (I work on a different provider daily).
Both times it was automotive-related IT. Dunno if that's something to do with it.
My take-away so far: Whiteboard sucks but at least it respects your time while the take-home proj does not necessarily keeps you from wb and is usually ridiculous to a large extent.
When I interviewed people and gave them a take home assignment I was very confident in my estimate for one simple reason: I completed the tasks myself. We had 4 tasks for this for the 4 different types of jobs we hired for. The "estimate" was just double the time I needed to do the task myself. We got feedback from some of the candidates that our estimate was very good.
If I could always estimate the time AFTER I complete the task instead of before then estimation would not be hard at all.
If you don't mind me asking, how big were those tasks?
Coding on a whiteboard in an interview is not "broken". What is "broken" however is making the problem the candidate needs to solve "hard". I can't stress this enough: a whiteboard coding test is nothing more than a negative filter. It's to filter out people who can't turn a simple idea into code as these people exist.
This is why FizzBuzz was such a simple problem.
But interviewers fall into the trap of thinking "this problem is too easy" and make it hard (eg figuring out a problem is g union-find on the spot and then implementing it) or, worse, they make it a crap shoot of whether you know the "trick" or not (eg reversing bits in O(log n) or the tortoise and the hare). This actually makes the test completely useless.
A negative filter is simply a filter whereby if you fail it, you almost certainly aren't a great hire but the reverse doesn't apply: there should be no A+ on a whiteboard coding test. It's straight pass/fail and an easy pass at that.
I realize some people have anxiety about doing this. Having helped or coached a bunch of people in the past about this, I can say that mock interviews and practice help the vast majority of these people. Some may be helped by doing this on a laptop in a shared doc instead, which is fine.
A few will struggle with the anxiety of that, even with coaching and practice. While I feel bad for those people, I do wonder if a reasonable-sized organization, which will generally require a certain level of communication skills and presentation ability, is the right place for you.
When I interview people, I give pretty straightforward questions. My goal is not to run some algorithm guessing game. If I'm interviewing for a senior FE position, I make sure they know enough about async patterns in JavaScript that I feel confident that they could build an event handler without dropping promises on the floor.
A back end position? Can they use (not implement, just use) a hashmap efficiently and do they think about boundary conditions in input sets (you'd be surprised how many people ignore punctuation when breaking sentences into words - they just do a split(' ') and call it a day.
This is really basic stuff. I'm always able to (a) filter out people who just need more time learning to code (or are too nervous to perform well, which is still a problem with even these whiteboard activities) and (b) prompt a discussion about engineering practices.
My criterion is simple: do I think I could see this person submitting code to my project without me needing to constantly fix what they write? It isn't about memorizing algos or anything.
Aaalso: I didn't study leetcode, and managed to get my job just fine. I know there's a reputation of the process being just algos, and I'm sure a lot of the interviews are. But not all of them, even at the FAANG.
Id take a look :-)
Yeah, I tanked the interview but I really did not care if I pass when I got to that point.
how would you have liked that?
It's really not about presentation ability. It's "oh shit I have x minutes to do this and I have to get it right or I won't get the job, oh shit 5 minutes passed already and I haven't done anything" type of anxiety.
"Oh shit, I have to get up and talk about my project. What am I going to say? What if my director thinks I'm an idiot? What if they ask me a hard question? What if I can't answer the question? Will they cancel my project? Will they bring in someone to rescue me? Will I get fired?"
The only thing that LeetCode questions predict is if the candidate has been training for LeetCode questions.
The fact that Google, Facebook etc still use it as a gatekeeping mechanism makes me believe they want to find cogs that will specifically spend hours studying for it. Making sure that those engineers will be ready to execute their mindless coding tasks at the company.
You missed the whole point of the OP's post it seems. The only thing that leetcode implies is that you can solve leetcode tasks.
I think people underestimate human nature in this case. It's a known thing that many interviewers just do the interviews for the sake of feeling superior than someone else.
It was another trend in the industry to ask stupid brainteasers like "Estimate how many gas stations there are in Manhattan" until Google banned the whole thing altogether. Google judged that these questions only serve to make the interviewer feel clever and self-satisfied. [0]
It's time to do the same with whiteboard interviews.
I've only gone to four interviews (across eight years), and been lucky to get an offer each time (three of which I've accepted). They have all been primarily discussions about personal experience, hobby projects, personal interests, would-be responsibilities, benefits, and company culture.
At one place (Ghost Games, an EA studio and the only company which wasn't purely Swedish) I had to complete a code assignment (create a clone of a classic arcade game in C++) ahead of the interview.
Only ever encountered WB questions at one place, and it was a FAANG-type American company with a satellite office here.
Everyone else were just normal interviews, with mostly behavioral questions, and some softball technical questions.
(Nothing against whiteboards or laptops or paint brushes, just wondering)
What type of job were they applying for?
I'm not including a compiler or having build tools or an IDE. Just a basic simple text editing area that allows the basic functions of typing in text and editing it.
I'm against these leetcode interviews. But if you did nothing else but change this one thing, just stop expecting people to write code on a whiteboard or paper/pencil and allow them to write code the way it's actually done (a computing device with a keyboard and text edit area), that would be such a huge improvement. Writing code on a whiteboard or paper doesn't test anything. Think about how limited it is and how different it is (for example, you can't just press an up arrow and add a newline, have to find an eraser and start over).
It's yet another useless skill to learn just for interviews (writing compilable code on a whiteboard/paper) which also encourages rote memorization (because you have to get it right on the first try since editing the text is so painful and difficult).
I hope people start pushing back on this. The reaction should be, wow you care so little about your interview process that you can't get a $200 chromebook in here?
Maybe it's just our phone interviews that allow us to focus on higher-level concepts in person, but I've really never found a candidate who did poorly on a whiteboard but seems like they would have done well on a computer. If anything, it's the inverse where candidates jump straight into code, to their own detriment, when placed in front of a keyboard.
Now while you may be aware of this and not give the candidate a hard time for not leaving enough space, I read a Medium post once written by a Google employee with tips on doing well on the interview. He explicitly highlighted this problem, and literally encouraged candidates to learn to think and write linearly and not leave these kinds of blank spaces.
* IT just fails: they’re unwilling to supply it, there aren’t enough, the current laptops on hand are junk, or the default password is wrong. These all happen with HR buy-in and funding.
* Recruiting just fails: they “have to move the candidate fast because of a competing offer,” they tried to work with IT but IT blocked them, they got laptops themself using their own budget but then ran out.
* I failed: I didn’t make time to prepare a non-whiteboard question, I didn’t tell my manager in each and every 1:1 that candidate experience sucks, I had three laptops and I forgot to bring my personal spare to the interview.
In general the core problem is there is zero incentive for hiring managers to do anything but shotgun candidate pools like they do today. There’s also no feedback loop to evaluators to help them improve. False negatives are completely tolerated despite there being a huge gap between CS jobs demand and CS graduate supply.
All these companies expect you to write code using a site like hackerrank during the phone screen. And yet they can't have a chromebook open to the exact same site during the onsite? If that's too much, they can't even get a cheap laptop, stick in a USB drive linux distro, open nano and give that during the onsite?
I think this is the bad assumption that's causing your confusion. Maybe this happens, but I think it's very uncommon.
The point of using whiteboards is to focus more on algorithms, data structures, and problem solving, not just spitting out code. If you make the applicant write actual code it limits the scope of the questions that can be asked.
So, it's actually beneficial that whiteboards aren't good for writing compilable code, because that's not what you want the applicant to do.
However, a whiteboard certainly can be easier to write anything that is not code on. Like for instance if the candidate chooses to walk the interviewer through a high-level approach, including a graph topology, before actually starting to code.
I know Google started letting candidates choose between laptop and whiteboard a few years back. I suspect others alos have.
Other questions I ask are things like do you work on your own projects and tell me a bit about the tech behind them, what’s your favorite pieces of open source or free software (I work with almost exclusively FOSS stuff though my own work is not FOSS), what do you do for fun, and if we were talking on the phone and I had never made tea, walk me through the process (this is a fun one). Then take them out to lunch if possible and watch how they treat service workers. I think this has so far worked well for everyone involved. At least as far as I know nobody I says yay to has been a bad hire.
I mean if you literally can’t remember anything of what you’ve worked on ever, no I don’t think you’d be a good hire. And also this part is surprisingly hard to bullshit. You can say “well I worked on an eCommerce site and it was hard because we had a lot of visitors” isn’t a full answer. If that’s as much as you can give me, no I don’t believe you contributed much to that project. Maybe I’m wrong and you are brilliant and just can’t remember, but it’s a risk of false negatives that I think most employers will find acceptable. But: “It was hard because we had a very complicated system for determining the final price of the whole cart because of discounts offered if you bought certain pairs of products together. I didn’t implement that system and we had plans to eventually rewrite it but in the meantime we started calculating the prices in our task system instead of during the web requests. It reduced web server load and made adding things to the cart feel faster.” Me: “OK that sounds neat but how did you prevent race conditions? Say while the first calculation was running the customer added a second item that triggered another calculation? What if the first task finished after the second?” Then: “Ah yes I recognized that it would be a problem when writing that code. At first I tried using a lock using Redis for the cart but that just made the tasks serial and the whole thing took longer. Plus if a task failed or was interrupted it could cause issues. So instead I just put the latest task ID onto the cart object in Postgres so I always knew which the latest task was. When a task finished and wanted to update the price first it would lock that cart with the Redis lock, then check if the ID still matched its own, then update and unlock.”
The above tells me how they think and what they think of. If I’m not getting enough technical detail out of them I will ask stuff like “what Redis primitive did you use to create the lock?” But usually the interviewee will go into lots of technical detail all on their own.
I think you can learn more useful stuff about a programmer by showing them a function prototype and asking how to black-box test it. Then show them complete code, and ask them to white-box test it. Test cases or a test plan, depending on scale. Then a discussion of test tools they have used.
The people who can produce quality code for your organization will show up pretty fast.
Performative interviews occupy a similar space. You spend all this time and effort studying for something at is ultimately irrelevant outside of the interview. Just a pure waste of brain cells and calories, with a few new gray hairs for you.
Very few people who are successfully employed would take a risk / have a luxury to forsake their certain employment for an extremely short term contract (with all the registration, taxation, regulation etc headache this implies!) and extremely limited chance of gainful employment.
(it's perspectives like these that make me realize how much of a grouchy old man I am and have always been - always being a buzzkill by thinking things through next 3 steps! :P)
It's not if you are in the US.
You will get a 1099 from where you worked, you write it on your 1040 and include the 1099 with your taxes if I recall from when I was an independent contractor for a while.
It takes maybe like 10 minutes extra and 1099-ing is incredibly common nowadays (unfortunately) so not a super weird case.
And it's not regulation. You don't need to worry about anything besides getting the 1099 like you would your W2s and appending it.
What you are saying in effect, is that hiring regimes can't be (or ought not to be) designed to include a group that is (commonly) discriminated against, if that discriminates against anyone, particularly the majority.
Implicitly, I see you defining the optimum as where everyone uses the same criteria that are as inclusive as possible, and just shrugging and saying "we're doing our best" when some are left out in the cold completely.
But isn't there an alternative vision where the optimum is employers using different ("diverse") sets of criteria that individually discriminate but create a society where there's a place for everyone?
source: work at a place that does this
It is a good idea, but nobody is going to be quitting their current job to be a contractor with a job MAYBE attainable. It remains me a lot of retail seasonal employees who work their butts off to MAYBE get a full time position.
I'd also want there to be a clear decision-point (i.e. it isn't a way to get you to be an indefinite contractor, they make a hire/no hire decision within some set timeframe or at the end of the current work).
I interviewed at one of these companies in the list. I'm nobody special, did the test, did the interviews which were more about doing the work and how it gets done and I how I think about it, and they made me a really good offer which I accepted.
I have a hard time believing that this many firms have take-home assignments that really matter, and aren't just another hoop candidates (or, maybe, disfavored candidates) are forced to jump through.
I say this as someone who was an extremely early adopter of take home tests and think they are the highest signal filter you can reasonably have in the US.
So many places have added take homes in the worst possible way without removing all the other nonsense. It’s as if orchestras started demanding high pressure auditions and then threw out the results if the person talked a good game in an interview.
If you’re only hiring candidates who can pass Google interviews, what are you offering that Google doesn’t? (Hint: “we give you a 10 hour take home, then ghost you” is not a competitive advantage.)
I think the first time I saw this was at Google, where the question was pretty much "We are going to design Google Maps from scratch – what does it look like?". The interview is about evaluating the candidate's ability to ask sensible questions when designing a system, checking that they know how to analyse tradeoffs, and understand where they need to think about issues like performance or scaling.
I liked it so much that I've built it into our own recruitment process for our most recent role – it's a process that we go through a lot, even when making changes to existing systems. We do it as a one-hour long collaborative system design process with a short specification for a system that we want to build. We let the candidate lead the design, and help by providing feedback, discussing any ambiguities, and asking questions about specific areas of concern. It's been really interesting to see how different candidates have approached the process.
(Anecdotally, the biggest indicator of a good candidate so far has been the use of abstraction rather than concrete technologies - i.e. "I would have a queueing system here and an object store there" rather than "I would have Kafka here and push to S3 there").
> a system that we want to build. We let the candidate lead the design
They never got upset and thought they were working for free?
If you were to use their designs and ideas, then what if they later claim you've infringed on their IP?
"I can best describe the spirit of what I have in mind by thinking of a music student who writes a concerto by consulting a checklist of the characteristics of the concerto form, being careful to see that all of the canons of the form are observed, but having no flair for the subject, as opposed to someone who just knows roughly what a concerto is like, but has a real feeling for music. The results become obvious upon hearing them. The prescription of technique cannot be a substitute for talent and capability, but that is precisely how we have tried to use technique."
- http://origins.sese.asu.edu/ses405/Additional%20Reading/Fros...
Current technical interview processes with its whiteboards and technical questions will undoubtedly lead to hiring the first type of music student.
The result of working on so many things is that I am not an expert in any of those topics. While I do well in generic system design interviews, I lack depth in literally every single thing that I have worked on. So inspite of having more than a decade of experience, I don't satisfy the criteria for companies who ask for 'x' years of experience in 'y'. So whiteboard and algorithm problems work well for me, because there's not really much to learn and memorize in those topics.
IMHO the worst of these options is coderpad/hackerrank/etc. - it's robotic, discourages pseudocode, and perpetuates the idea that rote memorization is required to pass an interview.
Scrolling through there are a lot of high-quality companies (Auth0, ASOS, Blue Bottle, Basecamp, Buffer, Canonical, CircleCI... and more and more).
Just because Google or Microsoft aren't on the list doesn't mean these are backwoods operations. Those companies don't need to bring in new candidates by being on these lists.
And Elvie! (If you make it to ‘E’)
We’re hiring right now :) https://elvie.workable.com
If you want some one who can build web sites using React JS you are better off hiring people who know and have built things using React JS. There is little logic in hiring people who can do some obscure fashion-of-the-week algorithms, assuming if they can do this they can learn React is just plain wrong. A person does what they are good at doing, if some one is good at interviewing their incentives are in changing jobs often to find the next company that can offer a raise. Not getting your work done.
Also this whole thing that one must know these algorithms because they might once in a life time face a need for Dijkstra's sometime during midnight at a place where they wouldn't find an internet connection, is unrealistic. C'mon. Get real. Companies are full of situations where people are digging with shovels and spoons, because people don't have the skills to use and build tools and system to save thousands of man hours of manual effort. Code bases are full of tech debt. Deployment problems because tests aren't written, or there is just no CI infrastructure. Lack of skills and productivity is a far more realistic and commonly occurring problem than these once in a decade algorithm needs. On top of this comes the need for proactive problem identification and solving(a.k.a innovation).
The fact that FAANGs have to acquihire or acquire companies to grow shows they are not hiring the right people either.
As of today if you are hiring for devs. You must look for turn-key projects executed, expertise in one main programming language/stack, ability to script quickly, skills to build tooling/monitoring, ability to produce deployable code, code maintenance, writing test cases etc etc.
Considering most of us spend about 90% of our time coding and a very small amount of time doing code reviews, your best signal in my experience is just giving candidates something to code.
It was good because:
1. There was no faffing about to set up the environment 2. There was some small, easy wins at the beginning which got me engaged and thinking about the task. 3. It was fixed time - I had to specify when I would start and then the test was emailed to me 15 minutes before that time. I then had 4 hours to return it via email.
This solved most of the problems I've seen with take-home tests - namely that they take far too long and ask the candidate to do all sorts of pointless admin tasks before they can start coding. Unless the candidate happens to be super familiar the exact environment of the test (including setting up a new project - how often do most people start new projects?), you're either making them waste hours of time before they can start, or disregarding good candidates.
Ultimately you want to find candidates who can solve problems and understand software architecture. Unless you're interviewing for a devops position, making them spend hours setting up a build environment seems like testing for the wrong thing.
Portfolios are a common requirement in other fields where creating a portfolio is even part of the educational process.
All of these would show 1. basic language ability, 2. some simple algorithms, 3. ability to do something to completion.
So far, so good. They seem to have an attitude of seeing the best in everyone, and everyone seems talented. Maybe I can learn a thing or two from them!
1. Whiteboard interview
2. Opensource contributions
3. Take-home assignment
4. Becoming contractors for a short period
5. Etc
So if they fail whiteboard interview, they can redeem themselves with their opensource contributions. If they don't have opensource contributions, then they can take home our assignments. And so on.
So it's fair (I think).
I was asked to bring in my laptop with some of my own code - I had a side project at the time, so no problem. We talked about the code, decisions I had made. I was asked to implement a simple feature. I did.
No time consuming take home exercise. No whiteboard riddles. No need to have a github full of work (lets face it most peoples work is in house and not easy to show off). Code I was familiar with, so less stress. The only downside would be not having code available, but then it will be the equivalent effort of a take home exercise to produce something.
Seems like a good solution if the candidate has a side project. However, many candidates might have their majority of the "good" code they've written owned by their previous company.
> I am sharing in the hope that this will become the standard
Hopefully it can become a standard option. There's probably never going to be a standard technique which fits for all candidates (or employers).
Which is a huge problem in this field that stymies wider innovation in technology by limiting discussion around technology through literal gag orders. You should be able to reuse whatever you write. It's silly. I'm not stealing cash out of the register, the company doesn't lose anything from me keeping a copy of my code snippet around for future reference. It's not the whole codebase, it's just what I've contributed. I swear, lawyers have ruined technology.
Hiring is a crapshoot. You'll make bad hires. That's an unavoidable fact of life. Professional sports teams spend millions of dollars on talent evaluation and still get it completely wrong all the time.
No, a bad programming hire is _not_ devastating to your company. You'll live. A bad CEO hire could be devastating, but you can't do a whiteboard or a take-home test for that.
"The perfect hiring process" is classic premature optimization.
At the end of this 40 minute nightmare I had a bizarre homework to do at home that involved writing a list of uses to a paperclip.
They took a month to send me an email saying I wasn't accepted.
Most of the software engineers are certified right from the start when they graduate in university. All these people hold a certification in their hand, namely their diploma which proves that they are actually deemed suitable to hold any software engineering job in the industry.
The whole hiring process could be replaced with a timely renewal of accreditation. Then anyone who passed accreditation could be deemed as passed the technical interview. And then case closed.
On the other hand, i've worked with plenty of people with CS degrees from respectable universities who were useless. A degree is not anything like a certificate, and universities simply aren't in the business of actually deeming anyone suitable to hold any software engineering job in the industry.
Even if you fixed this, so that somehow every competent programmer, and only competent programmers, had some certificate from a university, all it would do is let you hire graduate developers easily. It wouldn't let you sort the wheat from the chaff at senior level.
So how do we break the tie?
Whiteboard? No whiteboard? Pair programming?
If we accept that after a certain point the process is random, why not simply break ties with first come first serve?
In the final few rounds, is a candidate who can solve the exotic DP problem just because he's seen it before really any better than the others?
Hiring managers have a real problem to solve: “how do I hire?”. They don’t have the time to conduct actual experiments and work out what actually relates to better hiring. So they trivialise the solution into some assumption about how the information captured in an interview generalises to overall suitability. They then proceed to defend the process and it’s results as if it was state of the art because bad hiring = bad management in the politics game.
To me it feels obvious that whiteboards, pair programming , take home tasks, panel interviews , etc are all subject to being highly flakey and highly presumptive not least because repeatable processes are just hard to enforce and train people on.
What really works is survival. Practically every company has a probationary period but it’s rare that once you’re in you’ll get bounced out in my experience. IMO bite the bullet, over hire and then collect the cream based on evaluation of the total output and the teams impressions in the probationary period ...
That sounds at the least unethical and perhaps illegal in some countries. You're messing with people's lives, their ability to pay their mortgages, support their children, and other commitments they might have. While hiring mistakes sometimes happen, you should not be using firing as a tool to make your hiring more cost effective.
And I also think word would get around pretty quickly not to apply to companies who did this. Per the article we are already making lists of who uses ineffective whiteboard tests. I very much doubt any company wants to be on the list of "hires you and then fires you instead of doing effective interviews".
That might work if you have 110 people applying for 100 positions. Just hire everyone and then fire 10 people. Over hiring 10 people probably isn't that much of a problem.
However that does not work if you have 100 people applying for 5 positions. This is the situation we found ourselves in and we had to filter somehow. We chose to do a very simple code challenge which ended up being a very effective filter.
Companies that don't have a broken hiring process
Only because they don't currently do whiteboards. I feel like interviews in general is a tough problem
Literally no company wants you to make actual production code on a whiteboard.
So I'm not sure how accurate this list is.
What does an interview as an accountant, sales or manager look like?
If the interview bar is too low, you will be exchanging hours of frustration by months to years of frustration.
At the end of last year I wanted to change jobs but I was paranoid about my raw algorithm knowledge. I've been a consultant for over 15 years. The last company I worked at for a decade. I've developed software given a specification, worked with teams to design applications, lead teams, presented to management of companies, and even was part owner in a company -- but I never was good at rote memorization.
I know my limitations and I know how to find answers.
So, for 4 months I practiced online programming problems, read interview books, and had my wife quiz me nightly. The nightly quizzes were whiteboard answers and I had to explain the solution enough that my wife understood.
In the end, I was interviewed at 4 companies: Daugherty Consulting, Google, Amazon, and Target. (For Google this was my second interview in two years. The first interview was a shock, I froze during the preliminary interview, and for two years contemplated if I'd ever quit my job.)
Daugherty never had me do whiteboard programming but did ask me some algorithmic questions. These were much easier to answer verbally. In the end I was told I didn't have enough experience in consulting working with large companies. (This was a bit of a shock but whatever.)
With Google, I never got past the first round. I felt very good with my solution coding in a Google Doc, but, they had wanted me to implement the Python bisect_left function. Instead I just used it to solve the problem.
At Amazon I made it onsite, but again, I failed to whiteboard a hashing function to their satisfaction. They told me it could have been overlooked if my architecture skills were stronger. They did complement me highly on my communication skills, which I appreciated. (I had worked for two weeks rewriting my accomplishments journal using the STAR[1] format.)
Target (where I work now), was completely different. I was given a choice of real-world-like problems to solve and a couple weeks to code. Two were pretty heavily algorithm/math-focused but the third was right up my alley -- implement a microservice backed by a data source and a different (potentially flaky) service. I took my time, wrote code I'm proud of, deployed it on Google Cloud, and explained my solution in detail to a Principal Engineer. There were still personality and experience questions (and I think also some algorithm questions) but nothing like my other experiences. It felt much more grounded in reality. Are you a solid developer, good communicator, and good fit for the company. In the end I didn't get the exact position I applied for but I'm still extremely happy.
My takeaways:
1. Maintaining an accomplishments journal as more beneficial than I could ever imagine. I write down everything I'm proud of - when I'm proud of it even if it seems minor. I can always delete it later. Also, the STAR format is actually really good.
2. Don't stagnate in learning. Technology and methodologies are changing all the time. I don't follow every fad or code in my spare time but I feel strongly taking some time periodically to maintain a level of expertise is a good investment.
3. Knowing my strengths and weaknesses really helped me focus while preparing for my interviews.
4. Learning from interviews and maintaining confidence was big for me. I took notes immediately after each interview of what I wanted to work on. I asked for as much feedback as I could get. These notes made it back to my journals and are things I'll refresh time-to-time because I know nothing is a given. Who knows what I'll want in another 15 years.
[1] https://en.wikipedia.org/wiki/Situation,_task,_action,_resul...
I think a good middle-ground is pair programming where the interviewer writes the code based on the pseudo-code provided by the interviewee. This way, the interviewee displays their thought process and communication skills without being bogged down by programming language nuances.
P.S. I used to have extremely successful in person networking before COVID19.
As much as we'd rather it not be the case, even for programmers writing skills are incredibly important. This comment is contains a few grammatical errors and is quite hard to read - I checked your comment history and many others suffer from the same problem. As some who hires, if I read a job application with this quality of writing, I'd probably dismiss it.
If you're having trouble finding a job, it may be helpful to have someone read over any written material you submit alongside your CV (cover letters etc) just to make sure it's clear and easy to understand.
Nothing on your resume will ever cause anyone to say, “We simply have to hire this guy.” On the contrary, the slightest typo, grammatical error, or stray bit-flipping particle will mean that your resume is instantly discarded.
When they do this to your resume, they are doing this to a representation of you: they are metaphorically throwing you in the trash like worthless garbage.
Are you offended by this? You should be.
So don’t use a resume.
Figure out some other way to get on the phone with the people you need to talk to.
You are no longer a tech wiz. You are in sales now. Your job is “selling yourself”.
Congratulations.
Japanese is my third language, so when I was shipping around my Japanese resume in Japan, I had an experienced native speaker review and revise. As a non-native speaker, there's a lot of nuance in writing that you just don't understand.
Your resume is often your first impression, so you want it to glow as much as possible. Honest, but glowingly so. :)
Might help you in terms of landing better jobs.
Also, to the point of the post, do you consider whiteboard interviews to be good or bad for your situation? With covid it's harder for companies to find which applicants should be interviewed in the first place, and short questions are helpful for filtering out people who can't code at all.
1 - First interview. Does the candidate know about the company? Does he know anything about the business domain? Conversation about problems he has encountered in his past experiences. it works like a knowledge sharing conversation, you really get to know how well things have been thought of.
2 - Freelancing period of about 3 weeks, 20h/week - You get to know the technical skills, cultural fit, communication skills and all other aspects that you only arrive to practicing it.
So far it has worked pretty well for us - we've hired about 11 developers and refused/have been refused by 5. It's an empirical process that works for both parties.