Hiring without whiteboards
github.com
github.com
That said, I'm always surprised how many candidates cannot even point to one problem they worked on they found interesting or one solution that they're proud of.
We worked with an HR consultant to develop a interview guide in the form of certain questions that we make sure to hit during the interview in order to be able to compare between candidates and make an informed decision.
However, we're small and not in the US. Anyone have experience with other companies in Germany/Europe? How does the typical interview work over here?
When I've been job hunting in the past I've also found that these have been the best situations for me. I find coding questions beyond fizbuzz totally silly and when something more is requested of me my attitude shows this.
This. I advocate this method of interviewing here in the states every chance I get. The most common form of push back I get is from hiring managers terrified of being hoodwinked into making a bad hire. It's like they don't trust their own judgement enough to be able to tell apart those who know what they're talking about from those who just talk a good game.
> I'm always surprised how many candidates cannot even point to one problem they worked on they found interesting or one solution that they're proud of
Junior engineers, I assume? I had problems with those questions when I first started out. It's hard to say with a straight face that the thing I was most proud of at that point in my career was creating a very simple templating system in PHP. Of course, the interviewers rolling their eyes and saying 'is that all?' didn't help.
Which is usually the case in larger companies, I assume. The person doing the hiring in that case is far removed from the people and work actually connected with the candidate.
> Junior engineers, I assume?
You're right, that's surely part of it. Not only, however - sometimes you get the feeling that work is something that happens to people, not something they seek out and try to get better at. Which for many positions is completely fine.
That says a lot about their maturity, not yours.
I'm not sure why there is such extreme hate for the whiteboard. Yes it has plenty of caveats when it comes to actual coding and recruiters should not expect any candidate to write precise code on that medium.
I do use the whiteboard for trivial CS questions limited to 5-10 minutes. Think fizzbuzz and string reversal. Candidates have the option to use my laptop (not ideal because the keyboard has a US layout) but they are welcome to use their own computer as well (if they thought about bringing it). It weeds out candidates who can't even produce basic code. And yes, a candidate with 2+ years of experience should be able to write a basic function on a whiteboard, a napkin, or whatever. If not, the interview is not lost but the candidate will have to prove his skills another way.
For most candidates, we also give a longer technical test which is to be done at home and takes 2 to 4 hours to complete. Candidates are given as many days as they want to complete it.
Whiteboard is an excellent medium however when discussing architecture and higher level ideas. It's also a tool that I've used during day to day activities with colleagues. Software based tools don't come even close.
I recently went to an interview that asked me to balance a binary tree on a whiteboard. It can be done, and I can do it. Thing is when you go to interview for that company that has 2 developers(small team, small company) and ask you that kind of question it puts you off thinking that those guys won't be great to work with (arrogance etc comes in mind).
I am fond of simpler questions, like you said fizzbuz etc. Obviously if you are interviewing a guy that has 2+ years of experience, he has to be able to pass the fizzbuzz test. When you are interviewing someone with 5+ years of experience for a higher up position I guess you do have to ask some harder question, but I personally think just speaking to the guy and asking him stuff about his past projects etc will give you a hint on if he has the skills he is talking about or not. Asking him to outline a hard task he took part and how he solved it is an amazing start. As a 5+ years guy personally would rumble about a few things and it would take me days talking about them. (That will give you an understanding if I've worked before or not on the things outlined on my CV).
If you're coming from a more FP background things like flipping a binary tree are silly obvious. If you spend a lot of time with C++, of course you know how to write a linked list with memory management. Python devs are probably way better at slicing and dicing data
There's a certain expectation of what's easy and hard, but it's really easy to have a mismatch given the diversity of education out there.
Last time I've had to do something with the trees implementation details (IIRC, that was AVL trees) was, like, 5 years ago. Right now, I don't exactly remember anything about AVL trees except for they have that nice O(log n) for lookups, insertions and deletions. Would I need to code them from scratch (or remember their internal workings for any other reason, like debugging some core dump), I'd go find the algorithm description (or particular implementation's source) and do so.
At the same time, last time I've worked with balanced trees was just a few days ago, but that was Erlang's `gb_trees` module that did all the stuff and I just had to freshen my memory on the syntax details.
All I would be able to do on a whiteboard is stare for a minute, trying to remember stuff, then probably say "uh, sorry, I don't remember this stuff - haven't did that in a long while". Yes, that makes me a worse programmer, I guess, but are interviewers actually looking for those who remember everything in their head and write code on whiteboard? Looks like a weird case to me that doesn't have to do anything with reality.
Given enough time and debugging I'm sure I could do it. I'm not going to be successful on a whiteboard though. If I actually had to do it I'd look it up instead of wasting my time figuring it out.
Unless you're interviewing for a position where they'll have to implement binary trees you shouldn't be wasting time asking about code for them. Questions should be relevant to the position.
You want candidates who can work through the buy-vs-build tradeoffs.
Reminds me of when I was asked to implement a filesystem API in C or C++ with file manipulation and path parsing and whatnot. The best answer I can think of is usually "With no weird constraints given (is this an embedded system, for example?) just use boost::filesystem and move on with your life." Not impressed with this answer, the interviewer would continue with "assume you can't use Boost!" Next answer is: "There are a variety of filesystem abstractions already written, far better than I could whiteboard. I'd go through each one and pick the one most suitable to the project." I thought it was a trick question at first. You really DO NOT want candidates who hear that question and launch into writing code!!
Similarly, I've never had to implement fizz buzz for real, actual, work. But I wouldn't trust a programmer who was incapable of doing fizz buzz without importing a fizz buzz library.
Fizz buzz tests for "does this person know what a function is, what an if statement is, and the modulo operator". And frankly, I'd give the modulo answer away for free if the candidate asked.
All programmers should know what a function and an if statement is.
Why do people keep saying string reversal is easy? Text is one of the hardest things out there. Unless you expect your programmers to support ASCII only?
Why not reverse a list or an array? That does sound like something that is doable in 5-10 minutes.
Other fun special cases: 🇺🇸 is U+1F1FA REGIONAL INDICATOR SYMBOL LETTER U, followed by U+1F1F8 REGIONAL INDICATOR SYMBOL LETTER S and should if possible be displayed as a US flag (otherwise falls back to text "US"), should reversing it create 🇸🇺 (replacing the flag with the characters "SU"), or still show the flag? (I'm not even sure if there isn't a case where both are valid country codes and it would change to a different flag?)
Similarly, Emoji can be formed from a sequence with combining characters inbetween, which don't display correctly if reversed codepoint by codepoint.
The worst-case scenario is a "self-hazing" where someone sees the encoding difficulties and freezes up instead of asking for clarification. I've never seen that happen though, in my experience though, everyone -- even the good ones -- just assume ASCII.
And I am happy enough if they do it correctly.
But if you are in python, and given a string object, and told to reverse it without using the reverse method.... Yeah, that's easy. You should be able to do that.
People are usually expecting the latter.
We also do a coding challenge after that, basically the candidate behind a computer, with all the resources they want accessible to them, implementing a simple example directly relevant to the type of work they'll be doing, while the interviewers are in the room and debate with them if needed.
To me, the most important quality in a candidate, once we determine that the basic skill level is good enough, is their ability to learn, quickly find the resources needed, and rely on the team when they need help. Whiteboard interviews don't seem to select for any of those qualities.
I have trouble recalling the details of things I worked on even say a week ago sometimes. I also don't find most of the work I do that interesting as I've been coding along time solving the same kinds of problems. However to play the game i would of course brush up for questions like that, and be quite convincing.
No problem here, it probably wasn't very exciting but have you never encountered something which stuck in your mind? Or that you could at least recall some part of a project if someone asked "So, you've written here that you've worked on project xy and did this and that, can you tell us a bit more about it"?
In general I think the question format favours people with a good memory for anecdotes and story telling ability.
It could be that this technique favors people good at telling stories.
Personally, I'm a horrible storyteller. If you were to ask me what I did over the weekend, I'll offer some facts like "oh, went swimming in a river and Bob lost his hat, but we found it later. The water was nice." Whereas Bob could easily regale you with stories about the epic hunt for his hat and throw in a punchline in the end.
If I didn't know that you'd be asking for solutions I'm proud of, I might draw a blank at that moment. Granted it's an interview setting and these are the questions one needs to prepare for. But if given the choice between speaking about myself or whiteboarding, I'll take the whiteboard.
I've always preferred math over history for that matter. It's easier for some of us to apply processes than to recite chronologies.
It's an alright proxy for other things, but there's a ton of good developers that leave their ego at the door, and their work at work.
I was working with a brilliant guy in the same company. He was very technical and you could say that easily he was programming all day long non stop. His intelligence and technical expertise amazed me. Thing is he wasn't good with people skills, hence he wouldn't fit the company I am currently working for.
There are companies that hire devs to solve a problem and give them time, space etc to go all hardcore. Then again there are companies that require from their dev's to be able to communicate with the rest of the team, and take part on meaningful theory-crafting meetings.
So neither whiteboarding, nor tell me a story question is wrong.
I just find it that whiteboarding difficult questions should be asked from the very technical jobs, and I've found whiteboarding difficult questions when interviewing for a company that wants to move out of wordpress to their own website in angular.
It does, but then again, maybe that's what they're going for (even if subconsciously).
A story is entertainment not work communication.
> It could be that this technique favors people good at telling stories.
Or people who think they're impressive. I know I would've been able to point out many problems and solutions I found interesting and were impressed with when I was 15. Today, not so much. The more I know, the less impressed I am with myself, and more and more stuff just feels "routine" and nothing special.
It could be that I'm just not impressive. At all.
Oddly enough, the opposite is true as well. I'm constantly impressed at the level of ability they have at this point in their career as compared to my ability at the same point in my career. They just simply know more. Of course, there's more to know these days but still, I find it impressive.
Keep a daily journal of what you work on. Nothing fancy, just stop by once a day religiously and add a few notes on what you did, what meetings you attended, who you spoke with, etc...
Save your evaluations and especially any award packages you or your team might get submitted for. At least where I work both are your supervisor's attempt to make you look as good as possible.
Then, when you're job hunting review your notes and bullets and collect the ones that sound the best and perhaps ones that have numbers assigned to them (size, savings, productivity, etc..)
Preferably you'd memorize these few stories about yourself, but if you must you could also bring a notebook with prompts to remind you.
My notetaking is not as rigorous as my Grandfathers, but I do something similar. Every month I make a new manilla folder with the month and year on the tab. Each week I write down every project I'm working on and every meeting I have. If someone brings me a new project I put it on the sheet. Every note I take a meetings and during projects goes in that folder.
At the end of each week I go through the notes from that week and write down anything interesting on that week's note sheet. At the end of the month I go through all the notes and write what I accomplished on the front of each manila folder.
It makes it really easy to keep track of everything you've done, without consciously keeping a journal in the moment. It's also a really easy organizational system. When someone asks, "What did we do for [x] in the past", all I need to do is flip through my folders looking at the front for anything that rings a bell, rather than trying to keep up with a tagged organizational system.
This probably doesn't work if you don't have a file cabinet though.
The key word there is "prepare". These are standard interview questions that you just have to have a prepared, rehearsed answer for that you can rattle off without thinking. There are tons of these types of "behavioral" questions. Tell me about a project that you worked on that failed. Tell me about a time you had to deal with team conflict. Talk about a struggling project that you had to help turn around. You can get a book full of them. Be ready with canned answers for as many as you can and practice them in front of a mirror.
Practicing algorithms for a whiteboard interview sounds a lot more fun.
Too bad (for candidates) that that concern doesn't appear to be shared by... well almost everyone in tech hiring these days. The general wisdom of the day seems to be "it's better to eliminate 100 false negatives than hire one false positive."
I worked with some really great folks that had trouble getting hired and it was because of the flawed interview process. One person writes concise and clear Rails code that's easy to read, and makes some of the wittiest comments on Slack (subtle puns that go unrealised for two minutes, then you finally get it and makes you smile). But he stutters a bit. And because of that, interviews are difficult for him. Which is a real pity, because in less judgmental situations, no one would think twice about that.
(Don't worry there's a happy ending and he's now at a good company.)
I learned after my first university job fair the necessity of being more outgoing during these situations, but there's great engineers out there who still live by the myth of "they'll know me by my work".
Pride is a very strong word, especially for those of us with impostor syndrome...
A lot of developers also have imposter syndrome. After I've solved an interesting problem, within a few days i'll probably have dismissed it as not being a big deal and started to forget about it.
These kind of questions are useful, but candidates should be preparing their talking points, because the risk of forgetting the story that's going to get you hired is too high
(When I think about the problem "objectively", all I did was use SSDT to simplify deploying database changes from QA to production without having to shut down either. No big deal. However, it was something that the company had been doing manually for years, spending three error-prone days each month, and they were extremely happy with my solution.)
I find it very hard to interpolate between these examples. And none of them are "hard"! I haven't written an OS or a more efficient linked list. All I've done is plugged away at something until it works.
My usual process (which isn't to say it's perfect, but it's a process I regularly edit to address problems I see) is to give a few, simple problems, and have them actually code it on a system.
* I know that the computer and environment are unfamiliar to you, so I expect there to be chopiness and typos * I know that most people are nervous during interviews, so I don't fail people if they freeze on one of the questions or miss a concept * I don't give "trivia" questions - the problems tend to be fairly easy ones that cover your ability to approach basic problems. Example: "here is a nested data structure (like an array of objects). Write a method to pull these sorts of elements out." * I'm actually hoping you'll screw up - I'm far more interested in seeing how you deal with a bug than if you can dash out an algorithm flawlessly. Nonetheless, I keep the questions simple because I want to minimize the impact of nervousness * I inform people they are absolutely allowed to use Google, StackOverflow, etc - I'm trying to mimic the actual work experience as much as I can. I WILL judge people on their searches, but I'm pretty loose - I can only recall one person that I dinged for searching, and that's because they went to about.com and copied the answer there without trying to understand it (and it subsequently didn't work). Usually this ends up GIVING people points, because if they demonstrate comfort with finding solutions, I expect that we can hire them and know they'll improve over time.
Despite this, I still ask a few questions that are purely verbal. I want to see if you are someone that will force your preferences on others or are willing to bend. (I'm not looking for a doormat, but I've never met one, so that's a bit moot) I want to see that you are continuing to learn things, because the skills you have now will just not be the skills we need in a year or two.
but overall - I have to judge candidates based on the limited info I can glean in an interview. Of: * resume * whiteboarding skills * discussion skills * coding skills
...I find the latter to be the best measurement of the options, even allowing that it won't be fully accurate.
We were building a tech team at a start-up in Germany.
Our hiring process evolved and stabilized into the following process, which worked exceptionally well for us: 1. Review of written documents, accepting any kind, any style and any medium. 2. Telephone interview with recruiter from HR about motivation, personality and the formal aspects of education and experience. 3. Personal interview with VP of engineering. Future employee is asked to bring some of his or her source code to the interview, any language any style any form is good. We don't copy the code, we don't keep it, we don't use it in any way. This is the only technical interview. 4. Interview with the CTO, immediately following. This is the most important social interview, and it also contains the main salary negotiation. 5. Decision.
Between any of the steps, we would briefly exchange our impression of the candidate, and it turned out that it is wise to sleep a night before taking the decision.
My function was VP of engineering and lead architect, and I would conduct the interviews of step 3. My principles are as follows: - Keep the candidate at ease as much as possible. We want developers to crack hard problems with "feet on the table" as the Dutch say, most of the time. Stress distorts things, and I want to see the undistorted version. - Get to know the candidate, and what makes him or her tick. This requires respect, tact and genuine interest in the person and his or her achievements. - Estimate the fit with team and company. I can talk a lot about this---but in the end it either feels right or it does not. - Gauge the technical capabilities of the candidate. For this I let the candidate choose a piece of the source code at will and have her explain what it does, and why it is the way it is. For all our hires, some form of dialog on software architecture in general emerged. This usually takes between 10-30 minutes. - Do your best to get the candidate interested in the company. For this I do not advertise the benefits of the company but simply explain what the company does, what the software we create does, which languages and systems we use and maybe even demo some of it live. Also I explain the risks of working for a start-up company to the less experienced candidates.
I don't do programming exams of any kind. Interactive exams exhibit dangerous positive feedback: you start with something in the middle. If the answer is good you make it more difficult. If the answer is bad you make it easier. Repeating this quickly converges to either extreme, which means you missed the chance to learn something more profound about the candidate, and even risk loosing suitable candidates at split-second blackouts. I don't like take-home exams either, although they are all the buzz these days. A take-home exam is a large investment for the candidate, and most of the people I hired never even touched the programming systems we were using (Erlang, Python, Ocaml, GLPK to name few). All hires did exceptionally well after a few weeks, and with a little bit of guidance. So what could a take-home exam tell me? Even worse, take-home exams are supposed to reduce the risk for the company, at the expense of the candidate. But do they actually do that? At what price? In addition, the large German probationary periods provide an easy way to correct mistakes later, which I had to do only twice so far.
Not a word is said, they are clicking at their laptops, and staring at the whiteboard, as waiting for the genie to pop out of a bottle. All the while your mind is frozen and stuck in a bad loop.
This lasts an hour, you are barely able to complete parts of the problems and are frozen. Of course this affects your usually creative and sharp mind.
The torture lasts an hour, time's up! You shake their hands, as a kiss of death, and head out. As you are walking back, all the answers to all the problems they wanted you to whiteboard, come rushing like a torrent in your mind. Too bad, another 'botched' technical interview.
This is my experience as a battle tested developer who is shipped many products and has been programming for the love of computers since the age of 12 (professionally for more than 15 years). I am not going to be working at Google any time soon (not that a Google job really matters to me).
The painful part is, I used to teach CS-101 and CS-102. I assigned that exact problem to my students and have graded several hundred solutions to it.
Now I warm up on lots and lots of coding problems before I start an interview process, but I always worry in the back of my head that I'll freeze again.
Had they not heard of Fizzbuzz? Or reversing a string?
Freezing can happen to anyone on any problem.
Ah, the memories come rushing back. I once froze in an interview, and force myself to blather something inane so that the phone wouldn't be silent. It went horribly.
I called back later that day or the next (it's been a long time), and basically said "that's not me. I know what I'm doing!" We had a conversation about the subject that I botched. It went well in part because I had already accepted that I'd lost that job; I just didn't want to leave an impression anywhere that I was a loon.
I got that job.
I think the idea of modern open office is built on the same idea that people would be thinking, communicating and collaborating at the same time, but I hate open offices for this precise reason.
What I'm saying is that there's an underlying fear of looking bad when spouting off your mental rough drafts, during an interview. What would be helpful if interviewers were clear about their expectations, as in, "You will be graded on your overall approach, not your initial ideas." or "We don't take points off for hints."
I interned successfully at FB, Microsoft and Google as a SWE in 2008-2010 and interviewed with a lot of other tech companies (not always but often successfully). Like you, I tend to freeze up in a whiteboard situation, and have most of my life, going back to college and prior (I studied in Romania where at the time you had to stand up and answer questions and use a whiteboard to answer them). I find that a few things help:
1. Practice practice practice. Simulate the interview with a few friends multiple times. It gets easier no matter how shy you are.
2. Smart interviewers and companies open with easy, friendly questions that put the interviewee at ease. It's a sign they understand this is stressful for everyone, no matter how confident, knowledgeable and smart they are.
While these interviews aren't perfect, with the right attitude they are helpful: it shows how the person thinks through a problem and communicates their ideas, and how they "work together" with the interviewer.
These are important skills that are important for all people regardless of their title.
I also believe a solid CS foundation is important even if coding is mostly putting things together. A systematic, mathematical understanding of complexity of Algorithms and Data Structures shows in one's everyday decisions.
There are only a few valid alternatives in my mind: 1. Take-home project work More realistic assessment of how effective a person is. Really good simulation of that, but no signal about their collaboration style, interpersonal dynamics etc. 2. On-site trial for a week or two This is probably the best way to assess someone's ability - internships and residencies are as close to real life as one can get. It's hard for people who already have a job, and expensive for a company to do. Simpler proxies/filters are needed.
Am I missing any other ways to assess candidates?
Yes. You are completely glossing over the fact that the candidate might be bringing something special to the table.
You have literally not left any room for the candidate to showcase what they've learned (again, not minutia of algorithms but something completely different..like the candidate could be a specialist in how compilers do register allocation)
How about the interviewer improving their interviewing skills to be able to identify strengths rather than being a pseudo automata that docks points?
Additionally, explaining how a system works is valuable, and I captured it when mentioned communication abilities.
The general process is fucked up because the interviewers are incompetent enough that they can't tell who is actually a smart guy. This is why incompetent interviewers resort to binary decision making on programming questions.
Here's a way to think about it. Let's take two candidates with equal communication skills:
1. This candidate solves all coding problems fast, with great syntax, good style and discusses algorithmic complexities well
2. This candidate is smart, can go deep in their specialty (because they've spent years developing complex expertise) but couldn't do a medium-level tree traversal without hints or has bad coding style (on whiteboard)
Who do you see getting picked (at Google)?
I remember a friend of mine at Google (a Java programmer) docked points off a candidate (a C programmer) because the candidate passed length of array as an argument. I was like wtf? Seriously? You don't know C?
The data-driven committee-based process completely ignores that not interviewers are equal. Their feedbacks on paper are not equal.
There's a reason scouting still exists in Baseball/Soccer/Football because data-driven process has given mediocre results at best. Interviewing is much like scouting. You cannot use mediocre engineers to identify who's a real talent.
They're both valuable, perhaps for different types of work.
A balanced tree does not tell me much about a person's knowledge but how they are able to apply and learn CS concepts does. Deep knowledge of a problem domain is valuable, but also highly perishable - the world changes fast and good engineers need to learn and adapt. Furthermore, if a candidate speaks about a domain they have deep expertise in, we have two more problems: 1. The interviewer might not know enough about the field to assess the candidate. (Generic CS algs and data structures are a good common denominator). 2. A candidate's knowledge about a field doesn't tell me how good they are at reasoning. They may have spent 5 years doing that one thing - that's enough time to learn almost anything in a limited domain - say OS schedulers.
How would you structure an interview process?
-- Furthermore, if a candidate speaks about a domain they have deep expertise in, we have two more problems: 1. The interviewer might not know enough about the field to assess the candidate. (Generic CS algs and data structures are a good common denominator).
So the problem is with the interviewer and not the candidate right? The interviewer is unqualified to interview this person
2. A candidate's knowledge about a field doesn't tell me how good they are at reasoning. They may have spent 5 years doing that one thing - that's enough time to learn almost anything in a limited domain - say OS schedulers.
Couldn't the same be said of candidates good at whiteboard tiny algorithm puzzle solving? They may have spent 5 years doing that one thing - say algorithmic puzzle solving. The fact that books and coaching sessions industry exists should prove this right? Your process doesn't give as much weight to someone who has the passion to go deep in something productive as much as passion to solve toy algo puzzles
Re experience - yes, if you're interviewing for someone with deep Java or ML knowledge certainly make sure the interviewer can assess that.
Logic puzzles are not perfect - I used to be really good at solving them and if I ever have to interview on them again, I'd definitely have to practice first, to get up to speed. But I do think that being able to solve them is a meta skill. If you get really good it entails that you are good at reasoning and solving complex problems, or at least a specific class. If you are good with trees you'll likely be able to answer a trie question. It doesn't mean you're just good at that, but that you can apply knowledge to new problems - which is exactly why you are hired. An equivalent real question I got in Java was how to access a private member of a class. If I knew the answer it would mean I have some random bit of trivia in my head. If I solve it using Java knowledge it means I can apply my domain expertise in a new way - that's compelling proof of problem solving skills. If I ask why would you want to do that, it indicates reflective and critical thinking skills.
A seasoned engineer in ML or Java should, imho also be able to solve puzzles. They are, in many ways, better proof of abstract thinking abilities and problem solving skills than specific knowledge. But I agree they are both important - if I want to build a search engine I prefer to have a seasoned ElasticSearch lady lead the project than an equally smart person with no background.
Re your friend, if it helps, I interviewed at a lot of companies and probably failed about 70% of interviews in the first week or two. I got much better after a few weeks, which proves your point that interviewing is also a skill. And I mean for both tech and behavior questions. That's normal, I think.
Re Google, while I don't represent the company, there are public statements that they'd rather have a false negative rate rather than false positives. It's better to say no to a great applicant (politely) than yes to a bad one. If you've ever had to let someone go, or deal with a bad team member, you know what I'm talking about. Of course there's the occasional interviewer who asks a silly question or gets something wrong. Do you never make mistakes? It's impossible to ensure everything goes perfectly in such a large company, but I can tell you, as someone who works here, that people try really hard to do a good job at it, and leave everyone with a good impression. It's the rational thing to do: if you treat people poorly 1. You're and asshole and 2. They'll hate you. Nobody likes assholes especially at Google, and I can share that I haven't met one yet. That's pretty amazing given the size. Thanks for engaging in this discussion, I really enjoy it.
Let me foreword by saying that I am not against coding interviews in general. But, I am against the interview process as it is implemented NOW. As implemented today, the process is overweight on coding optimal solution with good coding style.
Translated to humanese: "It doesn't matter if you invented linux kernel at your last job. We can't hire you because you didn't solve this tree problem in time or had bad coding style as COMPARED TO OTHERS who don't know shit but can code puzzles on whiteboard"
What I think an ideal interview score looks like:
Score = w0(whiteboard-coding skills) + w1(algorithms) + w2(area of expertise) + w3(design) + w4*(lets work together to invent something new/passion for engineering)
As it stands, the interview process has the following weights: w0=49%, w1=49%, w2=1%, w3=1%, w4=chunk change
See what I did there? The way interviewers in these companies are hiring is by calibrating their favorite coding puzzle over a wide range of interview candidates. They simply don't account for the fact that that person might genuinely be good at something that the interviewer doesn't even know exists.
>> A seasoned engineer in ML or Java should, imho also be able to solve puzzles. They are, in many ways, better proof of abstract thinking abilities and problem solving skills than specific knowledge.
You keep asking for proof that the candidate should be awesome. Where's the proof that the interviewer can adequately judge based on the equation I gave above? The candidate could be seriously good at ML but if you send the average java coder at Google to interview them then gosh, how would they even recognize his genius? Now you'll say we have 5 rounds to eliminate biases. Buuutt, the way the hiring committee reads interview packet is, and I quote you:
"It's better to say no to a great applicant (politely) than yes to a bad one"
Which means, even if two interviewers think this awesome ML guy is GREAT, the candidate wouldn't be hired because the other 3 interviewers were incompetent and marked the candidate no hire.
What I'm trying to say is that the hiring committee based process encourages actively removing smart people and selecting the ones who solve whiteboard puzzles. The candidate could be a shit engineer in person but as long as they solve code puzzles in those 5 hours they get hired. Make sense?
Look, I'm not saying we need to eliminate coding rounds. All I'm saying is balance it out. A person who can solve a tree problem in one round surely doesn't need to be asked another stupid coding problem (which is asked just to make sure some random crackpot didn't squeeze in through the cracks). I mean, if you don't have confidence in your own interviewers that you need to keep asking those problems, what does it say about the process? That you don't trust your own interviewers? Isn't that the real problem?
You can choose to ignore all feedback I give here but then, I'd specifically call Google out and say that they're not a great engineering company until they figure out that their hiring is broken and they need to stop having their mediocre engineers keep rejecting the good ones.
Or...if you really care about working with great peers, show this answer to hiring committee and see what they think. They need someone to tell them, "Hey, why are you implementing a process that self selects mediocrity"
P.S: Why do I know it is mediocre?
1. Other companies have tried the same process and it didn't translate to anything
2. You know there are enough mediocre engineers at your workplace :)
I'm pretty sure the weights vary depending on roles: interviewing a tech lead for ML requires deep experience in That field and will have more of those questions than basic CS algs. Or at least a higher weight than it would in a fresh graduate.
It's not my place to judge the average ability of Google engineers. I'd guess it's a normal curve somewhat skewed to the right since the bar is really high. In my limited experience they're on par with top silicon valley engineers but have a different profile than startup ones due to self-selection.
The point of many interviews is to reduce false positives, both for skill and professionalism. It works amazingly well. Every single person I talk to has something interesting to say and is at the very least competent at what they are doing. Not everyone feels brilliant but everyone does seem bright. And I'm not drinking the cool aid.
Hiring good people is easy if you just test for skills. It's much harder to make sure someone is a genuine person, or at least not an asshole. Google does a tremendous job here and I worked at enough startups and a few large companies to know how incredibly difficult that is.
That other companies failed trying to do this can be because of many reasons, from bad engineers to start with, lack of resources, mistakes in assessing ability or rush to get devs on the team asap. Again, not my place to comment.
Every company asking its employees to interview prospects implicitly assumed they are good, or at least competent. If they don't, they have a bigger problem to start with.
If they hired someone they already think they are good, right? And if they ask them to interview someone else they think they are able to judge, or don't understand that logic (In which case they have even bigger problems).
How does a mediocre engineer fail a stellar one? If the candidate is really better than the interviewer, won't they be able to solve anything they ask them to?
I think the reason multiple interviewers ask CS questions is to avoid the issue you mentioned earlier: asking one question the candidate just so happens to know really well. Random sampling should solve it.
I interviewed at Palantir a long time ago and was asked super hard CS questions over the phone and in the first rounds and I killed it. The last round I was tired because I hadn't slept well and tanked one problem. Just one, everything else I had solved really well. Yes I thought it was silly to reject me and wish they had given me another shot. If you are at Palantir and are asked to interview someone, think they are bad, and they get hired, it's demoralizing - you just wasted your time and the company doesn't value your opinion. Sure, you could have a score and average things but again, firing is way way worse than saying no to a good candidate who just had one bad question. And everyone I interviewed with was legit. (I had another series with them later, for a different role and got an offer :-) )
I agree that weights could be adjusted but 1. How do you know that would yield better results? (Show me the data - I don't think anyone has it) and 2. It can also be a cultural choice to bias more towards CS skills vs design or whatever else. Perhaps the company has great design patterns and can teach those skills v effectively. I'm not saying it's true, just an example. If you had a company and ran the interview process your way, I promise there will be someone just like you who vehemently disagrees with your weights :-). But if you could prove that process A vs B yields better outcomes, most companies would definitely follow suit (if they pay attention, are good at adjusting their processes etc).
I don't actually know how they read packets, I just know they work really hard to avoid false positives, and I wholeheartedly agree.
Btw I was rejected by Google twice before - resilience is part of the game ;-)
"Real talent" is innate and thus no amount of outside valuation will ever do it justice. No one else, but the "real talent" holder, can effectively judge his "real talent" so it is up to him to make it known to others that he does, in-fact, hold "real talent."
The process is designed to find people that are good enough to do an array of jobs. "Real talent" gets scouted, because it's shown that it "has got the right stuff."
This argument boils down similarly to the "smart but lazy" exasperations. "I'm talented, but no one realizes it!"
Candidate 1 is getting the job at my startup and Candidate 2 will be getting a referral if he ever decides to take up a teaching position. Deep specialization is useless unless you're doing consulting (in-house and out-house) or teaching.
White board interviews are supposed to be interactive. When I solve a problem, I always verbally say whatever is going on in my mind and share it with them. This does two things : 1. Helps me focus on the next steps instead of letting my mind race through different solutions, context switch often and lose track of my own thoghts. 2. Gives feedback to the interviewer so he knows I am on the right track and if not, give me clues (with or without him wanting to) that I can pick up on and proceed further.
In the end, my white board interviews always went like a conversation, a deep discussion about a problem that involves coming up with a solution and attacking it to see its pros and cons. I have come to appreciate the process itself because it tests several things like communicating to the team, problem solving, being open to other ideas etc.
Edit : I'd also like to note here that I didn't get these skills in my first interview. My first several interviews went horribly wrong like you mentioned. It is a skill that you need to invest time in. And you have to believe that the time you invest in this will pay off eventually. It did for me in several ways.
I observe that anglo-saxon peoples teach their kids to voice their concerns, needs, wants, and opinions, and to expect (!) results from this. (Compare how american kids behave at airports, train station, and public places with kids from other countries, e.g. french speaking countries).
This helps them approach situations in life with a healthy dose of self-worth and the baseline state of mind that if something is wrong it's not their fault.
In contrast, many other cultures teach their kids that it is impolite/crude/disrespectful to bother adults with your half-baked thought, or (God forbid) unexpected wants, or complaints.
I come from a culture where you are expected to moderate your verbal output when addressing your superiors (e.g. teachers, boss, etc.). You speak out only when you have something worthy to say.
Example:
I had trouble going to "office-hours" when at university in North-America. I always got the feeling that I am bothering the prof, and it is my duty to study more to understand the material, and I should not take his time when I merely did not understand something.
My anglo colleagues seem to not have such qualms. Their thinking is more along the lines of "I don't understand (not my fault), it is his job to teach me (most likely his problem), he has to explain this better, I pay for tuition after all...".
I am happy to have unlearned some of the above things.
But generally, I find the speak-your-mind-at-the-white-board-while-we-watch interview tailored for cultures where kids are taught to be vocal, and are difficult for equally intelligent people from other backgrounds.
I believe this is exacerbated by the sorts of cultural differences you highlight - I'm (relatively) loud and brash, many of them are reserved. I have no doubt there are lots of signals of confusion/understanding that I'm missing, but I don't know how to find them either.
I recently got my mid-term evaluations from the students (anonymous) - I was shocked to see many of them spent (or at least claimed, which I have to trust in absence of other data) 11+ hours/week on the work outside of class - triple what I expected. (and the answers formed a believable curve of results, so again, I have to trust it absent other data). I had only one student up until that point say how much time they were spending. Everyone else was silent.
Engineers really should communicate clearly about technical stuff. People solving complex problems really should concepualise their approaches clearly, ideally so that can be expressed in words. People really should reach out for help when blocked.
And if Anglos find it easier to do those things, then good on 'em.
I understand your comment isn't advocating for what I am insinuating, but in general, I think the sentiment that companies need to adjust their culture for their candidates is problematic.
I sought help from my professors and teaching assistants in college not because I felt it "wasn't my fault" or even that "it is his job to teach me" or that "I pay for tuition after all" but rather my thinking was, quite simply, "I need to learn this and I'm finding challenges with the current approach I am taking... maybe there is another way" where "another way" just so happens to include pursuing some direct face-to-face help from someone else. The line of thinking was outcome-oriented (I need to learn this/accomplish X) and I'd do whatever it took to make that happen.
There's a lot of variation within Anglo-Saxon cultures too. In New Zealand, we're usually a bit more reserved, but I can see that we're more forward than in many Asian and Pacific cultures. Also people everywhere have variation in introversion/extraversion and in their self-confidence.
It would be great if there was a way to do hiring that could identify good candidates without bias among these different things. Probably the only good way to hire is to simulate the real work environment as closely as possible. That's difficult and expensive though.
The best I can think of is to have a 45-60min written test with technical questions/problems going from very easy (where failure to solve is a red flag) all the way to more moderate, some difficult, and maybe even one very hard problem (solving which is a green (?) flag).
No need to be in the room with the person. Can put a webcam to record test if cheating is a concern.
After the test, the candidate gets a 5-10 min break, a drink, while the hiring team of 2-3 people take a quick look over the result of the test.
In the second part of the interview they all go into a room to have a chat based on the problems in the test. The candidate gets a chance to defend his choices, show additional skills, while the hiring team gets to ask tangent questions (how would you speed this up? why this data structure? if space was at a premium how would you improve this?).
I think the secret is to adapt the tone of the discussion to keep the candidate on the edge but avoid intimidation. If the candidate is a confidence artist, test him with more and more direct questions that he cannot weasel out of. If the candidate is more shy, engage him/her with a brainstorming like discussion, and go of on tangents from there.
The written test plus the opinions/notes of the 2-3 people present at the discussion should paint a decently detailed picture of the candidate, and also provide a written record of his/her performance for reference and comparison with other candidates.
Test results probably do correlate with job performance, and it may be better than an interview (not sure) but it's say it's still quite a long way from giving an accurate answer.
On the whole the West tends to have lower power distance than the East. And Anglo and Germanic countries tend to have lower power distance than Romantic countries, while Scandinavia has the lowest power distance anywhere.
France has quite a high power distance by Western standards, although it's still lower than most Eastern countries.
The US tends to have rather high power distance by Anglo/Germanic standards although if you look at all the cultural dimensions the US tends to be an outlier towards moderation in its cohort probably due to its amalgamated culture (the US is hardly a strictly Anglo country).
http://www.clearlycultural.com/geert-hofstede-cultural-dimen...
That kind of discussion between co-workers happens all the time. You suggest ideas. Have you tried this? Yes, but it has that problem. Oh, so what about that other approach? And so on.
It's a lot less stressful than an interview where you are expected to find the right answer, and it's also much closer to your real job.
How else will you prove that YOU shipped those things over that period?
* Github account? Ha, my code is on github in many places, mostly without attribution, none of them uploaded by me.
* Recommendations? Ha. Unless the person recommending you already works at the company, their trustworthiness is zero - they could be a paid accomplice.
* Resume? Unless you were the sole employee, there is no way to tell how much actual work YOU did.
* "Tell me about that one time you solved an interesting problem?" Ha. Can easily be a well-practiced lie.
(yes i have seen all of the above)
Turns out that the only sure way to tell if you can do X, is to make you do X and watch you yourself actually do it, in person.
Now, I am not accusing YOU of fraudulently trying to get a job, but people do. An in person interview is a good way to see.
(as always, i speak for myself only, not my employer, my cat, or our Great Lord FSM)
I don't write code on whiteboards for a living, nor am I applying for a job that will have me coding on whiteboards, so watching me do that will tell you almost nothing about the job you're actually hiring me for.
I agree if that was the only thing you asked, you wouldn't get a good answer. However, if you then dug into their answer you can probably tell if it is BS or not.
I remember having this exact experience with a junior dev who claimed to worked on some interesting projects at Uni, but when you dug deeper, you realised he knew very little about the technology he supposedly used.
That said, my company isn't really interested in hiring people who can code an algorithm to balance b-trees. My company just need people who know how to ask questions, write maintainable code, and use a profiler.
Also, how common do you think these people are? IMO, not common enough to design an interview process around. Then again, I don't subscribe to the "one bad apple will torch the orchard" paranoia popular in this industry.
I told the candidates that they did not necessarily needed to show me a working application (although that would have been great!), but simply to document what architecture, language and approaches they would use to implement the task.
Also the task could be documented in a few hours, and those that got close to coding a few classes might have spent a day at it. Now the argument could be made that I did not remunerate this people, but keep in mind the work they did was not expected to be more than a few hours.
For me, this is an ideal technical interview. Also, I ended hiring a candidate though this method, and he is by far the most productive and creative force in our team.
I interviewed once with a couple of owners in a restaurant. They asked me something relevant, and I all but froze. I knew the subject, but it just would not reveal itself to me.
I candidly admitted "brainlock." They were very gracious about it, I soon got through that subject and we moved on. I was eventually hired.
The lesson being, for myself at least, not being able to produce "on the fly" is sometimes compounded by being afraid to show or admit a problem. Good people, however, will be forgiving and encouraging. They've been there themselves, we all have.
There have been a number of times, over the decades, that I have basically admitted ignorance or temporary brainlock, and it has always gone well, whether I was hired or not. I think, if nothing else, that it's refreshing and novel for the interviewer.
It really reduces stress if you allow yourself to be human. In turn, that's much easier if you don't put yourself in a position that "I must have this job."
Please replace these guys with HackerRank. It so much better than this.
Next time it happens, tell the truth.
"Oh, hey guys, I get really, really nervous about whiteboard coding. It's really scary for me and right now I feel like my brain has stopped and I'm going to fail miserably. Would it be OK if we do this another way? - maybe on a computer, or maybe I can just talk about it?"
If you need it, you can also say "Can I please have a 5 minute break to get my head straight? Maybe a quick walk to the bathroom and a glass of water?"
I would be utterly and completely shocked if a rational, intelligent human being denied you that. And if they did, would you want to be working for/with them anyway?
If you tell a candidate to prepare, they should be able to come in and do this. If they can't write some simple code unaided, then I don't want to work with them. I was a copy-paste programmer once too.
Using their own computer way to many struggle with creating a new project or other basic things. And that dont make them comfortable either. It also takes much longer then using the whiteboard.
But in general, I think the idea of the whiteboard interview is a chance to see someone "think out loud", to see them reason through a problem. Getting a chunk of source code in an email isn't representative of that. (At worst, they paid someone to do it for them; at best, you have no idea what their approach was. Somewhere in the middle is copying and pasting most of it from SO.)
This is exactly why I do this kind of stuff as well when interviewing. It's not about syntax or the ability to recall (regurgitate?) a function, but how does someone approach a problem and where can the conversation go from there. If you sent someone pre-work to write an array matcher, you'll see a final result, but not how they think through it, and you'll miss potential queues to further the conversation.
Which would easily come out at the interview when they fail to answer basic questions about their code.
> at best, you have no idea what their approach was
So? Did you get code that would pass a code review? Did you get code with good commits/history?
It is much easier to talk about a solution, when the solution is right there in front of you, and you read an explanation of the solution online.
Regurgitating someone else's solution is fairly easy.
So thinking out loud while coding is a job requirement now? I have been an engineer for over 14 years, and a software engineer for 10, and I can count on one hand the number of times I have done that outside of an interview. I collaborate all the time, but almost never synchronously while actually writing code.
Sure.
>>And doing that out loud is the one of the few ways people can see how/if you do that.
Strongly disagree. I would posit that written explanations are a much better indicator of someone's ability to reason about problems, because:
a) They can provide hyperlinks, images and other supporting materials to aid their explanation
b) You also get to see if they are a competent writer, which is also a very important skill for an engineer
I'm not a newbie (like, 15 years coding). But I haven't even bothered to properly remember standard libraries peculiarities - since they're different across the languages anyway, and with modern software stacks (that are quite diverse) it's not unusual to write Python but do some C or C++ one day (speed-ups), detour to JavaScript, try out Elm, then have a day patching some Go code that an useful tool is written in, then go learning some Elixir. Is it strpos(substring, string) or strpos(string, substring)? What does base64.encode(blob) returns, again, ASCII bytes blob or a text string? Which module/header file contains/exports that symbol? Maybe I'm doing it all wrong but I honestly never remember such details that for longer than a week or another - a proper tooling always hints those kind of things to me, so I don't even recognize I tend to forget them.
On a whiteboard, I'd have some awkward moments.
Adds some extra anxiety as interviewee still worries about their code being correct or not. No one wants to risk being frowned upon just because they've messed up some syntax fine point. And since people are different, sometimes it really happens.
This can be avoided if interviewee's specifically told they just have to write algorithms in pseudocode, but it's not how it always goes in practice.
Of course, some interviewers are better than others, and can manage to do things to make it a reasonable experience. You outlined some of the things that you do that make it seem like you're in that camp.
I'm not sure there's any one-size-fits-all interview technique that is universally better than others. The homework approach seems popular here, but then there are others that are put off by it. Probably the best thing is an interviewer that's smart enough to change gears when they sense the approach isn't really testing the candidate's skill.
As always, some companies spoil it for the rest; whiteboards are fine but the crazy Hackerrank kind of interviewing, coding, on a whiteboard, correct algo's you studied in uni 10+ years ago and so on got a lot of people annoyed.
Nothing wrong with having someone sketch an architecture or a brief plan-of-attack on a whiteboard. But demanding someone to write a correct Bailey–Borwein–Plouffe implementation (I read a few months ago someone got that question in an interview; find the nth digit of PI efficiently as you can. How useless.) or something from their brain without a computer is just rather insane imho.
Seeing if they can write code would be actually writing code. Behind a laptop with access to internet and time to do it. Also; previous work.
Probably they didn't like the candidate and they dismissed that person through a question they weren't expected to answer.
On the other hand, if you dismiss someone because "is not a cultural fit" or some other subjective reason, the person can sue you, accuse you of discrimination and challenge you to prove your point over a legal mine field.
No company that I have worked for has ever said, "You know what? Let's look at the resumes of previous applicants, call them up, and see if they are still looking for new jobs." Every vacant position is always filled by direct, active applicants. Even backchannel applicants have to send a fresh copy of their resume to someone, so that it can be added to the applicant tracking system.
Tech resumes have a short shelf life. If the person is any good, and actively seeking to change employers, they will already have a new job by the time you have an opening for them. And if they are constantly doing a low-key, back-burner search, just to stay tuned-in, they will probably apply again to whatever you advertise, if they are still interested in your company.
The only companies that will keep your resume forever and call you back later are recruiters.
It may be that I have never been considered good enough to get hired, but not quite as good as the people they chose to hire on the first round, but for most companies, I either get an offer or I never hear from them again. Those still willing to communicate after a no-offer have never contacted me again regarding another open position. I admit there may be a theoretical nonzero probability of this occurring, but it has a lot of zeros between the decimal point and any other digit.
I am regularly contacted by recruiters from companies where I've interviewed successfully (with offers) and unsuccessfully (no offer).
They do it because it's cost efficient.
The book "Cracking the coding interview" also speaks about this in the early chapters.
I work mainly with companies with a significant physical presence in the US Midwest and near-South. The furthest west I have ever interviewed was in Denver.
A significant number of the companies I have interviewed with lack even the ability to respond to a follow-up message the day after an interview, so never mind contacting someone months later.
I can't imagine it's a very successful tactic, and it probably only happens when they're desperate.
The problem I have with whiteboard coding in interviews is that there just seems to be better alternatives. Using a computer as other people have suggested, but also just talking to them you can figure out if someone knows how to code. If you ask someone how they'd check if a number is in two arrays you can immediately tell a developer from a non-developer (or an inexperienced developer) by the language they use and the questions they ask. If they mention loops then that's a bad sign. If they start talking about whether or not the arrays are sorted, how big they are, whether you could refactor the array building code to return the data as a object index instead ... then it's clear that the person knows how to code simple things, and all without needing to pick up a marker pen.
Having worked with people like this, I have to call BS. There are a lot of people with the distinct skill of being able to effectively imitate the speech patterns of actual programmers. Asking questions, engaging in productive discussion, etc. But when faced with an empty editor, they simply cannot produce working code.
If you're already talking about coding a really simple problem like your array example, why not let them use a computer language to express a solution? After those 5 minutes, you can move on to richer parts of the interview confident that you're (probably) not going to hire a total bozo.
Having never worked with anyone like that, I will have to call BS on this because it is simply too hard to believe, without evidence, that someone can understand this level of detail and not know how to create working code. A parrot cannot engage in productive discussion, and aping speech patterns only goes so far. Plus, if they're so good at aping speech patterns, who says they aren't equally good at aping code patterns?
One is a social skill; the other requires understanding. Even in a technical conversation, a very large percentage of the interaction is social if the conversation is taking place face-to-face. I have also worked with people who could talk the talk, but not produce the code. Without social skills, those people seem to fail from job to job while not really understanding that they are failing: things just inexplicably go wrong all the time. With social skills (which is much more common), those people bafflegab their way into being system architects, middle or upper managers, and executives, where a lack of detailed low-level knowledge is more of an asset that keeps them from getting bogged down in details.
Sure, if someone were to start asking questions, finding out that we're doing this operation a lot against the same arrays, so it makes sense to rebuild them into a set, or we're doing it against only sorted arrays so we can do a binary search, or whatever, is great...but that's more interview technique, not actual coding skill. You're clearly asking a contrived problem, have provided no additional context (no business requirement is "search two arrays to see if a number exists in both"), so it's not really fair to assume a prospect will treat it as an ambiguous requirement that needs further refinement.
This is really the heart of the issue; you're not testing for basic coding ability, which is what you're thinking you're testing for; your own criteria is testing for whether they assume you're trying to trick them. Talking it out isn't testing for coding ability either; I've met people who could say "You would search through each array to see if you can find the item in question", and then literally could not code that correctly, in their alleged preferred language.
If you want to measure whether someone knows to clarify business requirements, give them a business style requirement. "We have an interface that allows users to provide a list of their favorite foods. We want to provide a way for users to compare their own list of foods with that of a potential friend, and see what they share". Okay. That leaves the underlying data structures up the developer (as they are in the real world), and describes a problem they can ask questions around.
The skill and art of "coding" is in finding solutions to problems, not "writing code". When I interview people I'm looking for someone who can explore problems, ask questions, and find good solutions. If they don't feel like they have enough information I expect them to question things. If they're working on assumptions (like 'what does the array actually look like?') I expect them to question those assumptions. Expecting someone to go deeper in to a problem by finding out more is not "trying to trick them", it's a discovering whether or not they can do a fundamental part of any development job.
For every interviewer who asks how to find if a number is in both arrays and is expecting a detailed discussion of he to do a nicer job by not actually having arrays, there's another interviewer who just wanted to see if the candidate can write two loops, leave the loops early when the value is found, and not run the second loop if the first one doesn't have the value; expecting to do all that in five minutes so more complex questions can be asked.
The fundamental part of the job being that writing code without having a good understanding of the problem is a waste of time and often leads to throwing things away. If an interviewer was annoyed that I asked for more information about a problem before writing any code I'd take that as a sign I wouldn't really fit in at the company.
I was a cofounder at a startup that was trying to improve requirements management for a couple of years. This might go some way to explain my attitude about this. I really hate writing code to solve problems without having enough information first. It's so wasteful.
Yep. You're giving bad interviews because you're expecting someone to have the same previous recent bad experiences as you. I would never ask about the context of the problem you gave because it is so obviously just a quick coding test. You want questions about the requirements, you need to ask a question that feels like a real world problem.
"The companies and teams listed here instead use interview techniques and questions that resemble day-to-day work – for example pairing on a real world problems"
I would contend that if you're doing pair programming with a candidate, you're looking at people writing code in an interview situation while somebody is watching every letter you type.
In fact, I think hacking on actual code is worse than whiteboarding: everyone knows how to use a pen and board, but don't do so frequently. Different candidates being placed in front of the same computer and IDE, however, may be vastly more comfortable or not - they may know the bindings, etc. If they're not the "driver" in the pair, they still have to grok the codebase and the problem quickly.
Whiteboards may be abstract, and people may focus too much on algorithms with them. I don't think that there is much fundamentally different about pairing, and I would have more reservations with pairing personally.
- There is no insert. If you need to go back and insert a line of code somewhere you must either erase all the code below it to make room (then rewrite said code), or you draw a line from the insertion point to somewhere else on the white board and write the code there. For the former you waste a lot of time. For the later the code quickly becomes hard to follow.
- There is no context driven autocomplete or quick method lookup. If you spend your entire day coding in one language and have done so for the last >3 years you should be fine. If you code in multiple languages and spend time with other tasks (devops for example), you might not realize how much you rely on this.
- The white-boarding process works well (given previous two caveats) if you code in a linear style. If you code iteratively the whiteboard is not your friend.
My recommendation to someone starting the interview process. Go buy a small whiteboard and use it to code solutions to interview questions you find online.
How many times, for your job, have you had to code up a solution to a problem, even a trivial one:
1. Just given to you 5 seconds ago
2. With no access to documentation (paper, textbooks, offline or online resources)
3. With no access to mentors or colleagues
4. With an editor crippled as you described
5. Have to get it right within minutes?
6. And then have to defend your code against someone who's had plenty of time to think about the problem.
I can tell you, throughout my programming career, the answer is "never".
> There is no insert.
This really isn't a problem. If you need to insert a line, in most cases you can just draw an arrow and then describe what you forgot to do there. Code may not even be necessary. Anyway, an arrow is fine.
> There is no context driven autocomplete or quick method lookup
Again, not really a problem. Just say "I forget the method name here, I'm just going to assume it's called 'append'" (or some other appropriate name). No reasonable interviewer cares in the slightest as long as it's clear you know what you're doing and it makes sense.
> If you code iteratively the whiteboard is not your friend.
This is absolutely an issue. Everyone codes iteratively though. Successful whiteboarders learn to do the iterative part of coding verbally instead of in code. You talk through the algorithm you're going to implement, and refine your plan in words, and then only when you've got a solution you're happy with do you pick up the marker. As a side benefit, this demonstrates your thought process much better than coding would. Actually, this is just a better flow even for development on your own computer. Once you develop the skill, you'll find that a little more planning time before you touch the keyboard almost always leads to faster implementation overall.
This is absolutely not true. I've interviewed at many major companies (Google, Apple, Snapchat, etc.) and they all say at the beginning "Make sure you have working code". They don't necessarily care about syntax errors or spelling errors, but they absolutely want working code.
I'm not sure how code with "syntax or spelling errors" qualifies as "working code." The fact that they don't care about those things is pretty much the exact thing I was trying to communicate.
Other things they probably don't care about: unimplemented placeholder functions (they might ask you to go back and fill those in if they thing it would be informative and there's time), or misremembered function/method names (though it's important to communicate that you know you don't remember).
I mean, obviously they do want code that demonstrates a well thought out solution to the problem. But no one expects that you could just copy it into a text file compile it, and have it run - that's what I meant when I said no one cares if you're writing working code.
Lovely. Someone put up with you, but now you won't do the same and offer to teach.
If you want $140k and can't code productively, most early startups won't want you. Larger organizations may be able to accomodate you.
There are a shocking number of candidates who apply to senior positions who flat out cannot code. I have hired in the past without doing the whiteboard exercise and been burned by this; never again. The whiteboard is a filter to eliminate people who misrepresent their abilities.
You should consider offering multiple technical tests and letting the candidate choose. Ideally the choices would range from whiteboard, take-home project, quiz site (hackerrank), or pair coding.
We ask them about their experience with the tech we use / plan to use and test what they tell us with specific questions. If you tell us you know ASP.NET, I might ask you to describe the default routing or some specifics about razor views. If you tell me you are a git expert, I might ask you about interactive rebases and cherry-picking to find out when and how you'd use them.
General know-how is usually more viable than the ability to write working code in a pinch. The closest thing to a whiteboard test we have is giving you a page of code to ask you what's wrong with it (although, we don't do that often).
Here's the deal.
As a developer, I have about 30 repos out on GitHub with plenty of examples I've done in the last three years. If you want to interview me, then go look at my code first. I have a pretty specific style so all the code is consistent. Go look at my code, dissect it, ask me questions about it, ask me why did this instead of that, why I prefer some library over another one, etc. If you feel like my skill set is what you're looking for, then bring me in and then we can see if we're a good fit for each other. If you don't like what you see, then don't bring me in - it's really that easy.
Having one or two whiteboard examples will never, ever, show someone the depth of your skill set. You have to look at their code, which will give you insights a whiteboard interview never will. All the things you mentioned about "basic skills" are obvious when you look at someone's code. Did they use classic or prototypal inheritance? Did they know what closures are? Did they use a factory pattern? Are they using ES6 yet? What tools are they using for their projects? All this and a ton more simply from looking at their code.
And if you're a developer, you should have an account on codeplex, or github, or bitbucket where interviewers can go look at what you're working on and the stuff you're building. It completely cuts out the "coding" part of the interview and should simplify the whole process.
Showing off that I know how to write (or recognize and copy) good code from scratch does not signal for that. I can't tell you how many times I have had to code something that is objectively incorrect because the old version did it that way, and the customer doesn't want the new version to be different.
The problem with the whiteboard interview is that you can't show an old, crufty code base on it, and ask the candidate to sketch out how to implement a simple feature request without overhauling the whole thing to use industry best practices. And you can't show an existing relational database schema that is pathetically ill-designed and ask about a new report for the customer. Those are the sorts of problems businesses have. They don't suck it up and pay the big bucks for a software pro until after the amateurs have already tried fixing it with an Excel spreadsheet or an Access database. And they don't want you to mess up the solution they already have, no matter how much baling wire and duct tape is holding it together.
Not everyone that can write code can also work well within an existing codebase. The only situation in the wild that I would trust to indicate that skill for my company is someone who actively contributes to an open-source project with a lot of other active contributors. A one-person side project that shows off everything you can do all by yourself won't reassure me. In fact, if it's too good, I might be worried that you will want to overhaul working code just because it's really, really ugly, and you can detect multiple code smells when you look at it.
Honestly? This should be ferreted out in a screening interview.
As someone who's hired developers both as a manger and a senior dev, you know what's really important to me? Knowing where your head is at as a developer.
What I really want to know is:
1 - When shit hits the fan and you're buried under a mountain of stress, how are you going to handle that? Are you going to fight through it, or just give up?
2 - I want to know if you're open to other approaches to coding? What about other frameworks or tools? Do you believe there is one way to achieve a result, or multiple ways?
3 - Does your code have be perfect on the first try, or do you get something working first and then worry about how sweet the code looks later?
4 - How do you feel about fixing bugs? What if I made you fix bugs for 6 months before you started working on any project, how would you feel about that?
5 - Do you have the courage to stand up and say something if you feel like it will negatively impact a project you're working on?
Knowing how to solve basic coding problems at a whiteboard can't substitute for an impending deadline and tons of work that needs to get done in two days. I'd rather have an average developer who can handle a shit ton of stress and get his job done, then some rock star developer who folds like a lawn chair when things get tough. I want the developer who sees a tough deadline as a challenge, rather than the dev who whines constantly about it. You can't figure that out by standing at a whiteboard. You need to sit down and actually talk to the person to get to that stuff.
1 - I've been active in sports my whole life. I work 40 hours a week, play in several competitive hockey leagues and still have time to tinker and build stuff in my own time. I'm also studying to get my commercial drone license, and have a host of other hobbies that keep me busy outside of coding.
2 - The only way I feel like a developer can possibly stay in tune with how fast our industry is changing is to code outside of where they currently work.
A good example is at my current gig we're building apps with AEM (Adobe Experience Manager), Java and AngularJS (1.4). If I wanted to go work in a smaller startup, do you think my knowledge of 1.4 is going to be helpful when most of my friends who are working at similar places have been developing with Angular2 for almost 6 months now? If I'm not working on my own to keep up, my skills will be completely obsolete when I want to get a new gig.
That's just a fact of life. I don't have any friends who simply show up, work 9-5, spit out some code and then go home and not do anything. Likewise, they don't go home, crack open Visual Studio and code until they pass out on the keyboard. It's pretty easy to do stuff in your off time and still have a social life.
The general feeling that an engineer can't be any good without working unpaid outside of their job seems like a self inflicted and unnecessary cultural norm we've adopted as an industry. I can't think of any other industry where someone who goes home and doesn't practice their craft after hours is looked down upon as much as it is in software.
I don't have the label for it. It's similar to how employers want to hire developers for junior pay but don't want to provide any training and expect them to have taught themselves on various tools while lamenting the lack of skilled developers. Someone else in this thread mentioned the view of software engineers being viewed as artists who have to have a burning passion for their craft and I think that's bullshit.
Yea there are always guys who actually have that passion and enjoy programming as a hobby, but they are the minority like any other profession. It's become some sort of virtue signaling. I've talked to a number of devs who go through the effort of having publicly available work to show off who have confided to me that they do it just so that prospective employers feel that they are passionate
Just because somebody freezes up when being watched and critiqued by an Evaluator, doesn't mean that they are 'copy-and-paste programmers'. There are probably a lot of extremely good engineers who find this style of interview difficult.
People that call 'whiteboarding' a skill don't really get it. If you have irrational fear over people's judgements and this manifests when you're being interviewed, you can't merely expose yourself to this fear often to lessen it. It is very difficult to change how your mind and body react to uncomfortable situations.
Of course, you need to test ability, but you should also accommodate people's preferences on the environment that they are evaluated in. You can negotiate whatever this is by speaking to them: they might have a Github with projects which they can discuss with you, or they might be happy to do a small project, etc. You need to keep your standards high, while also ensuring that they feel that they are able to do their best.
I disagree on a few levels. I can't back anything with data but I'll point to where you can find many people who will offer their anecdotal stories of desensitizing themselves.
Being able to talk and interact with people is a part of the role. Software is not written in isolation - it is done best with a lot of collaboration and critical review of work. Being able to communicate and give and receive critical feedback are cornerstone traits. Good code is not written in isolation and giving and receiving critical feedback is not comfortable.
If I interview for those traits and qualities, I hope that I'm not discriminating against social anxiety but I spent 5 years in toastmaster's throwing myself in uncomfortable situations until I could come out of it.
I feel like interviewing is becoming this PC thing where we try to form to evaluate people in the way that is comfortable for them, where we live in a world where people need to break out of their comfort zone and learn to shape themselves into their environments. Go to Toastmasters, speak in front of people, keep going and you'll improve your skills - yes the soft ones too. You'll become comfortable - you will - ask anyone who stands and speaks confidently in front of you in those rooms if they were always that calm and composed while speaking in front of people. They were not and they learned to speak 'off the cuff' on topics they were not prepared for or subjects they didn't expect in there.
Maybe I'm alone in my thoughts but I'm looking for people who will grow. I want someone who will stand strong if I give them a critical code review - I want someone who will hang their ego at the door and be able to talk about what we're working on objectively.
If I'm still coming off too hard on this, what if I say that it's a consulting company that I'm hiring for and that the staff will be in front of customers who will interview and drill them every day?
Ultimately, we can grow and change. I know we can because I have grown and changed. I want my staff to have a growth mentality, not a fixed view of themselves.
Interacting - be it in a normal non-stressful environment, or in most stressful emergencies when everything's on fire - is almost entirely different experience. I guess, primary difference is that no one judges you (at least, not explicitly) but you're working with the team to reach some goal.
Maybe there are interviews where interviewers are there to actually help candidate to tell about their good and bad sides and even receive useful feedback and not just "thanks, we'll contact you [never]". Wish I'd be to one, because every one I was to (not much) felt like an exam with a taste of scrutiny.
There is a difference between collaborating and receiving critical feedback from a fellow coworker, and being evaluated by an interviewer (or worse yet, a group of interviewers) who might or might not offer you a job (since your livelihood depends on that). That's why people who are typically comfortable working with others can become extremely anxious during whiteboard interviews.
> I hope that I'm not discriminating against social
> anxiety but I spent 5 years in toastmaster's
> throwing myself in uncomfortable situations
> until I could come out of it.
> Maybe I'm alone in my thoughts but I'm looking
> for people who will grow.
Asking people to reason about complexity is great, but that's not the way such interviews usually go. More often, the interviewer asks about complexity only so that they can compare to their own solution copied from the best of a hundred other candidates and then hand-tuned for days. It's a gigantic ego trip, and not even a fair fight. Candidates can see that, and it tells them a lot about what kind of person the interviewer is. Hint: it's not a pretty picture.
I was fortunate to go through an interview process recently where I had to do coding interviews, and it wasn't just a big ego trip. Second time in thirty years, vs. dozens upon dozens (all passed) of the worse kind. I could tell the interviewers had actually been trained, because I've been on the other side of that desk. They didn't care about the code I wrote, so much as how I approached a new problem, dealt with edge cases, explained various decisions, responded to challenges, etc. That's how it should be, but doing this stuff right is hard. Very few who pretend to do it have invested the time to do it properly, so they get results that are worse than useless. That's why there's so much hatred of the approach. Most people have never seen this kind of interview done right, and it's not their fault that's the case. It's on people who went from copy-paste programming to copycat interviewing.
That's not necessarily true. There are people in "tenure" roles as I like to call them. They're not ostensibly bad, but they're far from good. They get promoted to senior developer and have pay raises each year. They're not bad enough to be let go, but after 5 years, they're still around and earn a senior title. They're not really in charge, but they're technically in charge with time at the company and the title.
I'm not saying throw deep algo questions at them, but promotions don't mean coding chops.
I don't take "multiple promotions" to mean anything in itself. If they can talk about what they did and why, then I consider that they're probably good at their job. And that's the point of interviews, IMO.
I'm decent with a whiteboard. I routinely use one for brainstorming and discussing problems at work. I don't mind writing code on a whiteboard, even specific language code and including stupid shit like semicolons. I also understand that despite the fact that I can do it (and even enjoy it), it's a miserable experience for some people and expecting everyone to remember minor details that their editor/IDE takes care of for them isn't looking for the right thing.
Honestly, whiteboard interviews are for lightweight startups with very little programming knowledge. You should be looking at logic, not syntax. You should be looking at how the person thinks, not if they can memorize a specific language to do a simple task.
My experience lately has been an immediate request to complete a coding project right after speaking to a recruiter but before speaking to anyone technical. And perhaps the height of absurdity I have even had automated emails saying "thank you for you interest we invite you to complete this coding challenge/project."
Are you really expected to do one of these now for every single company you send your CV?
The whiteboards seems far more respectful of people's time at least.
If you are good at selling yourself, whiteboard interviews are awesome.
If you are nervous in interview situations, starting with a project could be an advantage.
Edited to add: coming in for a whiteboard interview will also take few hours of your time (or longer if you need to travel). A sample project can be done whenever you want.
Now, if you were to tell me that nobody ever looked at the test after you sent it in, then it becomes disrespectful. I had several companies do that to me a few months ago.
Not that it matters much. I don't think code tests will stick around. It's too time consuming for the hiring managers.
By requiring these tasks you're implicitly screening not only people who can't code but also people who don't have three hours to spend on applying to your company. If all you want is 20-somethings with no other responsibilities then continue as you are, but if you'd like to see a bit of diversity then you're going to have to compromise.
You could simply get rid of the take home test; that will shift the burden to the resume and cover letter, to the advantage of humanities graduates like me.
You could mandate a full day on-site, which will bias against people from out of town who want to relocate and people in delicate situations at their current employer.
For the time poor candidate, there is always the whiteboard exercise... Whose biases we are gathered here to moan about in the first place.
I always feel in these threads that people think hiring is 'broken' and it can be 'fixed' with the proper application of 'one weird trick'. But then all the comments actually have contradictory complaints. So obviously it can't.
What I am asking is that companies don't ask me to invest hours of time before they decide if they're even interested in talking to me. At the very least I want to have had a phone interview and the chance to assess whether I'm even interested in working for you. That's particularly relevant when looking for remote friendly employers, I've had several calls in recent days from recruiters responding to me saying I'm looking for remote work which I stopped on finding out what they meant was "at least three months fulltime on site, and then maybe we'll talk about some remote work".
Home test is totally fine after a phone call or a two. However, they are not fine at all just after submitting an application and no real contact.
I feel like this is the crux of it as well. I am not against projects or tests to show I am technically qualified. But this should be after talking to some potential team mates to see if there is even a fit, personally, work culture-wise, technical interest or any number of factors that you could glean by speaking to potential colleagues for 20 or 30 minutes.
I used to invite people for an interview without any prior testing, but it was generally such a big waste of time for both me and applicant. What's more, quite often I can see from CV that someone is underqualified, but I do not want to reject them immediately without giving them a chance, so I send them a homework. Finally, you would be surprise how often even people with good CVs turn out to be quite bad.
I don't mind long coding exercises, it's what I do all day anyway. Better than watching yet another episode of NCIS or what have you :)
[1] https://github.com/mdpopescu/public/tree/master/SocialNetwor...
So by your reasoning CVs are not meaningful at all. Should we get rid of them? Why stop there though, should we not even discuss past projects we have worked on in interviews since those are also likely exaggerations and possibly outright lies?
Except in practice it seems that its not a "little code" its a 4 hour project that is usually closer to a 6 or 8 hour project in reality.
- How is this any different than spending an unreasonable amount of time studying balancing binary trees and similar trivial whiteboarding problems?
I'm of the opinion that studying all these variations of the algorithm problems is a bigger time-sink and waste of time than getting into the habit of problem solving a take-home project.
And also take a few hours of time, and possibly more.
Some companies will happily bring you on site for a full round even though they have no idea whether there is an open position for you (infamous example, you can pass Google's hiring committee and then be told there's no headcount). And you've just wasted one day off.
On the other hand, take-home projects can be used in a way that saves everyone a lot of time. I don't disagree with you, they shouldn't be used for screening though.
In practice though this is kind of rare especially for the larger SV companies. I'm not saying it's never happened but that Google incident sounds like an anomaly.
How do take home projects save time if they take between 4 - 8 hours? That's the same time as an onsite takes. The difference being that before a company brings you onsite there has probably been at least a technical phone screen or two so the candidate has ben "qualified" to some extent no?
I can do a 4 hour[1] assignment on my own time, assuming I'm not given an unreasonable deadline. For a full round of interviews on site, I need to take a day off. That's a huge difference to me.
[1] I've rarely been given any assignment taking more than 2-3 hours, I would flat out refuse anything that can take more than 4.
Edit: oh and I can give you a very long list of companies that thought I was "qualified" after 1/2/3 phone screens, wasted a lot of my time afterwards, and ended up deciding I wasn't :)
https://medium.com/@meowlicious99/my-software-engineer-inter...
That completely defeats the purpose of a small, self-contained assignment like the one you were given. And your code isn't even in Java 7 (no diamonds), I think it was a legitimate remark.
I would have dazzled them with java 8 if I knew I was going to be rejected a based on that :D
I was under the impression that this was the usual algorithm/data structure/complexity rigamarole, not show us latest language features.
These are just off the top of my head, however theres no shortage.
It's nice that you get REPL and a keyboard, but the unnecessary pressure of being watched, critiqued, and timed are all there.
My mind just goes completely blank whenever I am put in this scenario.
Are we programming or defusing a bomb?
But yea that analogy fits coderpad well, along with all the other collaborative code editor + webcam websites.
With a repl I could easily pin myself on some trivial bug that interviewers would ignore on the whiteboard...
Some guys I've met, with different occupations and age, sincerely believe that smaller screens are more usable. Like small windows on 14" CRT when everyone had 17/19". Now it's guys that claim that 11-13" ultrabook is enough for them and they don't want more ever. This honestly baffles me every time.
I can't think of any way to do an interview without pressure. Pressure is unavoidable when someone's future job is on the line.
If you feel you didn't accomplish much at your last job(which would be a pretty common reason for quitting), then a question about it would be pretty stressful, since the company is evaluating you based on work you didn't find to be your best.
Speak for yourself. I'm far more relaxed when diving into a technical question than when playing a guessing game about what social signalling I'm supposed to be doing at that moment.
So, yes, if someone wants to force another person to code on a whiteboard then certainly, they can do better, but it's like asking someone to draw with an axe on the stone floor. Waste of good axe and floor.
1. You get to introspect the environment. I've had lots of candidates forget whether certain objects support simple methods (map, sort, join etc). CoderPad lets you perform quick tests to make sure.
2. You get to debug how you would normally debug. Whiteboard interviews basically rely on the interviewer saying something "Are you suuuuure there's no bugs on, say, lines 10 through 12?". I find this incredibly stressful as both the interviewer and candidate. CoderPad lets you run the damn thing yourself and see if it says what you want it to say.
3. Compared to probably the next most relevant comparison, graded homework assignments, I think CoderPad's free-form format is much more forgiving. As a candidate, I've definitely wasted hours trying to get some hidden automated test suite to pass with no feedback as to what sort of failure was encountered. CoderPad blends code evaluation with live feedback from the interviewer.
Yes, social pressure can be daunting for many candidates. I don't think this is something you can avoid in a live interview in general. I think you can get pretty good signal from project submission from candidates, but at the cost of greatly increased time spent by both candidates and interviewers. For many people this may be a worthwhile trade and I think additional options are good, too.
- Rob, you use Unix! Come quick!
* To disarm the bomb simply enter a valid "tar" command on your first try. No googling. You have ten seconds. *
- Rob?..
- I'm so sorry.
tar tf filename.tar
tar xf filename.tar
tar xfz filename.tar.gz # Newer tars don't even require the z
tar cfz filename.tar.gz directoryname
tar --help
More interesting questions would be like: "which lower-case letters are NOT valid options to GNU ls". :)
The reality of programming is, these days everyone Google's their problems. Sometimes you end up at StackOverflow or sometimes you find yourself on a random blog or official documentation. Occasionally you ask a coworker for help, but the ego a lot of developers have to be smart means most feel intimidated to do so. And sometimes you just bang your head against your desk until things start working.
Whiteboard and highly technical interviews that expect you to know high-level computer science concepts are tilted in the favour of graduates who are freshly minted from whatever respective educational institution they graduated from.
Yesterday. Although, usually I pull out a notebook instead, and work things out with colleagues (or on my own) on paper at my desk. That happens several times a day.
The thing is, the timelines are completely different -- in real life, we usually write the problem down, work out obvious details and a plan of attack, then come back to it hours or days later when people have had time to research and think. Never have I had a coworker interrupt me after 15 seconds to "say what I'm thinking about" any time I pause narrating.
In real life, going "hmmm" and taking a couple minutes is acceptable. And you have people being constructive during those times, instead of interrupting you.
Whenever I've had a whiteboard programming interview, it's always been a trivial problem that's a smoke test for "can this candidate actually write code or is their CV faked". I think it took somewhere between 5 to 15 minutes, which I consider to be rather well spent time compared to spending hours or days on a take home project or anything bigger.
The only time I've been given a take-home assignment I refused to do it because I was extremely busy at that time (finals at uni). The employer gave me an offer anyway. Someone else made a better offer, though (based on a whiteboard session).
The whiteboard is not a good way to assess the level of programming skill of an individual. But it's a good binary indicator of whether they possess any coding skills. If the result is yes and their resume is good, experience is relevant and maybe there's some GitHub or other code they can share, that should be enough to consider a person to be an able programmer.
Like any tool, the whiteboard can be misused. The task should be rather trivial and the time should be short. The purpose should be to weed out the people who have faked their resumes or are applying to programming jobs even though their talent is something entirely different.
That sounds pretty reasonable, but I don't think it's the norm. I had an interview over two afternoons with 5 or 6 different whiteboard coding challenges, each around half an hour and each much more substantial than a smoke test. They were things like parsing and evaluating a string with a simple arithmetic expression like "2+3*4+5" or taking a phone number and enumerating every possible alphabetic representation (2 is "a" "b" or "c", etc.).
Google has example whiteboard interview questions on YouTube, and one of them was writing a function in C++ that takes an array of integers and a target integer and calculates whether some subset of the array sums to the target integer.
Not the most difficult things ever, but hardly smoke tests, and pretty annoying to whiteboard in my opinion.
Of the three examples you mention, I think only the phone number alphabetic representation is a reasonable one to do on a whiteboard.
All the 5 to 10 whiteboard interviews I've had over the years have been smoke tests. This is in Europe, and I got the impression from this thread that it's a common smoke test in Europe and Americans tend to have longer whiteboard interviews.
Also: can you discuss technical issues with them? Code quality feedback, finding bugs, explaining their thought processes, acknowledging mistakes, etc.
Imagine they're part of your team, and you come to them with a problem you're stuck on. Do you come away with a positive experience, and closer to a solution?
I've had a few, and it's never trivial problems. Here's a trivial problem: - Write a method that takes 2 parameters - an Integer, and a list of Integers - and return a boolean indicating if the int param is in the list of Integers.
Instead of something actually simple like that, it's trick questions designed to appeal to the interviewers desire to appear smart and clever.
Programming problems, not so much. Domain problems, almost daily.
> When was the last time you can honestly say you used a whiteboard to solve a programming problem
Every time there is some interesting hard problem which I want to share with the team. We used whiteboards almost daily.
> Googling and ending at a StackOverflow question with a great answer
I google like 5% of stuff I do, because:
* a lot of time is spent on debugging and you just can't google why your program crashes on in file X on line Y * a lot of stuff is too easy and there is no need for me to google it * architecture - you usually can't google "how do I architect system X"
> We used whiteboards almost daily.
May be I am a little overconfident, but I always go into an interview and treat it as a formality to getting the job. By the time I'm in front of the whiteboard, I've already spoken to the recruiter and know what they are looking for, seen the job req to know what technologies they care about, and I've had a phone screen. Why wouldn't I be confident?
Another interviewer told me at the end of the interview to submit an application and they can get an offer emailed to me by the end of the day. We did a collaborative white board design session and spent a lot of time just talking.
Either all of the job offers I've gotten have been mostly just discussions or I talk tech so much I can't tell the difference between an interview and a typical day at the office.
I think that the perception that it's just a hazing process comes from a failure to look at it from the company's perspective. Most people know that they're good programmers (whether it's true or not...), and so they're like "This is bullshit...why do I need to prove my worth at the whiteboard when I know that I'm a good programmer." But the cost of a bad hire is really, really high (both for the company and for the employee), and the applicant pool is skewed through adverse selection, and a company knows nothing about you when you come in to the interview. Many people with stellar resumes can't actually code; some that can code don't actually want to work with you and would be bad cultural fits; and some outright lie on their resume. They're trying to figure out, in a 50-minute hour, whether you'll be productive on the particular problems that that employer faces.
Especially in the context of Google's public commitment to increase diversity among their engineers, it strikes me that revamping their interviewing process to eschew whiteboarding ought to at least be considered - it would be amazing PR for Google if they swallowed their pride and innovated in this area. And it would have a real effect outside Google, since so many companies' interview processes mimic Google's.
I see comments on HN time to the effect of "Why do I need to know how to traverse a tree or sort a trillion 32-bit integers? Why should I care about big-O notation? When will I ever use those skills on the job, when I could just Google for a standard library function?" I used those skills all the time when I worked at Google. When your data set is measure in petabytes, you can't just reach for a stdlib function, you have to actually know how the algorithm works, in depth, so you can implement it across thousands of machines. And any algorithm slower than O(N log N) is not going to work. Even in frontend work, I used tree traversals all the time because JQuery's byte cost was too high to use on Google services.
They have been taking other steps to increase diversity, though, like redacting names & genders on resumes, ensuring that every female candidate has at least one female interviewer, and trying to source candidates from non-traditional (i.e. non-Ivy-league, non-brand-name-tech-company) places.
Has Google innovated much in the interviews themselves in the last, say, 10 years? My perception is No. I get the impression that Google's old-fashioned process is a point of pride that they hang on to because it works (at least, the business is succeeding) and they are afraid of change. But it's a missed opportunity for a technical leader to advance the status quo in our industry.
For example, onsite, Google interviewers will still ask you to write code or pseudocode on the whiteboard. That's something they could re-examine (perhaps by giving an option of whiteboard or computer, and ensuring a level playing field between the options). They could get some good publicity from re-examining this, I think.
I interviewed on-site successfully at Google Seattle about a year ago, and I was given the option between writing code on a whiteboard or in a text editor on a Chromebook, being projected onto the screen in the room, so the interviewers could easily follow without awkwardly looking over my shoulder.
Of course, I picked the text editor. I can't imagine wanting to write code on a whiteboard. However, I did use the whiteboard voluntarily as a thinking and planning aid.
Unfortunately, I also heard that the text editor option was only available at certain offices at that point. I really hope coding in a text editor becomes a universally available option because it's just so much more natural. It essentially put my mind at ease and helped me think more clearly.
On a more general note, my strategy, which seemed to work well for me, was to treat the whole situation like a pair programming task. Even though the interviewer won't directly tell you what to do, they will often comment on things if you question yourself out loud enough.
Actually, speaking of looking at it from the companies perspective, in 2010 google and several other big name silicon valley companies were successfully sued because they all agreed with each other not to cold call each others employees to try to recruit them. They actively worked to try to make it harder to switch jobs from one company to another.
Right after that, they started doing whiteboard interviews, which serve the same purpose of making it painful to switch companies, but without the risk of being sued.
Now, if a company insists on an adversarial process, there are better ways to do it. The nature of the problem is given in advance. Both the subject and the evaluator go in blind, and they both do it.
A toxic work environment.
Of course, I believe a significant number of those "interviews" are hazing meant to find excuses not to hire or perpetuate H1B scams.
A less intriguing but probably more accurate reason is that finding good people is hard, nobody really knows how to do it, whiteboard interviews provide a rough filter, and they're what everybody else does, so we're gonna do it....
http://www.latimes.com/business/technology/la-fi-tn-tech-job...
It was happening, only more blatantly than is suggested here. Of course they are all motivated to find good candidates, but they are also motivated to keep their existing folks. Decreasing liquidity and mobility serves the latter purpose at the expense of the former.
I was always puzzled by how unrelated metric used for hiring is to actual work at many companies. I don't mind algorithmic brainteasers - on the contrary I would absolutely love to solve such problems at work yet those seem to be scarce. I also don't mind other types of developer work - be it more research or architectural oriented or "just" coding.
But I am not happy learning something different for interviews every few years and then forgetting it over the course of next period of day to day work. Feels like a waste of time and is confusing in regards to what one would actually do in certain job.
For more complicated algorithms I still love to draw lots of pictures before implementing them.
I can understand that people complain that they have to solve tasks (on a whiteboard) that won't have to do much with what they will do at the job. But this has nothing to do with whiteboard interviews, but with badly chosen tasks (say: data structure brainteasers) for the job interview.
I have experimented with short take home projects. This drastically reduces the available pool, since great freshers can get job in other places with whiteboards interview. Also, there is a sense that interview process is used to get people to do free work.
Also, non whiteboard interview processes are extremely time consuming. Non-whiteboards interviews will be better. But as a matter of practical experience, I have found it to be the only viable option for hiring fresh undergrads.
I agree that asking people to write dynamic programming problems are not good interview questions. But if the job requires primarily writing code, I think it is fair to ask people to solve coding problems with loops, conditional statements, hashtables, etc.
If you're writing a loop for instance, you might code up the per-item logic first, and then you remember you need a loop variable. Which there's now no space for, because in a normal coding environment you can just insert the line where it's needed.
It makes me feel like I need to have solved the whole problem before writing, and that's just not how coding happens.
I don't think all interviews should use it but for us it was good for filtering out candidates who would not perform in those situations.
Whiteboard coding is also a different kind of pressure. I can't imagine a situation where someone who already knows the answer is watching over your shoulder critiquing you while you work on a problem.
right? unless you are in downtime mode, there's really something wrong with your schedule/deliverables/tasks.
writing code is not a race, it's more of a marathon where you have to be at a healthy pace to get to the end, otherwise you will not be able to finish.
I think it's unlike any real situation, and you're fooling yourself that one kind of stressor is like another.
This sounds like an incredibly toxic working environment, where developers are either being allowed to create fires on production systems, or management is completely incompetent or both.
Also, being able to "handle it" has nothing to do with anything in this discussion. I'd much prefer to hire someone capable enough to not need to "handle it" than someone proud of the fact that they often s*it the bed but are willing to jump up and do the laundry under pressure.
I personally enjoyed it but it's not for everyone.
I think it comes down to expectations - in an interview I have to focus on impressing my audience and that's always there. In real life, I can focus fully on the actual problem being solved.
Got a great offer from both and accepted one of them.
I strongly prefer a coding project to a whiteboard. It is still a bit artificial, but it is way closer to my actual work.
Would be interesting tbh.
https://blogs.janestreet.com/what-a-jane-street-dev-intervie...
I also did a second code test, which I got to do alone on an offline laptop. The test had a time limit and it was designed in a way that you should NOT be able to finish it on time. This was communicated to me several times before I actually sat down in front of the laptop. The goal with this test is to see if a person can solve problems under stress, and if the person writes decent code that works, or just hacks something together and moves on to the next step - leaving a trail of bugs.
Overall, I think this has worked out great for us, but it might not be a good process for everyone.
1) Meeting where you are presented a problem and asked to immediately give your thoughts on how you would approach it.
2) Some days, for example weekend, to work on the problem. Candidate gets a monetary compensation for this time.
3) Next meeting where you present the solution.
Yes, there's always a risk that somebody will get somebody else to write the code, but having to explain before and after reduces the risk. Paying for the work shows you are serious about the recruitment.
In that case candidates should also pay companies to show they are serious about the position
If I have a problem to solve "under stress", usually it's just a one off to fix something in production until I have time to write it, refactor it, put a unit test around it, and a code review.
I've spent so much time in my career explaining architecture it's second nature for me to diagram while I'm talking.
During a whiteboard coding session in an interview, I would do some code and some pseudocode.
But at this point in my career if I saw that the interview was focusing more on coding than architecture/development process/automated testing that would be a big red flag.
In short, companies should approach interviews like free consultations with an expert or peer.
"Oh this guy solved the problem on the whiteboard with good syntax but he uses new and delete. Must be a bad programmer."
I think docking points on whiteboarding just reflects how bad interviewers are at judging skills, possibly because they themselves suck at self-awareness and skills beyond their domain.
What's the purpose of them?
The whiteboard session is a way to spot people who can't code early in the process.
In the actual interview, I IM the written parts of the question to the candidate. Then, I ask the candidate to share his/her screen, and use a program like Notepad or Textedit to complete the question.
It's useful if the candidate copy/pastes my IMs into Notepad / Textedit. Often I've provided a sample API that the candidate needs to program with.
I discourage using an IDE like Visual Studio or XCode. (They get in the way when you're just trying to quickly express an idea.) Most candidates "get it" that I'm just trying to see how they think and approach a problem.
Some candidates just feel more comfortable in an IDE, though. Other times, though, when a candidate uses an IDE it shows that the candidate will get too bogged down in details to communicate; or that the candidate really doesn't understand the language that he/she claims to be proficient in.
Edit: In the context of the article, I'm a HUGE fan of whiteboard coding. It tells me a lot about the candidate. Being on both sides of an interview; a successful whiteboard interview has more to do with how the interviewer plans the question and his/her expectations about what the candidate will answer. A whiteboard interview isn't there to determine that a candidate can think of a tricky algorithm; it's there to judge coding ability for very obvious problems with well-known APIs.
Where I think the industry needs to go, though, is a universal certification that demonstrates competence. Doctors have to go through this, thus doctors don't need to demonstrate competence when interviewing for a job. That's handled by the impartial 3rd party.
If you really want to test a candidate, give them unlimited time, a "real problem" and something that they don't know.
The skill that I look for in another developers is the ability to learn stuff that they aren't trained. This way I know that they will be able to solve new problems, to find better solution for old problems. By testing then in something that they don't know, I will see if they are open to learn new stuff and able to apply new concepts, and ask how they felt about it later.
Whiteboards questions can be very good or can be very bad or anywhere in between. Just like any interview question in general.
I for one prefer whiteboard questions to generic open ended personal questions.
Good whiteboard questions can quickly assess whether someone knows the basics of programming or not.
Open ended questions only judge one's ability to tell stories.
Even when it comes to CS algorithms questions. Yeah, asking someone to whiteboard quicksort might be bad, might optimize for the wrong things for the actual job.
But should it be a red flag that someone expected me to be able to do a more basic operation, that I've more or less purported to do professionally and efficiently, on a whiteboard on the spot?
[Edit: At my company we don't really do whiteboarding. We pre-screen with a time-limited open-book, open-internet test in advance. This helps account for candidates knowing how to quickly read/research/apply openly available information.]
I like that guitar question for a similar reason. Someone once told me that you can't program something you don't understand. Being able to break down a complex topic in to atomic parts is the difference between someone who can program, and someone who can develop applications that solve problems.
I'm an analyst rather than a developer, but most of my value comes from being able to communicate information to other people. I know there are people who are better than me at programming, or statistics, or modelling, etc. However, I know that I can be incredibly successful in any given role if I take the effort to learn as much as possible about whatever I'm analyzing, and if I focus on communicating the information as clearly as possible.
To that point, I'm really proud of the fact that the man who interviewed called me afterwards to say that I had given the best instructions for tying a shoe he had heard in 30 years of asking that question in interviews. That being said, I can see how someone who is a nervous interviewee would resent questions that seem designed to trip them up.
There's nothing like having to pull out your phone and take snapshots of a whiteboard because people can't be bothered to use something persistent.
15 years later I am a far better coder (I class myself as a full stack software engineer now) but I haven't needed to implement the sort of solution that they ask for in years (except in interviews).
Sure some bad companies expect you to be able to write something very specific on the spot, but thous are pretty rare imo.
It's not about "pressure", it's about the fact I'm an introvert and do poorly at composing while speaking. My brain requires time to focus inward to formulate complex thoughts, and interrupting any time I pause narrating doesn't give me that time to think.
I've literally never had it be an issue in my career, because deadlines happen on the order of days to months and in real life conversations it's acceptable to pause talking for 5 minutes. (Heck, in real life, it would be rude to talk the way you do during an interview, because it wouldn't give the other person a chance to share their thoughts about the problem.)
So basically what you are saying is that you are some kind of snowflake that requires it's own room to concentrate and be productive and you can't even think while doodling on a whiteboard.
In my books that counts against you, that doesn't mean you are straight out, but it definitely makes you a liability to some degree.
What I said is that I share a trait with about 25% of the population, more in software, where I can't do my best composing if I'm narrating at the same time. By "time", I mean 30-120 seconds without having to speak.
In actual development that doesn't matter: pauses to compose an answer are fine in conversations and most work time is individual anyway (even in open offices).
I do note, however, you didn't actually answer my question: in what way is whiteboarding like real work deadlines?
hurdur, what is stress, I dunno. Why is everyone on this site suffering from autism?