(edit: fixed fizzbuzz name)
I have questions with a variety of add-ons, so I can tune the difficulty to each person. A more junior person might take 30 minutes to give a complete answer while a more senior person might finish the base question in a few minutes so I need to add things on to make it more difficult. Some people might not have the ability to solve the problem at all. Either way, I try to get the interview to a place where the interviewee is beyond their ability to easily spit out answers.
Then the interview should turn into a back and forth discussion where they're asking me clarifying questions and possibly getting completely stumped. Either way, now we're communicating and seeing how quickly they can learn and utilize new information. That might be providing them clarifying info or teaching them something they don't know yet.
I don't care if someone can answer FizzBuzz or spit off facts about hash tables or anything like that. All that can be learned. If I'm interviewing a developer, I want them to be able to work on a project in a programming language they've never used and doing things they've never done before and become reasonably competent in a few weeks of learning.
> I don't care if someone can answer FizzBuzz .... All that can be learned.
Of course. But... sometimes it isn't. If a software engineer cannot do a question like FizzBuzz (and this does happen) then you can save a lot of time by cutting short the interview. If you're hiring SEs who cannot FizzBuzz, then I wish you the best of luck.
This isn't some leetcode or data structures trivia question that excellent candidates might fail. This is a test that I cannot imagine any good candidate failing.
There's a difference between a diamond in the rough who many not know $lang or $datastructure but is proficient elsewhere, and candidates that literally cannot put together a basic looping conditional. It's like hiring a network engineer that doesn't know what a network route is.
I've seen someone say they have never used a linked list, not even implemented themselves but used, and, on top of that, that they likely never will. I've also seen someone putting down the most basic knowledge about the most simple data structures (i.e. simple questions about what you can and cannot do efficiently with an array) as "premature optimization".
That thinking worked for their job, so they thought that's universal.
I don't disagree with your first point.
The problem is that asking questions like FizzBuzz is also a filter in the other way. Higher quality candidates aren't going to bother answering your filter question and bail out of the process.
I spent several years working on a large tech hiring platform and a thing we noticed again and again is that putting up any filters immediately drove away top tier candidates. They're just not going to jump through hoops when they have so many other options available to them.
Now if you goal is to hire a more average quality candidate, then you can get away with filter questions, coding tests, etc.
> Of course. But... sometimes it isn't.
Right, it goes both ways. If FizzBuzz is so easy to learn, it's reasonable to ask, why are you applying for this position without having already learned it?
You want a network administrator to know what a routing table does, a C engineer to know basic pointer arithmetic, a web developer to understand URLs, and so on. Yes, those are all things that can be (and have been) learned, but if someone applies to an aforementioned position without that knowledge already, massive red flags should go off for all kinds of reasons.
I've listed very basic knowledge, but the same goes for more deep knowledge in a particular field. The question here is about finding the line that is still useful. Categorically saying that no such line can be found because no question is basic enough to serve as an early filter is not useful to anyone, including the candidate.
I do care if someone cannot solve FizzBuzz, because it's so bloodily excruciatingly simple that if a candidate cannot solve this, please take some time to learn the basics of your craft before wasting your own time with interviews.
The problem is that asking questions like FizzBuzz is also a filter in the other way. Higher quality candidates aren't going to bother answering your filter question and bail out of the process.
I spent several years working on a large tech hiring platform and a thing we noticed again and again is that putting up any filters immediately drove away top tier candidates. They're just not going to jump through hoops when they have so many other options available to them.
Now if you goal is to hire a more average quality candidate, then you can get away with filter questions, coding tests, etc.
I wouldn't even know how that happened in an in-person or video interview, or even phone screen. "I'm sorry, I do not want to answer your simple upfront question, because it is beneath me"? Well, good luck to your future endeavors then.
To be clear, as I explained in my first post, my filter questions are not FizzBuzz, they are much more domain specific. At least if it's past the phone screen stage and for more than an internship. But they are still identifiable as trivial filter question for any serious applicant.
Imagine you have 10 companies that you are willing to consider:
- 5 of them are willing to immediately bring you into a serious interview
- 5 of them want you to take a coding test or some other pre-interview step
Why would you bother with those last 5 companies? Just go for the companies that respect your resume enough to bring you in for a real interview.
Those latter 5 companies will lose out on a lot great candidates due to their filters.
I can tell you having worked on tech hiring at scale and having AB tested a lot of these things across many companies, you lose a lot of grade-A candidates with anything beyond bringing them in to interview.
The thing about hiring though is that no single company really notices their own inefficiencies. Everyone has their own filters and their own method of incorrectly rejecting great candidates. However, they ultimately hire people so they feel their process works. They don't really know if they could be hiring better people or hiring people faster.
I don't see the assumption anywhere that the filter question is a separate step from anyone else here. Even if you mean a phone screen, then the easy filter question is only one of the very first questions of that phone screen, the rest will be more substantive.
So let's continue with the filter question not being a dedicated extra step (really, I don't think anybody assumed so):
> Why would you bother with those last 5 companies? Just go for the companies that respect your resume enough to bring you in for a real interview.
Because presumably I want to work at any of these companies. If the thing that's stopping me from applying to them is answering a trivial question, either my desire was not strong at all to begin with, or there is something seriously wrong with me.
Even if this is about the phone screen and not a filter question per se, I'm still not sure I really like the idea of omitting phone screens for more senior positions because you might lose on "better" candidates. For such a senior engineer, a phone screen should basically be a formality, and actually a chance for themselves to dip their toe in. Not just in seeing what kind of technical areas they come up with, but also an opportunity to ask preliminary questions about the company and the position in general. If that is off-putting enough to not consider applying or stop the process, then, again, the desire to work there could either have not been that strong, or the other reason.
I can't agree with that. I don't see point of a filter question if its not a separate step before the full interview. If you're going to bring someone in for a proper interview, go ahead and give them a full interview session. It's weird to put a filter a question into a full day's worth of interviewing.
And I can say, having seen hiring practices across tons of companies, filter questions are almost always a separate step where candidates have a tendency to fall out of the hiring pipeline.
> Even if this is about the phone screen and not a filter question per se, I'm still not sure I really like the idea of omitting phone screens for more senior positions because you might lose on "better" candidates.
Phone screens are fine because they're usually more about culture fit and trying to figure out if the candidate and company are right for each other. Experienced candidates that get asked basic technical questions on a phone screen have a tendency to report a negative experience and not have a desire to move forward with the company.
Some candidates won't get the full day, to prevent wasting a lot of time.
It's true that I haven't seen an individual interview being cut ultra-short after a failed filter question, but it definitely informs whether the next interviews will happen.
> Experienced candidates that get asked basic technical questions on a phone screen have a tendency to report a negative experience and not have a desire to move forward with the company.
Sorry, but, thanks for coming, and good luck in your future endeavors. As an experienced candidate, if I could not stomach that, politely and with a strong desire to move on swiftly on the other side as well, I am asked a simple technical question for the apparent reason that the interviewer cannot divine if I am actually able to program (unlike so many examples in this comment section alone), I'm not sure why I would really be considered a "grade-A" candidate.
It’s outdated, useless, shows a lack of imagination.
I don’t want to work with people asking me whether I can perform a loop with a basic conditions.
And it’s a hill I am willing to die on.
I am financially independent, mostly here for the intellectual challenge. And I have never met anyone asking me about FizzBuzz having then the capacity to entertain me intellectually.
I don't force you to work anywhere, though.
It would be like me trying to hire someone to improve my SEO, someone sent me his CV with 8+ years of experience, who talk about long tail keywords, cannibalization issues, internal linking and I ask : "Do you know what is the Google Console?"
I think both parties can have red flags
I'm thinking you might get better engagement with a binary-search algorithm. Let's say that a candidate's skill is on a spectrum from 0 to 100. You might start by asking a level 50 question, which if the candidate can't answer then you'll assume they're 0-50. Then you can ask a level 25 question and see how they do there before deciding if their skill level is 0-25 or 25-50.
Obviously this is overly simplistic: knowledge/skill can be over several domains and dimensions, one question might not give sufficient signal to determine whether to raise or lower the difficulty of the next question, etc. But I think the principle may be sound.
Also, if you're searching for top tier candidates, you might aim for a nerd-sniping [0] approach where you ask about a genuinely interesting problem. The good candidates will be fine musing over the interesting problem and discussing possible solutions. The lower tier candidates will either be completely stumped, give only one solution, or give poor solutions.
No loops are required, no data structures, no algorithms, really: just a simple predicate expression. It is one example of what I think of as "programming in the small", this basic survival skill of being able to relate some values and correctly characterize a situation. You just have to be able to do this correctly if you're going to write correct if statements and while loops. Lots of bugs boil down to errors in examples of this kind of expression.
I've had PhDs from good schools write doubly-nested loops looking for a point in common. I've have very experienced engineers write expressions that have false negatives or false positives. A depressingly small proportion of candidates can think of a good way to test their answers.
Tests, QMC, the works.
So be careful with this kind of stuff. You may not be filtering what you think you are.
And sometimes I am.
Source: I've lied on every resume I've ever sent out.
1) Obvious: If the question is failed, even a non-technical hiring manager understands how bad this is.
2) Fast: Quick to ask, quick to answer. Everyone agrees in advance that if it's failed, we can shorten the rest of the interview.
3) True negatives: Some bad candidates might pass the test. But no good candidate will fail.
4) Real: Write real code live in any programming language. Emphasize that perfect syntax is not important.
I've helped interview candidates for student positions, programming teacher positions, junior and intermediate programmer positions. It is shocking to some how often people fail filters like this. My current favourite:
In any programming language, write a program that prints the numbers 1 to 100, except if the number is divisible by 3, then print "turtle" instead. Example:
1
2
turtle
4
5
turtle
7
...
100
By the way, after they write FizzBuzz, ask them to modify it so that it prints "Rawr" or something if it's divisible by 7. If they printed "FizzBuzz" if it was divisible by 15, do they change their approach? If not, keep adding more strings for more prime factors until they do, or until they melt.
mean = liftA2 (/) sum (fromIntegral . length)
?I suspect it’s the OP unwillingness to find the good in people or find commonalities interview process.
That's not a positive method to approach an interview or conversation. If I knew that an interview was going to be a "gotcha" type interview, I would politely pass, I don't need this nonsense.
Because I need to hire someone competent. And idiots can (and do) still write good-looking resumes (or get someone else to write it for them).
Write a program to make
a == 9 && b == 7
No, I don't care how many new variables you use and what language you use.
Yes, I had people not be able to complete this. I would say this is something that is so basic that:
a) It's not testing any knowledge b) It's basic enough that you can figure out under stress, while never seen it before c) It's short, easy to explain (in 5 different ways if need be), and easy to validate d) It's an idiot filter. It doesn't even test if you can program, it's more like "If you know how type out code, are you also capable of reasoning about the basic logic of that code".
- function to reverse a string - function to sum an array of integers - function to get a deeply nested property value from an object
Since I work a lot with React and also interview people for that, I use the following to determine if people are at least somewhat familiar with react:
- take a static array of objects (first name, last name) and render a list of those objects
If people can’t write the map function, or are confused about the console warning for needing “key” that is usually a great indicator that they don’t actually know react.
What consistently surprises me is that I’ll interview people with 5-10 years of directly related experience, who will see questions like this and laugh or say it’s trivial, and then absolutely bomb. One person complained that they were senior and above these types of questions, and then spent 20 minutes trying to reverse a string.
But I mean, you should totally get back into at least being able to write something like fizzbuzz in the language you're probably using to write some sample code.
Except trim() vs strip() - that will forever trip me up per language.
Tell me everything you know about the hash table data structure.
Plenty of Staff level applicants don’t know anything about it.
There are 3 tiers of answer:
1) Don't know what a hash table is, when to use one, etc. Have never had a candidate in this category this do well in the rest of the interview, even for totally unrelated tasks like distributed system design.
2) Know it's a key-value store, know it's "fast". Some candidates whose main experience is in frontend dev get to this point, and then demonstrate other ability in the rest of the interview and get hired.
3) Rattle off that it's O(1) amortized, might be O(N) in the resize case, chaining vs linear vs quadratic probing vs other clever methods like cuckoo hashing, discuss how those methods effect cache behavior, etc. These candidates typically crush the rest of the interview effortlessly, even totally unrelated tasks like distributed system design.
Now I have several years of experience and am a much more effective engineer. I don’t remember many of the implementation details of a hash table anymore, but I still know how roughly how fast the operations are.
I guess I wouldn’t be applying for “low level programming” jobs, so the expectations are different. Still, I think I’d be better at it than younger-me who just learned data structures.
I still gave the rest of the non-hash table interview to every candidate. What I'm saying what the hash table question was a tremendous predictor of candidate success in every other aspect of the interview. System design, algorithms, problem solving, etc.
Candidates that didn't know anything about a hash table, even just the most basic idea of "fast insert/lookup/delete key-value datastructure", almost universally failed almost every aspect of the rest of the interview.
Candidates that know the basics could go either way.
Candidates that know all the details almost universally crushed every other aspect, including very unrelated things like distributed system design.
There are people out there who have built hash tables, who don't even know what they've built is called a hash table by educated folks. I know this, because I formerly happened to have worked with such a guy, after needing to change some of his code and discussing it with him.
This low level programming job, we were implementing filesystems from scratch. Very, very data-structure heavy.
Design a website with a user login, a password reset feature. And then go on from there.
E.g. talk about how to build your infrastructure. How to protect against CSRF. What is your stance in tokens in the front end. What about sessions. How to do force logout. Which database, why? When to use caching. How would you store Information about a password in a database in a safe manner. Salting hashing.
Synchronous vs asynchronous api. Idempotency. Different database transaction mechanism. How to do monitoring. Why is cardinality important for prometheus. How to do api first design.
This gives the candidate time to shine and I can at least understand where they are. And you can dig deeper.
My first question is a very specific coding question and it's a weed-out question. So if someone doesn't figure it out, they've been weeded out, simple as that. I ensure that candidates don't waste time figuring out nit-picky edge-cases and give appropriate hints when they're utterly stuck, but the question does its job very well.
A lot of this doesn't work with junior candidates, they just don't have the exposure to this stuff in school. Knowledge is really just one part of the equation too.
In the end you want to work together on things and you want to know how the person ticks.
So a self reflected, honest candidate is good and at least I have the impression you can assess where you would need to provide support /trainings for the candidate, or how self learning would work
Works for non-software engineers too. Filters out if they've done any research at all into our work and if they're curious.
What if they prepared with specific questions but you answered them during the interview?
What if a previous interviewer answered their outstanding questions?
Is curiosity about the company the trait you want to optimize for? In other words, does that curiosity translate to job performance?
I also think this question being so open ended and vague will lead you to get a lot of bad/blank answers. What should I be asking questions about?
And, as you might expect, people do tend to put the answers to the common questions in the job description.
Ask questions that aren't easy to find out without asking current employees, and that don't put employees on the spot. I can do that for hours.
With execs, questions turn more to product roadmaps, marketplace fit, revenues and budgeting, but again, I bet I can come up with interesting and practical questions longer than any slot any interviewer has in their calendar.
It's possible that the recruiter was expecting questions like the above, and saw your reasonable confidence as either over-confidence, incuriosity, or both.
A lot of time I ask hard questions like what is the company's revenue model or the company's growth strategy, where do you see the company in 5 years? That can scare a senior developer... then they get angry with me because I have turned the tables and made the person think in a non-binary way.
What you want to look for is their ability to process information, ability to internalize it, and their ability to bring it to bear in solving the problems they will be confronted with. You may have to do this with markers that don't necessarily match your own.
The richness of the network of information they have acquired so far is a marker. The nuances of the insights into that information they have acquired is a marker. The organization of this information into readily usable tools is a marker. These are what you should be looking for, rather than some preconceived ideas of what the markers need to be.
Disclosure: I do not even know if the title "Software Engineer" in the USA requires a formal education as such. I do not hold that title myself, I've just been working in IT for three-plus decades and provided net positive bottom line value to every firm I've worked for along the way.
Most, if not all, of the "idiot-filter questions" (note the tone?) in this thread, I've seen for the first time today.
I don’t care if it’s buggy, how much time it takes or whether the first scratch is any good, because it shows fundamental things every decent programmer knows or knows how to discuss and reason about imo. Unlike fizzbuzz, if you pre-solve this question at home and remember these details, you’re already good enough for the next question.
I’ve been programming for a decade, was a Linux/cloud engineer a decade before that, I’ve had code on the front page of google, launched products for Microsoft, and had my code shown on stage at nodeconf and aws re:invent. I am currently top 0.07% on stack overflow. I can answer the other questions in this discussion easily. I would absolutely fail this.
I can understand forgetting the quadratic formula, but the idea is taught to every school student who takes algebra.
- figure out what parameters should look like
- detect edge cases (e.g. division by zero when a==0 or sqrt of negative value)
- and discuss its usage (how to return 0, 1, 2 roots and how you signal it to the caller so it’s convenient)
And this is not a silent-wait quiz. When they seem confused I ask directly what it would return if `a` is zero. Or tell that a, b and c are parameters to such equation.
Anyway, maybe it’s sort of a TIL moment and I should stick more to fizzbuzz with periods changed or to string reversal.
But I feel confused about it as hell.
Actually looking up the formula for the first time in 20 years it's readable (i.e. doesn't use symbols I don't already know) so yes I'd be able to solve it. That's not guaranteed with every formula though.
Besides, in interview a candidate wouldn't use ChatGPT anyway, so I'm not sure how relevant is your comment.
I wouldn't say you're an idiot if you don't know the answer, but we need people who can work with concurrency and a surprising number of developers haven't.
Even if you did manage to avoid them entirely, it would suggest a lack of curiosity if you've never looked into how concurrency is implemented beneath the surface.
I have asked to estimate the amounts of skittles in a litre cube box.
Even the easiest five-minute coding problem is better than nothing.
We end up applying multiple filters throughout the process, but try similarly to keep them as simple as possible not to too strongly bias for one thing.
I've not had skittles in years, are they as big as smarties? I think a little smaller? Sure, you can start your estimation by laying them down in a 10x10 grid, and stacked 20 high, but that's very far off. Maybe 13x13x20? So that would be the most loosely packed version, if we shift them off so that layer+1 is in the depressions of layer+0, maybe we can stack 30 high. So yeah, that's my best mathematical approach and I guess it's at least 50% off.
Did I pass?
It's clearly not something like "Code Complete"
Is javascript case sensitive (I got the answer right purely by guessing, but the hesitation gave me away) ?
What is a closure?
This culls a lot of non mathematical people. (And we find that Mathematical people generally make for better STEM employees in a tech setting).
If they answer correctly (and many who get through the initial resume culling process do) then we follow up with a second question that is not important to answer but will show a candidates sense of curiousity (super important).
What is the answer to the first question to the power of the answer to the first question?
Fascinating answer to this second question that can delight the candidates we are after.
The distinction is useful when reading aloud expressions like 3-(-2) as "three minus negative two".
Curiosity: How do you call the terminals on a battery?
But "minus five degrees" for temperature ;)
I've been working in tech for 15 years now, and can't remember any time that I've needed to use imaginary numbers. The only reason I still remember anything about them is because my hobby is electronics.
«Yes thats a quick 15 minute fix»
That number 15 turns out to be very imaginary most of the time :(
Neither as a low level system programmer, nor in any of my high level programming jobs a long long time ago, have I ever needed imaginary numbers. Privately yes, but that's just when writing code for my hobby.
not idiot-proof but enough to inform me on what level we're working with, what their thought process is like, what tech they know, etc.
Radix sort is O(k * N) where k is the length of the key, and it's worth noting that there is a fundamental dependency between the length of a key and the size of a data set in that k scales with log(N), so that ultimately radix sort is O(NlogN).
So at this point it just comes down to comparing radix sort against other sorting algorithms in terms of constant factors, and radix sort certain has some use cases, but not many. One area where it excels is sorting enormous data sets that can be parallelized.
Common less correct answers:
- Because it only works on integers and you need comparison for general sorting. (Wrong, you can embed basically anything you want to sort into a fixed width int)
- Wide eyes and staring silently
I like this as a filter question because:
- skips directly to, do you know CS fundimentals
- requires them to tell me I'm wrong (if they can't do that thier competency is irrelevant)
- requires that they actually think about the implications of what they were taught and reconciles them against reality.
If you don't have those qualities, you probably won't be useful as an employee or coworker.