In Defense of LeetCode Interviews
alexmolas.com
alexmolas.com
LeetCode questions as interview questions are mostly theater. Most people who do well on these aren’t actually “solving them” on the fly from scratch. They just happen to have seen the exact same problem before and retake the steps they’ve memorized to get to the answer. Testing whether or not someone can regurgitate the solution they have memorized to a math problem doesn’t tell you much about how they will perform in a truly novel non-contrived constraint problem scenario, which is generally what most dev work entails.
Perhaps if you are working at Bespoke Algorithms ‘R Us, benchmarking this would have more value to your org, but for most dev roles at most companies it is hard to see it as more than a compliance exercise, or maybe even as a tool to weed out those with families that can’t devote the hours/day to LC memorization.
> Let us know if you’ve seen the problem previously
and also:
> In your tech screen, you’ll be asked to solve two problems in roughly 35 minutes. Practice coding solutions to medium and hard problems in less than 15 minutes each to help you be ready for the constraints during the interview.
The only way I could solve two problems in 35 minutes is if I've seen them before or it is a variation of a problem I've seen before.
Or just say “I’ve seen this one before” until they get to one you actually have seen before and ace it.
Leetcode is a joke. I’ve hired a dozen or so high quality candidates using a short 2-3 hour take-home. It shows us more than leetcode ever could. And sometimes people take it places I could have never imagined, these are people we move quickly on and they are the highest performers in the org.
Where in the process did you give the take home project? As a candidate, I would consider doing a take home project as the final round of an interview, but anything earlier than that and I will reject doing one. 2-3 hours is too much time to spend on one company, unless you’re confident that the company is serious about hiring.
Projects can work great for smaller companies, but are basically impossible to use at larger companies.
1 quick culture fit one, basically do they even like what our company does? Do they say anything offensive? If not, they continue on.
Then a 2-3 hour take home (this is self-limited on their end). Once it comes back we give a 45 minute technical round that’s focused on really high level questions about technical knowledge and about their take home specifically.
If both of those go well, they go onto the final where we ask work-background questions. Basically trying to understand how they did in recent jobs. Finally if we’re feeling good we call some references for like 2-3 minutes each and unless something comes up (you’d be surprised how often something does) they get an offer!
I think you’re right about the small/big company thing. The candidate needs to have confidence that their take-home will actually be evaluated. This can be helped by guaranteeing an interview round after the take home is done but even still I get the point that this would be hard to trust in larger companies.
I think it's stupid to try to judge if someone has seen the question before. The only time it's wrong to have seen the question before is if someone tipped you off to that specific company's questions. I think that most people are not good enough at writing reasonable questions to attempt it. For that matter they are not good at picking reasonable questions for an interview out of a collection of problems either. People often choose problems that are excessively difficult, ambiguous, or even impossible to answer.
It doesn't matter if it is 'stupid', or 'wrong', or whatever other cope you want to invent, people will do it and if your caught in a lie because you do not even know the answer to that, you've disqualified yourself immediately and potentially get blackballed as a liar.
If I've caught such an immediate lie as an interviewer, I'd be a bit relieved on some level because I now have a legit excuse to end the interview series early and go do something else and save my coworkers from doing interviews, because for most interviewers, they are chores.
You would probably fail in an interview with me because you assume things that simply not stated. If someone says "I have seen this before" that does not imply that they know how to solve it. They might have seen the question and decided it was not worth their time, or they didn't actually solve it, or whatever. You CANNOT infer that they are lying if they follow up with "I don't know (or remember) how I (or anyone else) solved it." People have fallible memory. In a high pressure situation anyone can get mixed up, misread the question, etc. So, don't be a jerk.
>It doesn't matter if it is 'stupid', or 'wrong', or whatever other cope you want to invent, people will do it and if your caught in a lie because you do not even know the answer to that, you've disqualified yourself immediately and potentially get blackballed as a liar.
It's so trivially easy to get disqualified, that's stupid. If they really push you, you can say "Yeah I think I saw it a long time ago and I don't remember the solution. You decide if you want to switch." And that is probably the truth in most cases anyway. If someone would disqualify me over that then they're not my kind of people.
As for whether it is a "cope" to observe that these questions are counterproductive and pushed by a lot of smug and incompetent copycats, I think it is worthwhile for one's own sanity to recognize that solving leetcode questions is a separate non-work-related skill. Being good at those questions does not make you a good engineer, and vice versa. Yet, in some cases, your future may be decided by these pseudo-academic timed exercises, judged harshly by baboons.
>If I've caught such an immediate lie as an interviewer, I'd be a bit relieved on some level because I now have a legit excuse to end the interview series early and go do something else and save my coworkers from doing interviews, because for most interviewers, they are chores.
I think what you're really saying here is that you would rather hire a good liar over a non-liar, assuming they have equal leetcode skills. Because that's what you are selecting for if you don't allow people to comfortably say "I've seen it before and I don't recall the answer right now."
A local tech recruiting company asked our Python user group to try out their new "fun" programming challenge used to evaluate and rank candidates.
We had an hour. I didn't like the problem, so ended up helping and chatting with others in the user group. Then after 45 minutes I realized there was a game theoretical best solution easily implemented using recursion and memoization.
While I was able to implement it in time, it did not win.
Further analysis later showed that the game play was too short to rank any strategy reasonably. More specifically, the winner was strongly based on player order, not strategy.
In other words, the tech recruiting company's new evaluation tool was total bunkum.
I wrote it up and let them know. I don't know what they did with that product.
In any case, it took me more than 35 minutes to solve.
It was one thing to practice this for hours, and now I have to tell them, yeah I saw this problem, so given me something that I never solved, and have a 50-50 chance of win.
So working at Meta would be like, oh we need an Auth layer, but we have already seen an Auth layer being used, So lets scrape that, and build a new one, but then we are not going to look the spec or the problems from before, because we want to do the same things again.
Sure, if the course is very poorly designed or the student is very unmotivated, they may end up just memorizing everything while somehow avoiding understanding anything. But in real life, when someone says "Oh I understand it, just give me thirty minutes to solve it" and others "shit them out" in 5 minutes, it's usually the shitters who are ready to advance to the next level.
No it's not. I'll make it simpler. If you know the material really well but haven't memorized the problems, you will test poorly. If you memorize the problems and have no real understanding of the material, you will test well. This is obviously the opposite of what you want to test.
If the interview for a coding job doesn't involve actual coding, how do you know that the candidate can code? What you may get are people who are just really good at selling themselves. Maybe a good fit for the sales department, but not so much for the technical position you are hiring for.
LeetCode is not perfect, but no test is.
As for the "memorization" aspect. You can certainly memorize solutions. But you can't just memorize every character of every solution and regurgitate it perfectly. You will need to make some generalizations, just to fit everything into your brain, and as you type it back, you will probably misremember something, and have to fix the bug or bridge the gap. Those are useful, real life coding skills.
LeetCode problems seldom resemble real-life coding scenarios, which in theory is what you are supposed to be most concerned with.
Why not present them with a real problem that you actually had to solve for your business? And ask them to walk you through how they would try to solve that? Perhaps as part of a take-home assignment?
Leetcode test or nothing is a false dichotomy. Accepting it is lacking without attempting to look into alternatives doesn’t seem logical.
>You can certainly memorize solutions
You can most certainly memorize the high level steps to LC problems. In fact I would argue that this is already the status quo. You may still be able to learn how they would approach a problem this way, but if they are good at “selling themselves” they can make all that “seem” real as well.
A FizzBuzz coding test - which is also imperfect - will weed out those who cannot code, and without the false negatives from LeetCode.
Programming is far more than writing code. Do you test their documentation skill? Their ability to work with others? To fix someone else's code? To identify and resolve ambiguities in the spec? To estimate development time?
Is the additional time needed for a LeetCode test over a FizzBuzz test worth the time taken from evaluating these other factors?
And it tests more useful skills than FizzBuzz. I wouldn't recommend hard questions, as they usually require techniques that are not very useful for most jobs and that you have to specifically train for (ex: dynamic programming). But for easy to medium questions, in my experience, the bottleneck is usually misunderstanding the problem, and simple bugs like typos, off-by-one, reversed conditions, etc...
Understanding of the problem relates the the ability to read a spec correctly, and the ability to identify and resolve ambiguities is related. And if you can fix your own bugs, it also helps when fixing others.
Of course, it doesn't test everything, particularly not teamwork (I mean, it is similar to competitive programming, the opposite of teamwork), but it was never meant to, other parts of the interview can do that.
As for documentation, I think writing skills are important and undervalued. I mean it in the traditional sense, like writing novels. We could have candidates write essays, but like LeetCode, people will complain. I will, because I suck at this, but I understand the value. By the way, I find that LLMs help a lot for this, language models, are, after all, really good at language, especially technical writing that is more formal than creative writing. Of course, the LLM require guidance, but once it gets the technical part right, I find the writing is pretty solid.
I too did competitive programming. A team working together to prioritize from a list of programming tasks which cannot all be completed in time even by the best team is a far cry from a LeetCode interview problem.
I also did a mock LeetCode-style evaluation, which my company made me do as a benchmark. It was a lot like the kind of competitive programming I mentioned. But I don't count it as an experience because it was not a high-stakes, stressful situation. Note that I had no say on the recruitment process, they just asked me and other employees to do the test to see what to expect from someone of our skill levels, it also included some other, more knowledge-oriented questions.
[0] https://en.wikipedia.org/wiki/The_purpose_of_a_system_is_wha...
At this point I think it's a self perpetuating system, like medical residency sleep deprivation or 'accrediation', which incidentally makes changing hospitals a 6 month BS process for any doctor.
A certain segment of engineers voraciously and autistically defend the leetcode interview since they were selected by it and probably like competitive coding and they are exhausting to deal with. The sane reformers eventually give up and leave them to their empire of dirt since nobody gets promoted for changing these things in big tech companies.
Creativity gets in the way of leetcode... Leetcode requires focus; you need to recall only the most relevant techniques for the problem, if you're being too creative and see too many possible solutions and you try to identify the most optimal one, it will slow you down and you will run out of time.
I tend to do better if the problems are more difficult with more time given. I'm built for solving difficult problems without hard time constraints. I'm bad at solving easy problems within limited time slots.
LeetCode was never about LeetCode, it was always a stand in for culture.
It's now a signal for baseline compliance. That's generally good for companies that require mostly operationalists.
The problem is that anyone can learn to leetcode. If you're interested in doing something new and not just warehousing CS lawyers, you're gonna have to ask better questions than that.
I think the problem is that everyone thinks they can ask better questions and almost none of those people are qualified to judge their interviewing competency or have data to back themselves up. Every company has its own special interview techniques that it's convinced will lead to the best possible outcomes when those only demonstrably lead to the current staffing. To question any of that means pointing fingers at pretty much everyone and that's a political non-starter.
Fact is that no job can give a reasonable test for how it is to work there short of working there. There's team dynamics, developer / project fit, etc etc. All you can ever do is measure some proxies. Leetcode is just a much better proxy than the old "how many ping-pong balls fit in a school bus" questions.
I’ve never been good at tests in school. I probably averaged Cs on tests through college. Projects though? Aced them every time and sometimes got pulled aside and told I was highly exceeding the projects of others in the class. I just do well when I can think alone, or when I can actually work toward a solution with a coworker.
I’ve made the companies I work for millions of dollars. But put me in front of a white board and ask me an algorithm question? You’d think I was fresh out of CS101.
1. You're a newgrad. You have no work history, but you do have a lot of time on your hands and a desire to prove yourself. Go study leetcode and prove how awesome you are.
2. You're a senior engineer who has a small network (not a dig, lots of great developers are introverted and have small networks, myself included). In this case, it sucks but... just practice leetcode. Don't worry, the fact that you're a good engineer means you'll pick it up quickly. It's not the same skill as professional software development, but there's a ton of skill transfer, and it might even be a little fun.
Time crunch, test anxiety etc, those can be overcome by practicing. A lot of times really smart people are averse to practicing since they never had to do it in school. But, I'm telling you, any smart dev can learn to leetcode well while being timed. It'll be fine
Leetcode is effectively the opposite, because each one is usually a CS paper by itself, which by definition took a very hard problem a long time for a very smart person to create and test the solution.
You cannot practically invent the a leetcode solution from whole cloth if you treat it like an IQ test question should be done. It's a sport and you get good at sports via drills until it become muscle memory.
Believe it or not, most humans on earth can study as long as they want for leetcode and still won't do well on them. If you can drill and study for them and your scores improve... bam: that's the IQ test.
LeetCode is the _opposite_ of an IQ test. LeetCode tests one's ability to memorize, recognize, & regurgitate information in a timed test.
What? No. You need to practice to be truly effective at Leetcode.
I never give these type of questions in my interviews of senior/staff+, I build out topical problems for the space I’m actually hiring for, then simplify it down to the interesting bits. I give a ramp, a simple problem, a more complex tweak to the simple problem that needs an interesting data structure (maybe needs a heap or similar) and then another tweak that forces them to abandon that data structure and do something novel. You can also fail this and still get hired.
With junior engineers, I’m sorry but I need something that looks like leetcode so I know you put the work in, and I can’t ask the topical questions because they have no frame of reference. These questions are like the common denominator for someone with no real world skills. I need to see that you’re driven and self motivated enough to teach yourself this (probably useless, when is the last time you had to implement merge sort) skill.
I also don’t think there is a great alternative. I put an extraordinary amount of work into making my questions that aren’t Leetcode, when they leak I’m heartbroken. I don’t want to just let a fancy school be the decision point, so I need to find a fair way to test. Asking people to do 1-2 days work or pair program… it’s usually caused a lot of dropping out of the funnel. So I would love to hear alternatives that are working for others
What got me a job was that I was a solo founder running a business and learning how to make the most of limited resources. Aka I was someone that gets things done and spends the least amount possible to get there. Use the resources you have and show you can create value.
Again, it all comes down to show don't tell.
Leetcode is valuable as a way to practice and maybe reaffirm skills. It's not useful for hiring in a direct way.
Also at large scale you cannot trust the entire corpus of employees since your are well beyond dunbars number, so the process is there to prevent cheater and nepotism clusters forming where random employees hire their incompetent drinking buddies or cousins after a single "lunch".
That process works for small companies, and it's an advantage they should leverage, it does not work for the large ones.
IMO you need work sample tests as a minimum, which means a coding assignment. Some people are great bullshitters at lunch.
I hope I never see you in my team.
You don't need leetcode for that. It's sufficient to talk about datastructures. In fact, that is a much better and actually reasonable thing to do in an in an interview.
> I also like these questions because you can ask them to any kind of software developer, from frontend to DevOps.
Only those that will apply at your place, which will exclude me if you require a leetcode interview. So, in fact, you will be biased by prefiltering candidates.
It's like when companies say "we only hire the top 1%" and I ask "the top 1% of all engineers or the top 1% of the engineers that chose to apply at your company?"
They solve them instantly, and making matters worse they are 100% valueless to a company.
What universe do we live in that hiring managers want staff that are 100% as skilled as an LLM but probably not as strong in areas that actually matter?
The reality everyone with a brain is going to cheat on these, they are typically pretty early in the interview process and it will hopefully get replaced with a real world test.
At my last job we just used example problems that we had seen in the past, usually REST API focused with just enough nuance to make engineers think through it and potentially refactor their code. Then you can ask them specifics of their thought process and get insight to their experience.
Going by the Think Fast Think Slow, most of the people who were great at what they did, could be categorized overall, in terms of how they put in their effort, where they put in, and when they started from. (Apart from luck)
I think of this in terms of traits. Big tech companies, they would want to avoid any form of engineering silos, giving so much power to a single/single set of engineers, that would halt their business. Which is why they have this process.
They want a whole lot of easily replaceable engineers, who (might work on different projects), but overall have the same kind of mentality. This way they know, if the candidates have done what they need to do, and talked about things a certain way, they on a fundamental level would be the same. So even if the person leaves, the style of work for the next incoming engineer wouldn't change, and everyone can go along with minimum friction.
Also, thinking a bit more, /^[FAANG]*/ companies would obviously try to cross-hire. Same as BIG4, same as /TCS,CTS,Infosys/ and likes. A good chunk of people who work in MNC's stay in MNC's. Infact sometimes the projects are transferred from one MNC to the other, and then the manager or TL or whatever leading the project is hired first, and then he/she brings his/her team from the previous org.
Although end of the day, there will be a few hits and misses, but given the plethora of people working there, on a large scale of things, it doesn't really matter. I mean in a group of 20 people, 2 leaves. Its barely anything to make a dent.
Sounds like confirmation bias to me..
I would disagree with this premise. Leetcode identifies people who have just finished cramming for Leetcode questions. You don't need logical reasoning abilities to solve Leetcode, just encyclopedic knowledge of algorithms and data structures
In all the tech phone interviews that I've done in my life I have NEVER failed a LC/algorithm related interview. My secret is to try to engage genuinely with the problem. And even when I haven't been able to solve a problem I still would get passed to next round bcs I'd be on the right track.
I still don't know what ppl are complaining about. LC have a correct solution and a good interviewer will pass you if you are on a good track. They are also relatively easy to prepare for. Now compare to actual BS which is the requirement to have done work in a particular track that the company is hiring for.
Now that's total BS, I can't get 5 years in python in like 2 weeks before interview no matter how motivated I may be for the job.
We lack an agreed upon, specific way to evaluate someone’s talent in 30-90 minutes. LC problems are not a terribly efficient way to fix this.
The reason big companies use these interview methods, is because they have to interview tens of thousands of candidates per year. None of the common alternative interview methods that get tossed around can scale to thousands of applicants.
Asking standard array/string manipulation/sorting etc questions in a 30min phone screen is very valuable to save your engineers 5hrs on a poor candidate. Conversely, throwing an NP hard leetcode hard at a senior dev with 20 years of experience and excellent culture fit with your organization in the 9th interview is basically meaningless.
Worse than meaningless, it's a great way to toss away a diamond in the rough.
Leetcode problems skew towards competitive programming and grinders. They do absolutely nothing to show real-world programming skills which involve: 1) working within a large, existing codebase, 2) strong code documentation and commit message habits, 3) understanding of coding styles within the company culture so that you don't write in an unintelligible, bespoke style, and 4) communication skills and the ability to work within a team so that the candidate is not fighting strong headwinds.
I enjoy that this assertion is made with essentially no attempt to justify it, it's just said matter-of-factly as if it's just obvious truth. Well, let me speak at least for myself: Absolutely not.
A lot of them require you to make, what to me at least, is some non-obvious clever observation about the problem. Sure the problem is talking about a guy robbing houses, but if you stand on your head and squint just right you’ll realize this is actually a graph cycle detection problem and you should use Floyd’s algorithm to solve it.
Because there are thousands of these problems the amount of time it would take someone to become familiar with them is prohibitive. So you are at the mercy of the interviewer, have they picked a super clever one, are they going to be ok with removing duplicates from the answer by tossing stuff in a set or do they want you to pull some dynamic programming out of thin air.
It’s the part where you have to divine the trick under pressure that measures nothing of value. I’ve been a professional software engineer for 2 decades. I’ve had times when I’ve been trying to solve some very tricky problem and done research and thought thoughts and come up with pretty clever solutions, or at least I think they are clever. Not once have I had to do this under pressure in a 45 minute time box with someone looking over my shoulder.
That’s my objection to leetcode. Sure it’s great if a candidate can recognize that your riddle about topological map rain capture is actually just a restatement of Kolger’s postulate at first glance (a problem and postulate I’ve just made up because I’m not going to wade back into leetcode right now) but that’s an insane thing to optimize for.
The vast majority of the problems programmers solve are actually just mapping business domains into code. The most common problems that need solving is taking squishy, incomplete, and contradictory requirements from multiple stakeholders and figuring out what needs to get done. People in the real world are rarely rolling their own data structures, because the red black tree you slap together is going to be infinitely worse than the battle tested highly optimized one you can pull out of the standard library or off your package manager.
In my long career I’ve had a handful of occasions to actually build a data structure or solve a problem with some very clever algorithm. And in those cases you don’t really want people shooting from the hip anyways, you would want them to do research and see what prior art exists so they can discover something like Floyd’s algorithm for finding cycles in a unidirectional graph (ok this one is real).
It is not clear to me what exactly leetcode tests. My best guess would be your ability to take a disguised questions and convert it into a handful of problem shapes and solve those. But if you grade leetcode like the website does during an interview, expect to lose a lot of perfectly fine candidates along the way.
if even low parameter LLM can solve most of these leetcode examples that cost nothing to run, why are we using it to discriminate applicants to measure "how badly do you want the $250k/year"
It's almost like the people hiring are optimizing for masochists and there lies the devil, you want hyper-rationality because they are the easiest people to manipulate and brainwash.
Highly sensitive and type B personality aren't going to sit around and let them trade their life to destroy or trap large number of the population with addictive algorithms or drone target selection.
Matter of fact IQ was invented specifically to filter out the most programmable minds, you wouldn't be able to run a country's intelligence agency without being able to brainwash people or a corporation. The promoters at the helms are usually the exact opposite of the producers, unwavering, unforgiving, uncompromising.
Being quick at the command line, using the right tools, can get around VS code quick, can solve the more trivial code problems quick (or quickly critique LLM output or a blog post). It adds up.