Algorithms Interviews: Theory vs. Practice
danluu.com
danluu.com
Link to the question: https://code.google.com/codejam/contest/2437491/dashboard#s=...
The gist of it is: You are given 4*N points on a 2D plane. Can you draw two perpendicular lines to separate them into N points per quadrant?
I think Dan's missing context is that google code jam questions always have a small and large dataset. In this case, small is N=10 which makes a lot of bruteforce solutions possible and not much different from any other bruteforce puzzles common in interviews. Being a geometry question is the more unfair part if this is a generalist role (but not unfair if your role involves graphics, computer vision, self driving car mapping, etc).
Expecting a solution for the large (N=2500) is ridiculous of course. See analysis: https://code.google.com/codejam/contest/2437491/dashboard#s=...
For example, I was asked this question once: Given N points and 3 squares of side length L, what's the minimum L that allows you to cover all N points with the 3 squares?
I think this question is fairly unreasonable as an interview question, if you weren't given hints. In the context of an interview, however, I think it could be reasonable.
I was really not into doing it again, it was so traumatic.
The correct curse of action is for the interviewer to anticipate this and prepare a backup question.
I think people who say this just don't get what's really going on here. If you look at these types of interviews, a big part of what they select for is: 1) Some sort of problem-solving skill that's a mix of raw intelligence and/or ability to solve problems by pattern-matching to things you've seen before. 2) Ability/commitment to work on something that may not be that intrinsically motivating, in the context of getting/maintaining a certain type of job.
These interviews select exactly for that. To pass, you usually have some mix of: - raw intelligence. - ability to pattern match to similar problems you've seen before. - ability and motivation to spend time preparing for these types of interviews, even if they're not really what you care about doing.
That's really what they're trying to capture. It's not a perfect filter (you will still have some false positives and plenty, plenty of false negatives), but it works "well enough".
You really only need one Dan Luu per like 10 or 100 engineers at a FAANG. Most people aren't going to be optimizing at the level he is, they're going to be doing work that's mostly a mix of problem-solving by pattern matching, and ideally, they're motivated enough to have that job for as long as possible.
I say that as someone who has gotten a job offer from every interview I've been interviewed, and done maybe 100 on the interviewer side of the table. I put far more weight onto pair programming and anything that resembles a work sample than anything else. And even then you don't really find out what people are like until like the tenth pull request or so.
Anyway, yes, interviews are prone to bias. So are work samples (in different ways). Pair programming can be better but logistically harder (i don't know any companies that have been able to do them at FAANG scale), but even then, like you said, you'd need a lot of it to really judge.
At the end of the day, the entire hiring process needs to work "well enough" and with reasonable cost to both company and candidates. For many FAANGs, these interviews (plus hiring committees, bar raisers, or something else) meet that criteria... Though honestly, they often fall short on diversity but that's a whole other topic.
Where I work today, I'm in the last round of interviews, to try and reduce the time consumption.
My real point is that a work sample is really hard to get in an interview, and it's really not fair to fire someone in their evaluation period because they're only average, and not extraordinary, with what they come up with in day to day work.
I remember a blog post which was ranting about the current culture at trendy big tech companies (has since been deleted [0], probably the author still wanted a career at such a company) in which a quote resonated with me a lot: 'we're not problem solvers, we're problem endurers'.
Someone solving hundreds of leetcode problems to prepare for an interview just signaled that they are willing do do basically any tedious piece of work you throw at them.
EDIT: [0] Found it: https://web.archive.org/web/20160308032127/https://medium.co...
As a manager, I want people who are hardworking, sure, but how I can I know that they're working on their assigned tasks instead of studying up for interviews and trying to jump ship? Also, I want to work with people who are creative, empathetic and collaborative (the latter is a hugely important trait perhaps even more important than raw algorithmic interviewing ability in a highly teamwork driven discipline like Software Engineering) - none of which these kinds of interviews help with. Or maybe I want a mixture of people with different strengths and weaknesses, like a team with an algorithms expert paired with a creative problem solver that can figure out new approaches to business problems. Applying a uniform standard involving mostly rote memorization and practice of the same set of interview puzzles to everyone is in my view antithetical to building a strong engineering organization. But you're right that it works "well enough" and no one has found alternative solutions that work better so it is unlikely to change in the near future.
and
> This also happened in another way. As is common at a lot of companies, managers were given a team-wide budget for raises that was mainly a function of headcount, that was then doled out to team members in a zero-sum way. Unfortunately for each team member (at least in terms of compensation), the team pretty much only had productive engineers, meaning that no one was going to do particularly well in the zero-sum raise game. The team had very low turnover because people like working with good co-workers, but the company was applying one the biggest levers it has, compensation, to try to get people to leave the team and join less effective teams
are pretty important points to note.
Accepting that organizations are incentivized to extract the maximum amount of value from you means you should be: 1. Aware of the value you produce 2. Know how to sell it
This also comes up during promotion season in places that require that you perform at 50%+ of the next "level" to be eligible for a promotion. The unsaid part is that you will be under compensated for that 50% "next level" and when the promo comes, you'll be at the bottom of the comp band at the new level (no wonder people tend to quite shortly after a large promotion)
There is no objective criteria for the next “level”, despite the fact that corporations pretend otherwise. Level is more related to years of experience than actual skill from what I’ve observed. Working extremely hard for a promo and then being compensated far less than external hires who can barely code but managed to pass the leetcode interviews is a real morale killer. And that’s if you get lucky and get promoted instead of passed up for promo, since promos tend to be given for exaggerating impact “metrics” instead of actual total impact which is unmeasurable.
From a director-level perspective it probably makes sense to have one or two of your best people on each team, rather than some wholly excellent teams and some wholly mediocre teams.
Perhaps that is true at start ups, but in my career as a developer at very large corporations this has been almost never true. Most of my career has been waiting around wasting time or working on side projects to do my job for me. That time waste is due to any of the following:
* Unnecessary build jobs, such as a build job that takes 90 minutes. In that extreme example Java was the center-piece technology at a web travel company even though Java is not a web technology. As a front-end developer if I wanted to make a code change and test it in the browser I would need to rebuild and wait 90 minutes, because the front-end code that the Java developers don't care about is at the end of the funnel. No incentive to hurry and certainly no maximum value extraction.
* When I was working as a strategy consultant to a different giant web travel company I was there because they sucked. Let's be honest, if they were awesome they wouldn't be spending huge bags of money on an outside consultant for advise. Because the most sucky developers there were supremely insecure and their management was at war with another department they didn't really want me to do work. I really tried to justify the money they were paying to my employer for my time there, but it was really just an exhaustive exercise in keeping busy so I wouldn't be bored all day. Eventually that faded away and I just started taking long walks out of the office. No value extraction there either.
* A more common scenario I have experienced is death by process. Its like bleeding out from a thousand tiny paper cuts such that you are in agony forever, but there is no light at the end of the tunnel. I have reasoned this madness occurs because the organization lacks a qualified architect and the wrong personalities were put in charge of the technology decisions. In nearly all cases management looked around and said to themselves "who here mindlessly works themselves to death" and then that person gets put in charge and now everybody is working themselves to death but nothing is being produced. Nothing says working hard like running as fast as you can on a mouse wheel. In these cases there is need to be constantly in a hurry, but nothing gets done and everybody knows it and morale is destroyed.
* The most common failure I encounter is fear of writing original code. As a front-end developer there are only 2 APIs on the front-end: DOM and the Web APIs. Instead of writing a fast performing solution in the fewest possible instructions you typically need to write the code to conform to a very specific, though not documented, code style on top of a giant monster framework with a 1000 dependencies. Then you need to compile the code more than once just for good measure. Even though the problem could be solved slowly in 30 minutes with an additional hour for testing in multiple applications/environments it will likely take a week or more in the framework. Sometimes people claim there is a great hurry to ship product changes, but as a developer working on a framework made out of toothpicks and duct tape the hurry is clearly somebody's self-serving imagination.
* I think everybody's favorite value destroyer is death by JIRA. So, JIRA is an Agile planning and assignment tool that the corporate world has fallen in love with. When everything is a story point and nothing else matters your personal goal becomes completing story point assignments and then doing nothing else.
Perhaps the only times I have ever experienced a big corporate employer making any attempt to extract the maximum amount of value from me is when work used the old waterfall methodology, because the work was constantly coming in without regard for schedule cycles or administrative nonsense. The time I was most most valued is when I was told to go as fast as possible as the A/B test engineer for Travelocity and all restrictions were on productivity, aside from Q/A and defect resolution, were eliminated.
---
> 1. Aware of the value you produce 2. Know how to sell it
I, and most people I have worked with, have spent so much time undervalued that we clearly had time to really and continuously sell ourselves like a product on a shelf, but nobody almost wants to. The people that spend the most energy selling themselves instead of writing code or solving problems tend to use charisma as for bad self-promotion. If developers wanted to spend the majority of their energy in marketing they wouldn't waste time solving hard technology problems. It reminds me of end of year evaluations that most developers dread.
Do you think this is important even if one is already satisfied with one's compensation and mainly cares about the positive impact that they're making for end-users of the company's software?
What matters in the end is that you're content/happy with where your life is and the trajectory you're on - whether that's making 200k a year or having enough time off at work so you can do other interesting stuff
When I say 1/10, I don't mean 10%, I mean I have entered about 10, and only one got an offer. In my opinion, all were qualified and should have been given offers. For comparison, I've done roughly 100 interviews or something (though not nearly enough SWE interviews, because I'm in San Francisco). As an additional upwards bias, I actively discourage people from applying, so this is only the folks that said "I understand, refer me anyway".
All that said, Google is really more like dozens of independent companies now. There's Search/Ads, YouTube, Cloud, Photos, Android, Maps/"Geo", and so on. Your mileage will vary pretty drastically depending on how you're routed. Just like Dan was misdirected towards a software interview, I've had friends tossed into the SRE hiring pipeline, asked how to operate large-scale distributed systems and so on, when their background is "Umm, I just did my PhD in CS in Graphics... I've never written a networked program in my life".
To all the other Google employees: find the job posting internally that your friends would actually want, talk to the hiring manager to figure out what that maps to externally on the job postings page (it often doesn't), and then tell the assigned recruiter that you're serious about your recommendation. The recruiter may still override you in deference to their hiring targets / goals though (the one offer I've seen from my referral went this way despite my effort), so don't promise your friends that you can fix it for them.
I'm not here to excuse these hiring practices. Instead, be honest with yourself about why you want to apply, and how much hassle you're willing to put up with. In my experience, all other companies will decide more quickly on your hiring result, role, level, compensation and so on. I believe that's because of the scale of the interviewing and hiring (>10k full-time employees per year for the last few years), except it's always been pretty bad at Google.
I'd say that about mirrors my experience as well. Is this really that common? Here I was thinking that I'm just an idiot.
The gist of your comment might be correct, but I think we shouldn't perpetuate the mindset that mid-40s or even 50s is "so old."
In almost all knowledge-worker professions, age and experience is acknowledged as an asset. I think software companies are just starting to realize that, as the "move fast and break things" companies are now weighed down under technical debt and imposing hiring freezes on recent grads and junior engineers.
I feel like I'm in my prime in terms of experience and productivity, and though I may not want to work 70+ hour weeks or chug beers at the office anymore, I'm a considerably better engineer than I was 20 years ago. I think many of us intuitively know that, and we shouldn't let ourselves become victim to self-defeating SV groupthink.
> but it's those damn algorithm questions I just can't get past
I was under the impression there was a minimum bar on the algorithms that applies regardless of seniority and whether you pass the interview as a junior/senior/staff/etc. was mostly dependent on communication, system design interview, etc.
What are the expectations, in an interview for this level?
[0]: http://blog.interviewing.io/after-a-lot-more-data-technical-...
[1]: http://blog.alinelerner.com/people-cant-gauge-their-own-inte...
My reading of that is that out of all the people he knows, there are four that could plausibly "fail" 20 interviews in a row, not that 25% of (engineers?) he knows would do so.
Sounds about right. I can pass "take home quiz" style interviews at nearly 100%, but put me in front of a whiteboard and everything goes blank. Not sure why people insist on doing that.
These days, it's amazing how much "We do X because Google does X" you see. But Google is the new circa 2003 M$, so it makes sense.
My standard response was "The most distinguishing property of Microsoft (Google) is they have a monopoly, copy that part first".
1) They are exceedingly brilliant and they can program decently
2) They have studied hard, they can reason and code decently
Combine that with a system design portion and some behavioral questions, and the company is probably getting a solid hire. Does it mean that you're weeding out some false negatives? Absolutely. But large companies don't care. The system works well enough, and until it doesn't, I doubt much will change.
It’s an age screen. People in their 40s don’t have time to prep for those kinds of questions. Nor do people with children who might need a work life balance.
That is you see a lot of talk about this because "the system" isn't working. Some companies are trying new things, but you have to find them among all the ones who don't spend the time to try to find a better way.
Would profit/revenue sharing help with this?
Those "I made the company X" arguments are only interesting if it was an action/idea that you were uniquely responsible for. If it's something most equally qualified people would have done in that role, it's not that interesting.
but can your business earn more if had more of them? If yes, then it's a profit center. If no, it's a "profit eater".
But that's true of everything. If you hire 100% more salespeople (generally considered a "profit center") but don't also hire a proportionate number of extra devs and IT and janitors you're not going to make more money either.
Simplifying a bit, but there's "right amounts" of all types of employees - if you don't have "enough" devs/salespeople/support staff/IT/janitors, then hiring more will result in you making more money.
Profit sharing works by trying to make employees feel connected to the success of the business, but they don't make sense based on any kind of effort/impact/reward calculus.
Too bad this rationale isn't routinely applied to senior managements/officers, eh?
I think the best course is anxiety management. But unfortunately, some people get dependent on drugs just to calm their nerves before something important. I know people that can't hold meetings or presentations without taking beta-blockers and the likes.
So, YMMV, but people are often able to be accommodating for things like this :)
I'm interested in:
a) standardization. Right now interviewers pick their individual questions and sometimes the question types. You can't apply a wildly inconsistent measure across anything and expect to have confidence in your findings.
b) interview question diversity. If people have varying strengths and our day to day work goes beyond just solving algorithm style questions (because it does for a huge majority of dev jobs) then a more diverse body of questions seems to be a better yardstick to measure others by. Examples of questions types that seem interesting to add include refactoring existing code, pair programming, small coding project and even weird things I haven't heard of before like the ability to give constructive feedback on design proposals, etc..
> When I ask people at trendy big tech companies why algorithms quizzes are mandatory, the most common answer I get is something like "we have so much scale, we can't afford to have someone accidentally write an O(n^2) algorithm and bring the site down"
In 20yeara I've never heard someone say anything like that.
It's surprisingly easy to bring down complete complex services system with a unlucky degenerate case.