Google: 90% of our engineers use the software you wrote (Homebrew), but...
twitter.com
twitter.com
Succeeding at these whiteboard questions requires weeks of preparation. You need to practice, practice, practice. After enough practice, you are pretty likely to pass. So ultimately, it is more of a test of "how much do you want to work here." If you care enough, you can pass. That said, these types of interviews do favor fresh grads over experienced programmers (you have algorithms and data structures fresh in your mind), which means that they are flawed in IMHO.
Generally, you have a problem, you research, you experiment and then you wrap all that in a nice easy to use utility class or api and forget the details because there is something else you need to work on and that problem was just one piece of the puzzle anyway.
The worst hypocrisy in all that, is if you ask a specific question to a veteran developer on a average size codebase, he will very often need to look in the code and you will not think any less of him. But if it is an interview and he does not remember some trivia in a piece of code he didn't write - my god, he is a loser.
My team does whiteboard questions, but we try to keep them practical.
Typically they are the types of problems that we'd expect new engineers to have to look into on their first day. Oftentimes the questions are less "come up and an answer" and more "let's explore this problem domain and see what we can uncover."
As an example, the interviewer will write up on the board a data structure that is too big and has redundant fields in it, and start a discussion about how to optimize that structure. The optimizations go more and more in-depth as the candidate gains experience thinking within the problem space.
Another problem, the interviewer will give an example of a synchronous operation and work with the candidate to break it up into a set of asynchronous calls.
My team tries to avoid "puzzle questions with 1 correct answer".
I do ask simple data structure questions just to test basic programming skills. On occasion someone does make it through who cannot even write a FOR loop.
My standard go to question is "Please do an in order print of this already sorted binary tree."
It is 5 lines of code, and ~80% of candidates who haven't seen a tree in a decade can still figure it out.
IMH the goal isn't to try and trick people, it is to try and see if people can hold a conversation about a technical topic and make valuable contributions to a discussion.
It all depends on the team though, the interviewing process is completely non-regulated here. We get training telling us what to legally not say, but other than that is a free for all.
Teams ask white boarding questions because they heard other teams ask white boarding questions.
http://finance.yahoo.com/news/11-common-interview-questions-...
1) Actual programming and a technical interview are not and never will be the same thing. Assume you know nothing. And I do literally mean nothing. 2) Start with a basic intro to programming book in a language you aren't familiar with and try to do simple tasks in a language you already know. This simulates a bit how it feels to program under pressure. Things you know you know seem not to come out quite as quickly. This helps to emphasize point #3. 3) Practice. Practice. Practice. Practice. Practice. A decent chunk of your typical interview questions exist out there. I'm not saying to memorize them, but be familiar with their solutions. Eventually you'll get to the point that an approach for solving a new problem will come from pieces of old ones. 4) Since coding is done on whiteboards, do not use a compiler. Pencil and paper. 5) Find someone to give you an honest-to-goodness mock interview.
While this may be quite different from interviews for MS Redmond/etc full time jobs, this was my experience:
In my interviews, I mentioned off the bat that I wasn't great at algorithmic stuff. Now, don't get me wrong, I can do them, I'm just far, far behind the curve. (Students, esp. CS students, in India spend lots of time on sites like SPOJ and CodeChef solving algorithmic questions; I prefer to do stuff in open source). The interviewers were understanding. I got asked fun stuff like "what do you enjoy about [project I work on]". "Walk me through some large problems you grappled with on [project] and how you solved them". I got to have great back-and-forths on these things. It wasn't easy explaining nuanced and intricate problems, but I was able to give them a rough picture which I think is what they wanted. Shifting the focus from "can he solve this puzzle" to "what work has he done in the past" was great.
There were also algorithmic-y things, but not the usual kind. For example, one interview focused on designing and optimizing a build tool. The idea was that we had a bunch of components, with dependencies on other components, and I had to efficiently generate a viable order of building. There were some other constraints which I forgot. I realized that the problem was similar to a garbage collector (not that I knew how to model that either :P), and did some sort of mark-and-sweep thing. Then, after being asked by the interviewer, I optimized out some for loops and stuff.
I didn't need to know any algorithms beforehand. Sure, if I had known how `make` does dependency management, or had read up on garbage collectors, I would have done it faster. But I didn't need to know anything, and it worked out.
My fellow interns had really rigorous algorithmic interviews. They were okay with that because they were good at them. It's understandable that "solve this puzzle" is the default since not everyone has real world projects they've worked on (students). I don't like the status quo, but I don't think the interviewers will be intent on maintaining it either. If you have stuff you've worked on that you'd prefer to talk about, I think they'd like that.
If you think the interviewer will be open to it, I suggest mentioning it. They will know you're not from CS; you can mention that you don't have much practice with algorithmic stuff.
Major YMMV though. Also I've done very few programming interviews so the advice here could be totally misguided. :)
What I mean to say is, I have tended to use existing libraries that might make use of binary trees, but that I have never had to print or modify them. If modification is necessary, it's always through the library. So while I understand the basic concepts, I'm having trouble understanding why it would be a useful interview question.
Do developers at Google and Microsoft actually manipulate binary trees on a daily basis? Like, ALL of them? I can understand that they would be a common thing in the bowels of some code, but surely not all developers in either company would run into them on a regular basis?
Have a friend do a mock interview with you.
That said, memorization, in my limited experience, won't be that helpful, because the questions will likely come with a twist that requires that you understand the algorithms and are good at adapting and applying them - and you need to be able to do it in 45 minutes on a whiteboard. It is very difficult.
I did find "cracking the coding interview" to be helpful prep, and really, the hard problems are fair game. Again, memorizing won't really help you, instead, view these as the kinds of problems you need to be able to solve in 45 min at the whiteboard.
I am kind of blown away that people can do this. I went home and did one of the questions I remembered from my interview with a compiler, and it took me several hours (though I didn't look up a solution, I did use the compiler constantly to check logic, and so forth, something that isn't available of course if you write on paper or a whiteboard). I suppose might be able to get this down to 45 min at a whiteboard, but it would take me a lot of time to get that sharp, months or more of study and practice.
None of this is mean to comment on whether this is or isn't a good interview process, it's just advice on how to prepare.
Yeah...that is memorization. I've had questions like that before and the problem isn't adapting. Its the fact I don't have everything memorized. I have no problems taking an algorithm off a book/Google and implementing it in 45 minutes, even with some kind of twist.
The problem is I can't remember things I haven't used in 10 years so expecting me to do anything with them in an interview is kinda pointless. Sure, I'll remember the purpose of the algorithm, but I'm not going to remember the algorithm implementation details.
I didn't get past the technical interview at google or amazon, so you might want to take advice from someone else, but here's what I think I'd need to pass the algorithm section of the technical entrance exam at google.
You must be able to do the following in about 5 minutes or less at a whiteboard with no errors:
1) print all permutations of a set 2) build and print a binary tree (preorder, postorder). 3) create a hashing function 4) quicksort and mergesort
(I'm leaving out linked lists, arrays, stacks and so forth because you won't be able to do 2 or 3 without them anyway).
The reason you need to memorize these (yes, I'll go ahead and use your word now, I agree with you that this is memorization) is that you can't be dealing with how to implement these if you're going to do enough in 45 minutes to pass the exam. I think that these three things form the foundation for most variants on recursion, sorting, traversal, combinatorics, and so forth.
Next, you need to be able to identify when something is a variant of these algorithms and solve it very quickly at the whiteboard. "Cracking the coding interview" is good practice for this, but in this case, I don't think "memorizing" these questions and answers would be especially helpful, because you're unlikely to get that exact question.
Here are a few of questions I've invented on the spot (not from my interview, not from cracking the coding interview) that you'd probably want to get close to solving in 45 minutes or so. In other words, if a buddy wanted to apply to a top tech company and asked me to give him some practice, I'd give him 45 minutes at a whiteboard for each of the following questions.
"In a binary tree with positive and negative integers, find and print all paths with a positive sum. Now assume the tree is ordered, and make your code more efficient."
"In an nxm matrix, find all sub square matrices with a positive determinant."
This might just be me, but I was kind of struck with just how much progress and accuracy you are expected to achieve at the whiteboard. And like I said, I am kind of blown away that people are able to do this. While I can pose those questions and develop a mental model for how to approach it, I just couldn't solve them at a whiteboard in 45 minutes to anything approaching what I believe is the standard.
Again, whether this is a good way to filter people out isn't something I'm discussing here, this is just my advice if you are planning to apply and want to be prepped. Also, there will be more than just data structures and algorithms on the exam.
That is why I stress its memorization because without memorizing things you've long since abstracted to libraries, you really can't perform in the stressful interview situation to the level the interviewer expects.
If, however, you take something the interviewee implemented in the past 6 months as the "standard problem" they could probably do it without much trouble.
You spot patterns but there's so many different ways they can ask things and things that you just remember from studying it, that I imagine I'd do worse now unless it was close to what I've done since. On the spot I couldn't invert a BST right now.
I suspect you could probably pass a whiteboard interview a few weeks after taking a mooc on data structures/algorithms with enough depth for example.
No computers. Pen only.
For me, rehearsing an answer works out. Then, I try to see how to deconstruct it and apply it to a similar problem I don't know.
I can deliver an answer sounding confident and competent because I practiced an answer. It's a bit of a crap shoot.
What irks me the most is that people are doing the interviews. They can be fooled and persuaded with tones and intonations without them realizing it.
It's not. I hate whiteboarding as much as you but I try not to think of it that way. This method is just a way to increase your chances of succeeding. I've not succeeded in every interview, for sure.
I do what I must to press on. :(
The best thing to do is not to take the rejection personally. I always ask, as nicely as possible, what areas I should focus on to improve for the future.
Sometimes I get no response but I often get replies, good replies.
We do what we must.
That's true. I have other features that reduces my chances of getting hired, but doesn't eliminate them. Still, I wish I had someone else's mind.
I think you made an excellent point. It seems that outside a tiny minority of world-class experts, Google is more interested in hiring interchangeable workers.
Can they, though? Obviously they believe they can, but are they correct?
Google's not going away any time soon, but they could easily slip a long way down from the top of the heap. There are signs that's started to happen.
I don't think they can be nearly as casual about losing top candidates as they seem to believe. In five or ten years we'll know, I suppose.
I also know a few "top talent" engineers that rejected the google offer because they didn't like the process and the people they met. I know even more ex-Googlers that quit within the year of starting there.
It's not an issue for them? From where I'm looking, most of the experienced top talent (that it's not working at google already) is keeping away from it, in greener pastures and most won't look back.
Yep, google is not going anywhere anytime soon, but I'd be just a bit concerned if I were them.
Like I said, Google is still an amazing company, but it is by no means the only one out there. Talents won't come to work for it just because it is Google, without great compensation and interesting projects.
It never worked, what do you think happens when you start asking these kind of questions in interviews? People just go out and spend insane amount of time memorizing solutions to interview questions on the internet.
They continue the same after you have hired them. So their on the job productivity remains low. Please note, they need to prepare for their next job in the current one.
So the only thing this results is in people spending insane amount of time preparing interviews doing very little work in their day jobs.
I never realized that some people put as much effort into practicing interview questions as I put into learning how to actually write software.
Does this mean technical interviews, even a structured interview, are completely useless? I.e., if you follow an interview script, eventually candidates find out about your script, and then it's useless.
Good teams have a balanced skill sets I've found. You need some uber-algorithm guys who can pass questions like this, but you also need engineers who can fill in all the rest of the tasks that make up a production worthy system. I have not found that the uber-algorithm guys always pay attention to detail, put in good comments, think of all the edge cases, communicate well with outside teams, etc.
Those types of engineers are what you consider a "bad hire", and won't be working at Google.
What's to keep a good engineer like you describe from buying a couple of books and spending a couple of months working on becoming the type of person that can do well on these questions?
Google doesn't make everyone multi-millionaires anymore. If you want me to jump through a bunch of hoops for you, you'd better have a pretty solid case for what the outcome will be.
By actively selecting against people who don't want to answer bullshit questions, they're definitely turning away creative, talented people. They're selecting for different people instead. Perhaps that's what Google needs to feed the machine now that the company is mature, so I won't necessarily fault them for it. But that filtering or selection IS happening, even if they don't think it is.
In practice, they select for people with good memorization skills. If you can remember the details on a ten dozen different algorithms and data structures, you can pass one of these without having a single lick of creativity or skill.
I say this as an employee who has worked alongside many unskilled drones who made it past the algorithmic interview process.
Infatuation with "coding competitions" and high flux intelligence tasks is often an anti signal.
Sure, what they demonstrate looks good at face value, but unless you attach a micromanager over their tasks, they'll constantly deviate to the new-shiny every week instead of staying on task for the long haul.
I know even YC has been bitten a few times by accepting people who are top-performers in very narrow contexts (e.g. people obsessed with winning coding competitions every single weekend) only to see they can't do anything coherent for more than a week at a time.
Just because someone is great in a narrow context doesn't mean they can carry that over the months and years needed to deeply iterate (and suffer through the low points) on successful projects.
Basically, imagine that there's two ways to graduate college with a CS degree - you're either the kind of person who's smart enough to get things if you work your ass off for them, or you're a genius and you coast through. If you're a genius and you work your ass off, you get a doctorate or found a startup, and aren't in the candidate pool. So you wind up with a choice about what to compromise on: work ethic, or smarts.
Ideally, you compromise on whatever makes the least difference to your organization, compared to your competitors. Hiring smart/motivated people out of non-traditional backgrounds is a great option for this.
They're more interested in how you approach real world issues (the questions I got asked were conceivably real-life issues a company like Google would face with its products). If you can solve that issue by applying efficient and well understood data structures and algorithms, then that indicates you understand the problem space and solutions that may apply.
It's not like they get you in a room and ask you to draw a linked list or a binary tree.
And yet the OP was turned down for not being able to invert a binary tree (based on limited information, etc etc.)? What do you feel that is, if not a "whiteboard this algorithm" question?
As for many of the other companies who try and emulate Google - this is exactly what they do.
Your source is a series of salty tweets.
And in either case, my original comment was directed at the idea that "algorithmic interviews provide false negatives, but not false positives".
You say that as if it's a good thing. Do you expect that most people could be e.g. effective teachers, effective marketers, or effective taste-testers, with no previous experience?
On the other hand, knowing that DFAs are equivalent to regexes are equivalent to NFAs is already a step of theoretical knowledge that any "Just let me write code" so-called "dev" is likely to complain about being asked for.
I too don't remember much.
Regardless, I only meant to make the point that I disagree with the comment "They're more interested in how you approach real world issues" and that this statement is false: "It's not like they get you in a room and ask you to draw a linked list or a binary tree"
edit: by "expert" I just mean something I'm particularly skilled in, not something I'm an authority in from theory to practice.
I was later declined. Had a couple of friends at google ask the interviewer. He said I was declined for not finishing my data structure.
Their interviewing process was quite frustrating. I'm fine with getting declined but not under those circumstances.
That was my first technical interview over the phone so it was pretty obvious it was related to that single interview. One of the Googlers I knew that checked the system said the same.
I had the impression that they all had to agree for a hire. Hence one no means no hire.
Obviously he didn't give 1 shit about "how you approach the real world issues".
Once you get to that point, though, if they asked me "and how would you invert the tree in-place?" I'd answer "well, first I'd make sure we aren't just using a tree structure that already knows how to invert itself, or could easily be extended to know how. Then, assuming that didn't pan out, I'd google 'in-place invert binary tree' and read three or four solutions that already exist to burn the semantic structure of what I'm about to write into my head so that I don't subtly screw it up and have to fiddle with it for the next hour when I could be coding something else. Then I code it."
And that should be enough.
I had two phone interviews - the first had quite a few questions about my prior experience (over a decade) and how to approach real-world problems from a high-level, including some that I had worked on at my current position. The second was primarily purely technical, but did have some more theoretical/project-management-y type of questions. In neither one was I required to write any code.
On-site: one interview was entirely whiteboard-coding (in a collaborative style), on a set of related generic problems. Next was one with a set of problems related to my experience. Third was almost entirely telling war stories. Last two were some generic graph-theory-type problems and some simple problems involving data structures relevant to the my area of experience.
As I hadn't interviewed for anything in 11 years, I did spend some time brushing up on my basic algorithmic theory, and essentially followed Steve Yegge's suggestions. I honestly think I probably could have done the vast majority of it without any studying, but spending time practicing solving problems on paper (with a countdown-timer as an artificial pressure-inducing device) certainly did help me, I think, in the on-site. No amount of memorization could have helped with most of it, aside from basic knowledge that anyone who's taken Data Structures should know.
I didn't find the process to be insulting at all; it was an enjoyable challenge. Everyone I talked with on the engineering side of things I can only say the best about, and so far as I can tell the team I'll be with has some really great people. I am a more senior developer, so perhaps I got a different experience than some.
Another in the same interview got in my face (this should be easy) when he thought it was taking me too long too multiply two numbers like 637 and 41 during an estimating problem. I understand the point of most of these questions is quick and dirty estimation (600 x 40), but seriously...
637 is an awkward number, we'll start by rounding up to 640: 640 * 2 = 1280, anyone in tech knows 2x that is 2560.
Make up the missing 3: 3 * 4 = 12, 2560 - 12 = 2548.
2548 * 10 = 25,480.
25,480 + 7 = 25,487
25,487 + 30 = 25,517
25,517 + 600 = 26,117.
That said, I did make a screw up when calculating it, and added the 637 before multiplying by 10 (which may have been the differentiating part of that question, combined with the intentional stress, to see how you cope with pressure? Although I didn't sleep last night, which might be the cause :D). I dunno if they wanted you to do it as quickly as 600/640 x 40 though, which obviously makes it much more difficult.
Afterthought, due to sleep deprived state, quicker way to solve:
600 x 40 = 24,000 40^2 = 1600 25,600 quick and dirty, for accuracy: 3 * 40 = 120 25,600 - 120 = 25,480 25,480 + 600 = 26,080 26,080 + 37 = 26,117.
Sorry, this comment turned into a bit of a ramble.
It's important to know how a person approaches a hard new problem and how easily or not they give up. I'd guess that a person who didn't solve a hard problem, but got close enough by displaying creativity, curiousness and a solid thought process might have an edge over a person that implemented a familiar algorithm with ease but got a little flustered, and didn't show much creativity or inquisitiveness, when challenged with something new.
Whiteboard interviews might miss eccentrics who can't express their clever thoughts verbally or visually, but it can depend on how tuned in the interviewer is to these eccentricities or how much value their team puts on expressive skills.
Algorithm/data structure interviews, whiteboard or not, also have the added benefit of being exceptionally good at eliminating candidates that:
1) Drastically misrepresent their skills
2) Give up easily at difficult tasks
3) Don't seek help from others
4) Frequently have a bad attitude
I had them attempt to solve the problems on a whiteboard that I would give to our intern candidates. Both totally bombed hard, and our other "re-interviewers" had the same result when they asked some design-related and algorithmic related questions.
This wasn't "regurgitating algorithms". It was basic problem-solving and software design. From their track record of development we didn't believe them to be qualified (despite their own opinion of themselves), and they only confirmed it by totally bombing the "re-interview" process (which was identical to our normal interview process).
The only way to validate your highering process is to randomly accept people you would otherwise fail. Anything else is meaningless.
I recently interviewed for a security engineering position at a startup, and while I got offered the job, the interview was quite silly.
The first thing I was asked to do was to write a lisp interpreter. It was a pretty trivial task, but it left me scratching my head since not only did it not tell them about whether I would be qualified at all, but I had explicitly avoided writing parsing code since I found ANTLR during undergrad.
In the end I wrote a basic stream abstraction and wrote my lisp interpreter on top of it with no real difficulty, but it was a completely stupid question. I told them on the third interview that they weren't asking me anything relevant and I still got another question later on about how to iterate over a tree in a specific order.
Part of the issue was that most of my interviewers were fresh out of college, it was literally their first job and they had no idea what to ask besides the kinds of questions they had been asked in their own interviews.
In the end there were a lot of other issues that made me not take it, but not knowing what is necessary seems pretty typical wrt security. However, in those situations you want to have wide latitude to affect change because the thing they hire you to look at is probably not really where their problems are.
Say I pass the invert a tree question. Does that prove I am good? That I can design a product, listen to customer requirements, come up with ideas that elude others, read a research paper and turn it into something commercial, solder a circuit board, make a schedule, mentor junior engineers, write documentation? No. All inverting a tree tells them is I studied trees recently, and/or I'm at least superficially clever.
Google lets go of plenty of people. They aren't making perfect hires.
Other than to wonder why they still haven't done anything about it. (I interviewed there ~8 years ago, pre Android when they were looking for deep mobile expertise and got flunked out on a similar problem. I am not at all interested in working for them now)
IQ tests supposedly become inaccurate after 150/160, so you wouldn't really want to put too much emphasis on 180 vs 170 at those levels.
That's completely false, and anyone who says that is completely ignorant of Bayesian logic.
Here are some simple numbers.
Suppose that a "good" candidate is a 1-in-100 find. Suppose that a "bad" candidate has a 1% chance of tricking you into hiring them anyway.
Every time you pass on a "good" candidate, that is greater opportunity for a "bad" candidate to trick you into hiring them!
Counterintuitively, if you pass on too many of the "good" candidates, in your overzealousness to reject bad candidates, you're actually INCREASING THE ODDS OF A BAD HIRE.
This is management porn. "It's better to reject a good candidate than hire a bad candidate." It makes the manager feel good. "Wow! That candidate seemed smart, but I rejected him anyway! I'm such a great leader! I make the tough decisions!"
tl;dr summary
Because "good" candidates are rare, every time you pass on a good candidate that increases your odds of making a bad hire! This is simple Bayesian reasoning!
A bad hire can easily cause more friction than if nobody was hired at all. Net effect for the business is negative. You're also probably severely overestimating the likelihood of a bad hire going through.
FSK: If you pass on many good candidates, and you have a small chance of hiring any given bad candidate, then each good candidate you reject actually INCREASES your odds of making a bad hire.
Idiot: We made a bad hire once! Never again! Now we reject lots of good candidates to avoid that repeat disaster!
FSK: But, if you want to minimize your bad hire rate, you also have to minimize the number of good candidates you reject.
Idiot: NO! NO! NO! The way you make sure you hire no bad candidates is to be so strict that you reject lots of good ones! That's what everyone told me so it must be right!
In one ear and out the other. Why do people who don't understand statistics get to be managers? If you don't understand this statistics argument I made, you're unqualified to work in any sort of technical area.
Hiring a good employee is not nearly as impactful as hiring a bad one. If you can have a strategy that filters out 100% of bad employees, but unfortunately also filters out 90% of good employees, this is preferable to filtering only 50% of good employees, but also only filtering 90% of bad employees. You may have more good employees with the latter strategy, but the bad employees can kill the team.
Firms are hierarchy-based entities. "Technical Merit" is about a third or fourth order away from what really drives the hiring decision. Its also very imperfect predictor of actual performance.
The issues is that companies use the term all the time. They want people to believe they were hired for their merit, but merit is almost always 'fit' and not technical in nature. The technical hoops are just a CYA for when the 'fit' doesn't work out (mis-judgement) and they need something to point to as to that is not arbitrary in nature to explain how other people were not hired instead.
Whether or not this actually reduces the number of bad hires is beside the point. It's a classic principle-agent problem.
They fear they might be bad candidates. They'd rather reject too many than too few, so they make sure they reject everybody who they're not 100% is a good candidate. That means they may reject some good candidates that aren't easily identified as such, but it doesn't increase their chance of hiring a bad candidate; it decreases it.
But the cost and other effects of doing so vary significantly depending on where you are. Most of the US may have at-will employment, but much of the rest of the world does not.
We had a really bad hire about 2 years ago. So bad it caused us to review our entire hiring process to understand how he got in. It took almost a year before he was fired (and he was a contractor so it should have been easier). In that time, he used up untold resources while we tried to find work he could actually do, people helping him "just in case he just needed a helping hand" and so on. Finally after wasting other people's time for 9 months, a manager made the decision to get him out.
A year later I got an email from him asking for a recommendation. Nothing ventured, nothing gained I guess.
I.e., I don't know your particular situation, so I wouldn't dare to assume the reaction was wrong, but too often "we much at all costs make sure it never happens again" overwhelms any attempt at reasonable approach and cost/benefit analysis completely. Maybe getting one bad apple in occasionally is a reasonable price for quickly and efficiently hiring a lot of good people?
Why did the manager wait so long?
Employee X is hired. After a month it becomes clear to the whole team that X is a bad hire. But how do you turn a subjectively obvious feeling into an objectively obvious reason to fire. Because if that person decides to sue the company after getting fired that's what you will have to demonstrate.
Just imagine how hard it is to quantify what makes someone good at our job? Code Quality? Team dynamic? These are really fuzzy metrics.
You could document how many times the engineer affected production badly and how much that cost the company. But to be realistic even great engineers do that. Heck I touched production at Google and decreased revenue by millions of dollars and it was expected that this happens. So you'd have to let him affect the company enough times to be obvious He's worse than your other engineers.
You are going to have to have documented enough clear failures to do the job to justify starting the process of firing. If you don't have a performance improvement plan then you might be able to speed up the process a little. However not having a performance improvement plan is both a bad idea and, I would imagine (IANAL etc.), could cause problems later on down the road in a potential suit.
Taken in that light 9 months doesn't seem that long at all.
It is "at will employment".
There is no need to prove that employee is causing losses.
It's the employee who has to constantly "prove" to employer that this employee's salary is justified.
I personally have worked with several awesome people that have interviewed with Google, and none of them got hired. They all said the interview process was flat out insulting.
Interestingly enough, most of them ended up at Facebook.
Not sure if you have ever been a manager, because firing someone is NEVER easy. It hurts everyone emotionally. The person getting fired feels awful. The person doing the firing feels awful (unless they are a real sociopath). And team morale tends to take a big hit.
Here is my stance: if you hire someone who isn't a good fit, unless they actively deceived you, it is your god damn job to find a way to make it work. As a principle you should treat people with respect and dignity, and not as easily disposable and replaceable cogs.
That attitude is how bloat happens. There is a middle ground here. Not firing bad employees also hurts many people emotionally as does an underperforming, overbloated company. Employees are as replaceable as employers. To use a sports analogy, it's the big leagues and you may not make the team and often times it is hard to know whether someone can make the team without a trial. I believe these draconian interview practices and contract-to-hire approaches are a direct result of the never-fire-bad-employees attitude.
Ergo they own the problem, not so much the Noogler. Also generalists == mediocre code in my experience because there's no passion for such work in a lot of people who are fantastic with other in-demand skills, but I digress.
And when I joined, Google wasn't so much bloating as it was metastasizing. Most of the people I knew who joined around the same time have long since left. In contrast, I've been at the same gig ever since I left Google. The Google experience was the outlier.
I'm not sure if this is the norm--I believe it differs for industry hires versus those fresh out of school--but it's certainly not true that, as a rule, you can never meet potential teammates.
Because for the most part, Googlers aren't bozos (although a couple of the true believers in the Googleplex blew my mind with how crufty their skills and knowledge had become), but that doesn't rule out one of the many great engineers there promoted to their managerial level of incompetence. And that's what I had - a guy with no people skills whatsoever - and two teams with a total of 20+ engineers to "manage."
No, it isn't. This is why probation periods exist. If you hired someone and they aren't working out, then at the end of the probation period you don't keep them on.
Trying to force someone who doesn't fit the team dynamic to stay is going to hurt your org. It doesn't make you a "good" manager to say to everyone "I know this sucks but deal with it because letting them go would mean I was wrong."
> As a principle you should treat people with respect and dignity, and not as easily disposable and replaceable cogs.
Which is ironic because so many job descriptions I see today are basically written as "we want to hire someone with exactly these skills, who requires zero training, and can become an expert in our systems in their first week."
No one is that way, unless your system consists of pressing one button all day, but then the job description would probably require that the person have intricate knowledge of the button and is friends with the engineer who designed it so they know what to do if the button suddenly stops working.
If companies actually treated people with respect and dignity, I'd get the training I need to become a better employee, instead of going to management and begging them for any training every 6 months like the industry is now.
/rant
Doesn't this rather presume that your "team dynamic" is actually good to start with?
Let's say you had a team of not very good engineers. Then you hired quite a good engineer who looked at all the terrible practices (e.g. no version control, shitty or nonexistent testing, poor build processes) and said to themselves "look, I need to fix this shit or I'm leaving, and I've got ten better offers in my inbox".
They might not be a very good team fit, but that's because the team is filled with idiots who haven't figured out how to use version control or whatever.
So, you know, the best thing to do is to get rid of them for not being a culture fit or for not being good for the team dynamic...?
Yes, if you have a team like this and you hire someone who has a higher standard, there is going to be some friction between them and the existing team members. Letting that person go isn't your concern, because as you said yourself:
> and said to themselves "look, I need to fix this shit or I'm leaving, and I've got ten better offers in my inbox".
I've been in that situation before. I was hired to a company and when I got there I found out that every single day they were fighting fires because of stupid decisions management made with little foresight into how it would affect the team. Funny enough none of this was mentioned during the interview, although it was a definite red flag that they had high churn for this particular position.
I stayed there for my probation period trying to fix things so the team would fire fight every day and change management's mentality, but it wasn't happening, so I gave notice and went somewhere better.
> So, you know, the best thing to do is to get rid of them for not being a culture fit or for not being good for the team dynamic...?
No, you took me too literally. Obviously if you've got someone who has friction with the team, but they're a hard working individual who is trying to make your team better and more efficient, you should try to work through those stressful periods because in the long run it will be better for your team's health and the company's health. If some of your low performers leave during this period, that's okay, they were only going to hurt you in the long run. That being said, don't burn bridges with your existing employees. Try to find a happy middle ground that results in a better work environment for everyone.
I have also quit a job because they fired the wrong person.
In my opinion, if you fire someone and the reaction from the whole team isn't "thank god, that guy needed to go", then you made a mistake firing them and should have tried to make it work. In the past when I fired someone, the general reaction was much more along the lines of "what took you so long". In both cases I was the one at the table defending the person while everyone else said they needed to go.
Interestingly, in both cases the reason for firing them was that the person in question was not treating other people with respect. So what is worse, letting one person shit on your whole team and make everyone else feel bad, or getting that person off your team?
Managing isn't easy.
If the candidate cannot invert a binary tree and gets mostly there, and has demonstrable real world software, it's a different thing.
What does that even mean? Swap left-right child nodes? Write it linearly (n-th children at 2n and 2n+1) and reverse it as if it was a string? OP later described it as "to min-max the tree, ascending to descending." confusing me further.
That doesn't disprove what he said. In fact it strengthens it and makes missing a good hire even worse, because, as he said missing good hires -> MORE change of bad hires (that can "poison your team").
In other words, it's not a choice: "I'm better of rejecting a good hire than getting a bad one, because a bad one could poison the team".
By rejecting good ones, who're actually getting MORE bad ones. So it should be corrected to:
"I'm better of NOT rejecting good hires, or else I'll get more bad hires and poison my team".
Humans are more complex than that. I don't think you can assume that candidates will perform the same all the time. Sometimes an excellent candidate can perform badly for multiple reasons (e.g. nervousness, poor preparation, bad interviewer, personal problems, etc).
It seems to me, that rejecting a good candidate, and have him/her interview again after some time, if that candidate was a 'good-hire', then it would increase the chance of hiring him/her, since it is most likely they will prepare better, and know what to expect.
If your flawed process rejected a good candidate the first time around, what makes you think the same flawed process won't reject them a second time?
I'd never even apply to Google based on the stories I've heard, and I'm sure there are plenty of others in the same position who are even better at what they do than I am. There are just so many stories out there of how crappy the hiring process is that everyone who's any good must have heard about it. Some significant fraction will have said, "Yep, not for me."
The theory behind the "raise the bar" argument is that obviously good hires are easy to spot (this is the part most people reasonably take issue with), so people who do not pass that bar of "obviously good" aren't worth taking a risk on. The conclusion is statistically reasonable if you accept the premise (which, again, you probably shouldn't).
This is only true for new grads. The population of engineers with ten years of appropriate corporate experience (that match the technology survivor bias) and at the ~12 locations google can use is tiny.
When I look at my old corporation, only 3 people (of 500+ I would be aware of) were hired by google. One of them is genuinely great, but I doubt he passed the current process as a blind hire. The other two I wouldn't want to work with, but I bet they passed this process precisely because the right kind of focus. (I.e. everything people around them didn't know was their priority and since we normally focus on what's important to completing our project, none of that was so important..)
I was very interested in google when I was under the impression that (without experiencing menlo park on a daily basis,) I could either walk away with over $500k (gross) in 2- years, enjoy working there for 5 years, and/or work with the best >experienced< engineers in our field. When glassdoor and my own linkedin network revealed my best perspective into their setup, blind applying to them is now just a good filter to test my new age employability skills. So, now they've built themselves a new problem of true positive false positives who are just along for the ride.
The hiring market is already NP Hard for both sides when you are genuinely trying to get a good enough fit for yourself rather than the best deal on the market. If you try to misuse the other side they will see less ethical constraints in using the resources you'll have to put in.
How many developers genuinely want a job they can't do that they will have to be awkwardly fired from in six months and explain at every future interview?
Yes, but what hiring managers are doing isn't passing over candidates that they know are good, they're passing over candidates that they're not very sure about.
> Because "good" candidates are rare, every time you pass on a good candidate that increases your odds of making a bad hire! This is simple Bayesian reasoning!
Similarly, of course this is true, but again hiring managers aren't passing up candidates that they know to be good, they're passing up on candidates that they're not sure about, whom the acknowledge could be good.
Under certain conditions, every time you pass on a candidate you're not sure about, you decrease your chance of making a bad hire. The conditions are that increasing hiring standards must weed out more bad candidates than good candidates.
Let's work with your model. Suppose 1% of candidates coming for an interview are good and 99% are bad.
Hiring strategy A manages to hire all good candidates it interviews, and 1% of the bad candidates it interviews. End result: for every 100 people you interview, you get 1 good candidate and 1 bad candidate.
Hiring strategy B is more conservative and hires 50% of all good candidates it interviews, and 0.1% of the bad candidates it interviews. End result: for every 100 people you interview, you get 0.5 good candidates and 0.1 bad candidates.
The claim is something like, the team produced by strategy B is better than the team produced by strategy A, even though team B has less good candidates than team B.
1) This means that by raising your hiring bar (adopting B instead of A), you eliminate more good candidates than bad candidates. You now have to prove this empirical claim. 2) All I wanted to say was that the statements you made like "every time you pass on a good candidate that increases your odds of making a bad hire!" is only true under certain conditions (namely that raising hiring standards eliminates more good candidates than bad candidates), and it's not "simple bayesian reasoning".
As long as good candidates are rare, and bad candidates have a small-but-nonzero chance of tricking you into hiring them, it's very expensive to pass on a good candidate.
Even if you pass on 50% of the good candidates (no matter how many bad ones trick you), you're doubling the amount of time you're spending interviewing.
The only way to be sure about how many "good" candidates your are missing is to hire some random percentage of the people who fail your interview. Only a large tech company would have the resources to do that, and I'm not sure if it would be ethical, or even get the employer in legal trouble.
NO employer knows how many good candidates they miss out on, which makes this analysis very difficult.
Also, I question the ability of most employers to evaluate employees AFTER they are hired, so any analysis of post-interview performance might just be reinforcing whatever biases are present.
Also, candidates fall into THREE groups: great, mediocre, and toxic. The catch is that the toxic ones are most likely to trick you into thinking they're great. That makes it even more expensive to pass on a good candidate.
Yes, I agree with you. My point is that even taking this into account, even after taking into account all the other costs you mention into account, it might still be better to pass on candidates you're not sure about.
> The original slogan "It is better to reject a good candidate than hire a bad candidate." specifically says to make that fallacy.
So going by the slogan, even if passing on too many good candidates increases your odds of a bad hire (say from 50% to 80%), it's still better than hiring a bad candidate, since if you hire a bad candidate, the odds that you made a bad hire is 100%!
My interpretation of the slogan seems different from yours, mine is something like: if you hire a good person the value of your business will increase by X, if you hire a bad person the value of your business will decrease by Y, and Y is much greater than X.
But do you? How catastrophical would it be if Google accidentally hired not the top 0.001% but instead somebody from top 0.01%? I mean, same people that build everything else around us and the world still didn't collapse yet? I get it, you have to have standards, and you don't want to waste time and money on somebody that can't pass fizzbuzz test. But once you've moved past that - do you need to obsess so hard on ranking and quantifying it? There are probably some positions in Google that require people to invert binary trees (speaking figuratively) day in and day out, but my decades of experience in the industry shows that most positions aren't like that. I've not worked for google but I'm pretty sure it's also this way there. If you hire smart and competent people, it's OK even if they don't reverse a binary tree here or there.
"Hmmm... it looks like this guy really gets around. He's on the same project at GoodProgrammer A and GoodProgrammer B. He also tweets about functional programming and was mentioned in GoodProgrammer C's latest blog."
Nothing says Google must accept sub-par technical people, but Google fails to realize you don't have to be the top 0.0001% in absolute algorithms intelligence to do great work.
Google's basic theme seems to be "you must be low latency ultra smart in our narrow interest areas" to pass an interview (or well-connected in other ways to lower scrutiny), but Google's main problem these days isn't lack of smart people — it's lack of projects with meaning and lack of how to organize smart people into anything resembling a coherent world-moving force.
In The Plex is great read for insight into their entire process. Basically, Google is run by people who have never been told "no" to anything in their lives, so they continue to think they are the best at everything until reality forces them to reconsider their delusional assumptions ("montessori naivety").
I've been at Google for 3.5 years (in the NY office) and have never heard this. I don't think I've ever even seen anyone wear a college t-shirt/sweater.
> These aren't Googlers I'm talking about, except the boyfriend.... remember that most of these conversations happened when I was actually in college (a few even before then, while applying to schools), and I just have a really long memory
Sounds like someone who's just into remembering people's SAT scores?
I do wear my college t-shirt though. My graduating CS class was three people, so have to have a little pride there!
I jest.
But if you're Google, why wouldn't you aim for the top 0.0001%? It's not as though you have a shortage of candidates.
And even if there were such a thing, you're not going to find the top n% by asking questions from an undergraduate textbook.
And you're certainly not going to find them by pissing off people with proven practical achievements. Acting like that is evidence of anti-intelligence, not a sign of being smarter than everyone else.
It's one of those "gift/curse" things. It's good they can challenge assumptions, but they don't realize how wrong they are when they are wrong.
There are some notes at http://www.amazon.com/In-The-Plex-Google-Thinks/product-revi... and it's also drawn up nicely in the book itself.
Which is unfortunate, because objectively Google doesn't actually have a very impressive track record of creating successful products and services given its size and resources. It has a goose that lays golden eggs (the on-line advertising business), two strong on-line services (search and mail, both over a decade old) and two well-established platforms (Android and Chrome, both nearly seven years old). Other than those, most things Google tries seem to land somewhere between unremarkable and complete failure, with Google+ surely the most obvious example of the latter. They also seem to be developing an unenviable reputation for killing things off as a result, which is going to make it more difficult for them to succeed with future endeavours. Whatever workforce their hiring process is producing for them, it doesn't seem to be one that is very good at creating successful new projects.
Google Maps is certainly widespread, but it's hardly the only on-line mapping service available. They keep messing around with the UI, often not for the better. I often find the data itself and the live route planning/traffic news to be inaccurate.
Maybe it's better for those in the US, but here in the UK it's exactly what I meant by unremarkable. The local data from OpenStreetMap seems to be just as good for general mapping purposes and sometimes more accurate and/or current, and for car journey planning and real-time traffic news the dedicated satnav devices still seem to do a far better job.
I know some people get frustrated by those kinds of questions, but I'd just see it as a reverse weeding out process. If you still want to work at (insert company here), just build something valuable and have them acquire you. At that point they'll acqui-hire you and you'll leap frog all the people who got in because they had Knuth books memorized.
Any mature company should focus on finding those specific candidates - the ones who fit the company and specific team, on several dimensions beyond standard skillsrequired for the job. That's someone who will be called the best candidate in that specific company. And it's absolutely natural that s/he won't be considered "best" insome other company.
You know, there is something to be said for someone who has a good product sense, but not good developer skills. They sound very valuable, so why are they applying to a developer position? Just because you are unqualified for one job, doesn't mean you deserved that job, or that you're worthless.
Go do what you're good at.
As such, the best approach to these interviews might be to say, "Look, I could take a stab at that algo question, but I haven't touched that stuff since school, and if I needed it I'd google it. It might be more productive if we talked about what I've been doing day-to-day for the last ten years to ship successful products, because that's the main value I can bring to your company."
Google said early on they want to hire the best and brightest and that was the most important. We got the mantra that passing over a good person is better than hiring a bad one. And they were right. But they're a huge company now. They, and other gigantic tech companies are always complaining they can't find 'good people'.
But how do you make this philosophy scale? Do you say, "This person is smart but lacking some skills so we'll invest in them"?
I have no idea. :(
Google has decided that the only way to tell if someone is smart is by asking academic questions.
Even though anyone with half a brain knows that there are other ways to gauge how smart a person is... such as experience.
Does anyone really think Max can't look up how to invert a binary tree if he needs to do that? He obviously knows how to read and learn. So why is that a reason to reject someone?
Google is filled with smart people.. but their interview process is pretty stupid.
My guess is that they're unable or unwilling to improve the process but they don't really have to since they're deluged with candidates.
I think I'm really smart.
Therefore, our current interview process manages to hire really smart people. Why change it?"
Google is filled with people who can solve Data Structure / Algorithm questions in 50 minutes on a whiteboard. Their interview process selects for people whose past experiences led them to being able to solve Data Structure / Algorithm questions in 50 minutes on a whiteboard.
The extent to which solving Data Structure / Algorithm questions in 50 minutes has anything to do with building maintainable production software, developing talent, a good working environment, or even this mysterious concept known as "being smart" is up for debate.
I wonder about that, actually. I mean sure, right after they're hired, I bet they're GREAT at doing whiteboard problems. But then after a few years of real work, I bet they've coded about as many algorithms from scratch as the guy from Unknown, Inc. (which is approximately two) and they probably wouldn't even pass another Google interview without a boatload of practice.
The "can't find good people" argument can usually be solved either 1) raising the pay, or 2) increasing the labor pool (i.e. H1B, etc). Raising the pay is what is done in finance because it's harder to make the argument that the required "skills" are lacking in the labor pool. To enter investment banking, and then private equity, it often only requires a B.A. in French literature and willingness to work 90 hours.
I guarantee if dev positions paid as much as medicine or ibanking you'd have more devs. Not necessarily really good devs, but certainly people who pass the Ivy + 1600 SAT sniff test and can memorize the SAT book of Google interview questions.
The answer was transactions (database). You never forget that sort of mistake.
Many of the interviews could actually work if they broke the process down into 8 questions instead of one (how would you represent a tree? a binary tree? how would you find node X? how would you move node X to the other side of the tree?). But, the ability to accommodate understanding takes empathy and human connection, not being a jerk programmer on interview duty who is browsing HN while listing to the interviewee mumble on your speakerphone.
Never underestimate the lack of ability in the interviewer and not the interviewee. Except, the company always believes itself to be more competent than outsiders, so you're ultimately left with being condescended to unnecessarily.
They don't just stop at binary trees. Think of it more like memorizing hundreds of magic tricks that could be done using a deck of cards. The tree is more analogous to those deck of cards. The data structure remains constant but the tricks you can do keep changing.
I know of situations where candidates have been grilled whole full hours on tree traversal methods, and then asked to code up perfectly working solutions for them.
Do these people seriously think people who can invent 50 years of CS research by sheer analysis over an hour on the whiteboard, will be ready to work for them to write shell scripts? If not you are only testing memory skills.
Whatever! They asked me back but it sure is a frustrating feeling even if I tend not to appear flustered until I walk out, introspect, and realize what a doofus I was.
One thing that I'd recommend to anyone seeking any kind of job is taking some improv courses--they are pretty common in major centers and even some smaller towns, and can really improve your ability to deal with this kind of nonsensical interview situation, which tests a bunch of skills that have nothing to do with your ability to actually do the job.
Correct me if I'm wrong, but a binary search tree is a very simple data structure. It has rules, and just by it's definition, and the definition of "inverted", you should be able to come up with an iterative solution, even if it's just walking the tree and re-creating it.
It may not be the best, but it's a solution, it's a starting point.
But if you're interviewing for a job solving problems, shouldn't you be able to solve problems? Even if they aren't the best solution? Genuinely asking here, as this approach seems to have a 85-90% success rate for me.
If these problems were so reliant on individual problem solving ability, there wouldn't be dozens of books with reams of answers to these questions on the market. These questions test memorization and nothing else.
While there certainly are incompetent interviewers, I think ones that extreme are rare.
See also: overused brainteasers. I remember being asked the 5 litre / 3 litre / get 4 litres problem for my Microsoft intern interview, and honestly told them that I already knew the answer, so I wouldn't be "solving" it, but here it is anyway...
Once I notice that it's not really going well, I try to politely as possible suggest that maybe we adjourn, so as to not waste each others' time. That hasn't really ever backfired, and actually turned one rejection into an offer, but I find it's polite to respect everyones' time.
Engineering is the use of knowledge/science to solve problems, right? So why is asking someone to use knowledge to solve a problem not applicable to the hiring process?
To be clear, this isn't the same question as "why are manhole covers round", or "how would you move a mountain".
One of the challenges with Google's interview process [0] is they are very worried about hiring mistakes, and also know that the process is very noisy. As such, they interview a tremendous amount of people for hiring. In addition, they've found it more efficient to just keep a high bar (missing many good hires to keep out 1 bad hire) than to conduct more than 5 interviews.
The one thing I've wonder is - if they want the top 100 computer scientists in the world to work for them, will this process produce that? Or is there another process for them?
I think the most effective way to get a job there is to build a small company and get acquihired.
[0] I don't work at Google, but I've researched the process in anticipation of an interview.
Look at it this way. If you're a cream of the crop programmer and you're looking for a job, Google has the most drawn out and arduous hiring process. Even if you apply there and start the interview process it often takes a lot longer than the industry average to get to an offer. In the meantime a multitude of other companies have probably already determined that you are an excellent programmer and tried their best to snap you up.
Google seems to be happy with the tradeoff they're making but I'm pretty sure their process is not optimized for getting the best people in the field.
Google (and other interviewers) might as well be asking me to name the last 1000 digits of PI. The Kafkaesquen ability to recite how to do the task correctly reflects nothing about the candidate, except the strength of their memory.
It boggles the mind that the field of computer science does not have any standard way to detect the competency of a person. It is extremely discongruous for a company with as many smart people as Google to have such an absurd hiring policy.
"I see your resume says you've been writing software for 30 years, and wrote this really well known tool. Wellllll....That's good enough for me. You're hired!"
I don't support skipping interviews even for good candidates, but if you were going to it would work the other way around -- you'd value recent accomplishments over more distant ones.
People don't understand that cutting out folks who had their name attached to some big successes but can't solve a toy problem in an interview is usually a really good thing. It's a strength of the system more often than a weakness. You wouldn't believe the number of senior candidates I interviewed who had really impressive resumes but were clearly checked out from doing real technical work -- folks who obviously no longer lived close to the code and either were very rusty or just no longer had the temperament to spend time thinking through a slightly tricky piece of code.
One of the reasons not to skip an interview for a candidate you're confident is good is that a good interview can be a great sales pitch to come work for your company.
It was several hours, with many interviewers one at a time, of academic exercises. Some I could see were relevant to some of Google's projects, but that wasn't the emphasis- it was very much a "how much do you remember from CS courses?" (with a few exceptions- one had me explain why a snippet of C would crash- something more what I would expect a senior engineer to know that was more practical-oriented).
I _think_ I did pretty well (again, it was part of a bigger due-diligence-- another story altogether, so I never heard), but in the end I was struck by the fact that (a) not a single interviewer had looked at or asked about any of my publicly available work, and (b) not a single one even asked something along the lines of "what are your strengths / what do you think your contribution would be" (or weaknesses, for that matter, but that seems less important).
Of course, one of the other guys I worked with lucked out and got interviewed by one of the paypal-mafia, and ended up with a reportedly very stimulating hour long discussion about actual work, strengths, and actual situations that one would hypothetically need to handle at google.
It could be dangerous. Let's say you explain some interesting techniques you used on your GPL'd software and a Google product starts doing something suspiciously like what you showed them. You may even sue and end up making their source public after some litigation.
If they did makes copies of an applicants copyrighted work and distribute it, then they could get sued. Some companies has been know to explicitly ask applicants to bring source code from previous work places (also called industry espionage), and those been clearly intended crimes and not something that just happened to company during the process of an interview.
And ye olde "what's your greatest strength/weakness" question sounds like a terrible way to judge an engineer.
Asking an engineer what their strengths are and then to demonstrate or explain in concrete terms is how you find out their unique value proposition. I'm not talking about "I'm a hard worker" type garbage, I mean "my strongest experience is in systems programming on Linux." "How so?" "For example, I wrote a specialized TCP kernel module that works like this..."
Again, not saying that should be the whole interview- but there are some non-academic skills and experience (like managing and leading a critical system used by thousands of people) that are worth more than a few weaknesses in problems that were already solved years ago and that are easily looked up. It's how you should judge people looking to get into a graduate program, but falls short if you're trying to build teams of awesome fury that need more broad experience and even (gasp) unique experience that you don't have a ready-made question for ;-)
To illustrate, here's the kind of thing Mr. TCP kernel module should be applying for: https://www.google.com/about/careers/search#!t=jo&jid=107205...
And here's the kind of generic role they probably shouldn't be: https://www.google.com/about/careers/search#!t=jo&jid=34154&
i based my comment on the only thing I could find: https://wwwx.cs.unc.edu/~duozhao/entry/2014/12/binary-tree-u...
this is only slightly harder than the trivial (if u know recursion) "mirroring" a tree problem
Of course, the irony is that for vast majority of the people they hire, most of the work they do will be building and not much algorithms work. So then what will happen is either you have people who are great at building things and force them to suffer through the interviews or you have people who are great at algorithms but not that interested in programming.
Of course, you might also have people who are great at both, but I wonder how many of them there are..
Only to bad interviewees.
You'd be shocked to hear the number of impressive résumés I have seen who turned out to be absolute no-hires after 20mn on a white board. By "absolute no hire", I mean the kind where some of the interviewers tell me in the debriefing "I will quit if we hire this guy".
Not all excellent engineers have impressive résumés but they all share a common characteristic in my experience: they all really enjoyed working on the problems I asked them to solve on the white board. I even had some of them be disappointed when we ran out of time because we were having interesting conversations about the problem.
That's just a self-reinforcing selection bias - you hire people that are good at these kinds of problems/enjoy them - you filter out the rest by default and it becomes ingrained in the culture - it doesn't say anything about the suitability of the candidates you filtered out. If you said "we hired a guy who had a great resume but he couldn't do algorithmic problems, turned out to be a bad fit for the company" - that would still be a single data point but at least it would be relevant.
Hmm, at least in my experience it's really quick and easy to tell if someone is bullshitting about their resume or not by simply talking about the different projects they worked on. I know a few developers who didn't go to a 4 year school so they don't know all of the correct terms or algorithms (one even interviewed for Google and failed) but I would hire them in a heart beat because they are really awesome.
I still feel some sort of pair-programming exercise or something similar is better than simply trying to solve problems on a white board. Unless you're in a SCIF you're going to have access to the internet to refresh your memory, etc so I've never found utility in asking academic questions. In fact I've had a few people who could pass the academic questions but couldn't build an application to save their life so I don't even bother anymore.
IIRC, a Portuguese proverb states that paper accepts whatever is written on it. One should always take a self-assessment with a grain of salt.
Without running a `git blame`, the interviewer cannot say how much of the code is from the author and how much is contributed by someone else and even then asking questions is an important part of any interview.
I'd take the 30 minute interview route and risk not hiring a clever programmer.
If a guy has some kind of a git-mutating program that makes it seem like he writes good stuff, he's probably a good programmer to be able to write such a thing.
If they can't talk about what problems they solved, they probably didn't solve any.
Investment banks recruit bright young graduates, putting them through a tough selection process (c.f. all those hard-working interns we keep reading about), and select the ones who perform/conform the best. The rationale for bringing in young (as opposed to experienced) people is that they'll work their asses off, can be moulded to fit the culture (i.e. independent thinkers aren't necessarily sought), and (perhaps most importantly) are too junior to present a threat to the incumbents.
If Google's going down the same path, it may be a sign that their corporate culture is solidifying, which would be a bad sign, IMHO.
For the top elite of experienced engineers that may not be the case, as they will come with some overall good analytical skills that Google wants. But for most in our industry, what we have learned over the years is how to be good in various sets of tech around our platforms of choice... and that is mostly irrelevant to Google, as they have their own platforms.
For big companies that want to scale hiring process, there's really not much you can do.
You put difficult questions and you hope that bad engineers will not be able to solve them. Then you absolutely have to commit to not hiring the guys who can't solve the assignment, otherwise it's all for naught.
But then it occurs to me that they won't have seen them in -this- role acting -this- way for -this- film/play/whatnot and that's what the director needs. And given the amount of money and risk involved, it's probably wise to check first...
I should have turned it down. I've written on this thread already what a career-churning waste of time my brief stint at Google turned out to be because I was blind allocated.
Fortunately, the skills that got me noticed by Google were in demand elsewhere then (and, ironically, in demand at Google now) so I just exited the building when it became clear HR had labelled me a troublemaker for trying to find a position that made use of my talents rather than opt for the career reboot they seemed to expect of me. Great perks, but the 2nd worst gig I've had my entire career.
I quit after a month or so. Later I learned their bait and switch tactics were S.O.P. and the company mostly existed to fund the CEO's various mistresses.
Are you judging this on their ability to invert a binary tree? People here are ranting about having a problem with interview practices not representing whether or not you are good at your job. That's presumably (IMO) why the tweet was posted.
It's not that you're not _good_ at it, but you're not able to demonstrate under time constraints, lack of references, lack of iterative code/compiler/REPL feedback, and while being stared down by someone whose default mindset is "why am I wasting my time on this person?"
It's never 1 reason why you get rejected, it's the total score.
That depends on how far down the interview process you are. Often they'll cut you off at a phone interview if you stumble. Sometimes they'll cut in-person interviews short if they think you're not worth it. So, they don't always get a "total score" to judge against. It's often just summary judgement off subjective feelings.
well that is the interview. for you being good means to know how to invert a binary tree if you have access to a compiler and internet, but for Google it means to know it off the top of your head. they could let you have access to compiler and then make the questions much more difficult. which one is better? I don't know. first one takes less time at least.
Often they'll cut you off at a phone interview if you stumble
I'd guess that's because they have so many condidates to pick from.
Exactly. It's the Big City Dating Problem. There are so many people to pick from, nobody wants to compromise to build a relationship. Just leave at the slightest sign of something not being 100% what you want and on to the next one since there are 20,000 other acceptable people within walking distance.
if you have access to a compiler and internet
It's not exactly that, but you often don't realize how many micro-checks or micro-confirmations you do while programming. In the absence of any "is this okay so far?" feedback, you're left second guessing yourself and reading your solution redundantly to check for any errors your daily automation would catch immediately by default.
Because in my personal experience - as both an interviewer and a hiring committee member - what you said is completely wrong.
> they don't count a lot of factors while rejecting a candidate, you make one > mistake on their particular set of questions, you are out.
Completely false. I have personally seen many people get hired despite doing poorly in an interview or making mistakes.
The hiring committee looks at all of the available information - resume, recommendations, and interview feedback when making a decision. There is no "one-thing" that will make or break a candidate.
> They don't even look at your resumes until all your 4 interviews are completed
I have no idea where you are getting this from. All interviewers receive the candidate's resume prior to the interview. I always review the resume's to get a sense of what the candidate has done so I can ask appropriate questions.
Also how do you know he didn't fail a great number of other things they evaluate?
It seems to me like creating and managing a repository of more than 4800 contributors is a much better signal of a candidates software engineering, communication, and human skills than a couple of people's opinion of a couple of hours of white boarding.
Then why don't they? I wouldn't consider "the best people in the world" those that can pass an advanced CS class.
I'm starting to realize why Google is absolutely terrible at any sort of customer-facing product.
Esteemed schools of music probably produce dozens of guitarists "better" (in any way you would measure it academically) than Jimmy Page or Jimi Hendrix per year who go on to a life of producing nothing of lasting worth.
I'll take the guy/girl who has actually proved they can produce something useful every time over someone who is "better" but hasn't.
It's the difference between an holistic view of talent, and an academic and rather narcissistic insistence on specific limited social signals that are believed to correlate with intelligence.
And this really matters in practical ways. A lot of talented devs will be reading this story and wondering if they really want to work for Google. Negative PR like this is incredibly damaging. Consider the opposite - if Google could say "Yes, the Homebrew guy works for us." How much value do you think that would have had?
Long term, the really smart and inventive people start to stay away. And then you get something that seems to be happening to Google already - declining product quality, a poorer user experience, and diminished reputation. You know - Google's record of hits recently hasn't been that great?
So anyone who thinks Google is fine because it still has thousands of applicants is missing the point. Instead of being the kind of place where Hendrix plays, it's in danger of becoming the kind of place where there are a lot of people with a lot of solid but uncreative technical chops, and no one is making cool and interesting music any more.
One of the reasons why it's so powerful is because it gives lazy journalists a stream of easy reaction quotes. When else in the history of journalism has there ever been a private company that was so heavily promoted, regardless of the media source? And even then, the react quotes are poor quality, single-sentence shit.
This rant on twitter is absolutely meaningless unless you have a heap of context to go with it. And because twitter is popular, it's dumbing down public discourse along with it.
/rant, that I couldn't have fit in 140 characters.
1) Most companies (for legal reasons) don't tell candidates why they weren't offered a job. Maybe it was because of the binary tree question, but maybe it was for some other reason.
2) Homebrew is a Mac-only product, so the likelihood that 90% of Googlers use homebrew is very low. Moreover, Google does not track the software its employees download onto their laptops, so there's no way they would even know the percentage.
So when I was declined at Google I knew people who worked for Google. One looked up my profile in whatever system they used and the other simply asked the interviewer why. OP may have done the same but as you say it's unclear either way but knowing is still possible.
> 2) Homebrew is a Mac-only product, so the likelihood that 90% of Googlers use homebrew is very low. Moreover, Google does not track the software its employees download onto their laptops, so there's no way they would even know the percentage.
Most of the people I know at Google use Macs so I don't see why not. I don't know what the majority of them use but a good chunk at least do.
As far as tracking he could simply be tracking it himself and comparing his results of known Google IPs to employee counts. Not as accurate but might give a rough estimation. He may have further analytics based on data collected (if collected) by each machine.
job: here is your macbook pro, welcome! you: <install homebrew>
EDIT: Seriously, what am I missing that I'd need as a developer? My dayjob is on a Mac Pro writing desktop software and my hobby at night is on a MacBook Pro doing the same; it's on neither machine.
Personally I have the Android SDK and NDK, QtCreator, Qt5, cppcheck, cloc, mongodb, node, wine and others. If you are not using a variety of 3rd party tools to produce/automate your work then you probably don't need to worry.
Having all those kept up to date for me saves me so much time.
I didn't realise that they could be kept up to date with homebrew?? Is that the case?
Last time I looked at MacPorts (and perhaps homebrew) I saw it as a porting of BDS/GNU tools to OSX and therefore making the OS more Linuxy to my mind. I wanted a clean break from Linuxland when I moved to OSX, hence I removed MacPorts etc. (Additionally the ported software all ran under X and not Quartz so it didn't really fit in; also, the fetched packages and compiled systems were massive in size which I didn't appreciate on a laptop that shared space with recorded audio and video etc. so space was at a premium).
Granted Linuxbrew isn't nearly as prevalent as Homebrew, but Macs are prevalent at Google.
http://www.goldbeck.com/hrblog/employee-references-germany-v...
I did get a code review out of it though, and it did point out a few real issues, so I'm ok with it.
The OP has written some great apps, sure, but there is a huge difference between writing a package manager for Mac (among other Mac/iOS apps and utilities) and writing incredibly complex, highly performant algorithms for say search indexing, machine learning, ai, etc...
In that context, knowing CompSci basics like Binary Trees (usually taught to pre-major CompSci students) has a lot of relevance. It's clear Google was looking for a Computer Scientist here, not just another developer.
I am also put off by the extreme display of non-professionalism here. It's like the OP took this personally as an insult. "I've written popular things, how dare you not hire me if I want to work for you!". That's not a professional I'd like to work on a team with. Rather, that's a throw-back to the "Rockstar" programmer that nobody wants to work with.
This boils down to a person who was personally offended a company did not hire them at a drop of a hat, as-if he was "blessing" Google with his presence.
I'd say, Google (and other companies) are better off without this type of ego.
They made the proper choice to weed this individual out.
The OP is the type of person that, six months from now would be demanding special treatment because "he wrote Homebrew".
Writing a package manager (been done before, a lot of times) is far different from writing machine learning algorithms (new field, blazing the trail for the industry). One requires an engineer, one requires a scientist.
The OP flamed out because Google was looking for something he was not. Instead of taking it like a professional, he decided to retaliate.
The tone he took in his post is appalling, and I cannot believe HNers are defending him. "I'm special because I wrote something popular (and make absurd claims about 90% of people using it), so you better hire me or else" is what he just laid down.
That's not the type of person I'd want to hire or work with... and I'd wager most would agree.
The entire interview and hiring process is conducted on mostly pure speculation of how a future hire will produce, and mesh with the team/company.
Why do we call them "speculations"? Well, because those were your believes and most likely not the true depiction of who Max Howell is.
Have you worked or interacted with Max at all? I bet you not. If you did, you will see that he's not the egotistical and arrogant person that you depicted. [0]
To me, Max was just really frustrated and venting on the fact that Google passed on him because he failed in a whiteboard session of an algorithmic question, disregarding the fact that he has been proven to be a great developer who can ship code and a track record of teamwork and tremendous contribution to the open source community. And you know what, I am pretty sure Max can solve that whiteboard question with a little bit of preparation. It's unfortunate that Google interview process is designed to judge people on a simple algorithmic question. So, Google's loss on this one IMMO.
While I completely disagree with you on most of your judgements, I do agree with you on one point, that is, it's probably not going to be a fit (scientist vs. engineer). It's okay for companies to screen out experienced developers/engineers on academic knowledge, even though it's silly.
PS: I am not sure where the "90%" came from, but it's not "absurd" to believe considering Google's standard equipment issue has been predominantly Macs in recent years[1].
PPS: Max applied for an iOS developer role not a scientist role.
[0] https://github.com/mxcl?tab=activity [1] http://bgr.com/2013/11/28/mac-chromebook-google-employees/
And google does not care about real solutions (Grsec/PAX) and did not spend a dime on it. It's an evil hypocritical corporation.
I mean, a package manager is probably far more rooted in computer science than machine learning (which is basically just applied statistics: software engineering edition).
That said, I don't think Homebrew is one of these. Always seemed like a quick fix for an OS X deficiency to me, and its competitors MacPorts and Fink being mediocre themselves certainly helped with its popularity. Not to rain on the achievements of the developers, but still.
Blazing the trail in machine learning, ai, etc... that's new territory where there are no knowledge bases or places you can go to source information about how to do it.
That's a different ballgame, and requires a scientist, not an engineer/developer.
Moreover, I'd heavily dispute that there are "no knowledge bases" for machine learning and AI. There very much is, given that the fields have a history as long as most contemporary computing. "Machine learning" in essence reflects the transition from AI being symbolic to statistical, so there's plenty of reference material on the latter and the various intersectionary disciplines like computational linguistics, data mining, computer vision and so forth. Hell, OpenCV - there's some good source information, for instance.
We cannot just assume that since he applied for iOS something at Google that he's the best fit and what they were looking for.
Maybe the tools Google needs built are very CS heavy (hence the "Build us a Binary Tree" question)? Maybe during the interview his arrogant attitude was on full display? Maybe the interview team dug up past episodes of explosive arrogance like this very one we're discussing right now?
Even if they were looking for someone that matched his profile exactly, his post-interview display of non-professionalism is sure to hurt his chances of a re-interview anytime in the future (and quite possibly at many more companies than just Google).
He conveyed several things with this display, none good;
* he's incapable of handling rejection
* he feels entitled
* he feels he's better than everyone else
* he's unwilling to admit his own shortcomings
None of these are good qualities.
If that's really what they intended for him to develop, then why not ask questions about machine learning or AI?
What facts are you talking about? The "facts" coming from half the parties involved? The "facts" coming from an upset and irrationally thinking person who's caught up in the moment?
- The guy made Homebrew
- The guy interviewed at Google
- The guy was not offered a job at Google after his interview.
Anything else is unconfirmed and, speaking as someone who's thrown quite a few frustrated hyperboles onto the internet, sounds like frustrated hyperbole.
Frustrated hyperbole should not be taken at face value as fact; sometimes there's truth in a smaller version of what's said, but not always.
For example, there is no fact established that Google hired him because he can't "invert a binary tree". In actual fact, we don't know that Google even asked him to invert a binary tree (at least not specifically). It could be that they asked him a question that he thought was as irrelevant as academic datastructure exercises.
And we don't know that his answer was the reason he got turned down either. This is the part of the hiring process (and really any human interaction) that takes the maturity of recognizing that people and their motivations/reasoning are more complex than we reflexively flatten them out to be.
Additionally, I think people have a mis-perception of the skills required to work at Google:
> writing incredibly complex, highly performant algorithms for say search indexing, machine learning, ai, etc...
...
> Google was looking for a Computer Scientist here, not just another developer
This is not really the case. The large majority of positions are for basically "just another developer" to maintain an internal Java tool, or some web UI, or whatever. They are complex in that they have many dependencies and often lots of legacy code, but not all to different from Microsoft, Apple, Facebook, or any company with a large code base. In my several years at Google, 0-5% of my time was spent coming up with complex algorithms.
As for why management continues this interview process: probably because they can. Lots of people apply to Google, and they have a high acceptance rate for offers extended. There is also a historical aspect to it: "we have hired this way in the past, and it worked, so why change it?" and fear: "if we lower our standards, the company culture is at risk" The problem of low-level SWE discontent showed up on management's radar around the time that I left, so I'm not sure if they are doing things to change that.
Getting rejected from one interview at Google isn't the end of the world. The right and mature thing would have been to go home and figure out the problem that you got stuck on.. and then try again. We all go through this process, why should we expect anyone else to get a free pass?
I recently watched an online game of chess between a grandmaster and an anonymous individual.* The grandmaster got caught off guard and got lured into a trap. Instead of getting defensive, he sincerely thanked the other individual for a great game and spent the next few minutes after discussing the intricacies of the trap.
Shouldn't someone as accomplished as the creator of Homebrew feel the same way about a problem he got stumped on?
* https://www.youtube.com/watch?v=Voa9QwiBJwE&feature=youtu.be...
Nobody wants rockstars in teams, because rockstars are bad team players. That's the reason why they're rockstars. Nonconformity (sometimes) leads to innovation. Like it or not, a lot of the tools and libraries which we use every day and on which the internet runs, was written by lone-wolf 'rockstars' too weird and too eccentric to work in large corps.
You don't want your whole army to go over those mountains, you send scouts. Scouts are terrible soldiers, but they are quick to find a path between the trees and rocks and they come back with new information, new ideas and concepts of how to optimally win the battle. Then and only then you bring the whole army and win.
A company, imho, should always have a small number of rockstars - impossible characters, crazies and lunatics with universe-sized egos, who have a track record of successful projects in the wild, working on insane things, alone or in very small groups, carefully managed by someone who can earn their respect - usually a lone wolf turned general.
They might be terrible programmers, they may have no education, but they have this unusual ability to see into the future and come back with interesting ideas and prototypes, which the 'soldiers' - people with solid fundamentals - can polish and transform into profitable products.
If a company innovates 'from the top', then it's trend is going to slowly be downwards towards irrelevance, because true innovation comes from the trenches and is done by the rockstars nobody wants to work with.
That being said if I asked this as a warm-up and we'd spend the whole interview trying to get that done then of course my verdict would be No Hire. As an interviewer it is not my job to look at your GitHub profile - instead I am assigned an area to check and I try to come up with the best understanding of the candidate in that area. While failing to reverse a binary tree is a total failure in algo/ds you can still be hired since there are several interviewers (if you make it to onsites, that is), each probing a different area.
My biggest problem with Google style interview process is that it's easily passed by folks who already passed it in a different company. After having interviewed hundreds of Google's candidates I moved to another big company with the same interview type and the experience was really surreal. On my algo/ds interview I got asked slight variations of the same questions I was asking myself - and over time I've seen some totally brilliant, unexpected solutions. Must have been a strange experience for the interviewer who got his questions each answered in 3 minutes tops. I also made it a sport to solve each one in a different language because why not. Not sure though about validity of the signal the company got from this interview.
I've interviewed hires before, and if I ask for a github, I'm going to take the time to review some of the code, otherwise why would I ask?
It is a terrible question.
First, you can't invert a binary tree (as in flip upside down). If you did, you'd end up with multiple roots and since all binary trees are rooted, you'd no longer have a binary tree. It'd be a tree, just not a binary tree.
If the questioner meant mirror a binary tree (swap left & right at each node), then it is a no-op. You do not need to modify memory to do it. Left & right are just labels for two pointers. You can just swap the order your variables in whatever struct defines your node and cast it against your existing memory (or change what your accessors return in a derived class or create a reverse iterator or however you want to implement it) and there you go, a "reversed" tree with zero overhead.
Either way, it is a terrible question unless you wanted to see if someone understood the difference between how a data structure is drawn on a whiteboard for humans vs. how it actually works. Maybe they were actually asking that question, but that seems highly unlikely.
And if they actually meant for you to recurse down and swap left and right on everything, it would dramatically lower my opinion of them because it would make me wonder if they knew the difference between how a binary tree is drawn on a whiteboard vs. how it is laid out in memory.
Just because someone wrote on Twitter with limited number of characters that the problem was to "invert a binary tree", does not mean there were no clarification from the recruiter.
> And if they actually meant for you to recurse down and swap left and right on everything, it would dramatically lower my opinion of them because it would make me wonder if they knew the difference between how a binary tree is drawn on a whiteboard vs. how it is laid out in memory.
If you're thinking about making a real structure, I believe you should recurse down and swap. If you don't do this, rotating, joining etc, could be a real pain.
Sorry for the elementary questions. I'm bad at algorithms and just trying to get a grasp here.
This is the graph theoretic definition of a tree.
a --> c <---b ^ d-----|
While only the person asking this question knows for sure I'd assume that was the intention.
It's not a _terrible_ question - it's just a recursive equivalent of FizzBuzz. You would treat this as a way of making a (good) candidate gain some confidence. And that alone is extremely important since the more at ease the candidate is the better the signal that you're getting. What I was trying to say there was that if I asked that as a warm-up and didn't get a satisfactory response very very soon I'd realise that your man has no clue about recursion and would move to the main question, choosing one that does not involve it. It would be something easier than I'd ask otherwise and would be closer to real-life scenarios - like first solving a simple problem, then parallelising the solution and correctly managing concurrent access.
Inversion is a transformation that maps directed graphs to directed graphs. Binary trees are a subset of directed graphs, so applying inversion to them is not unreasonable. That subset is not closed under inversion, so you can get results that are not binary trees, but I see nothing in the question that implies that the interviewer was asking for the output to be a binary tree.
That may even be one of the points the interviewer wants to see the candidate address. Since the output no longer has a single node from which all other nodes can be reached, in addition to just inverting it they may want the candidate to mention the need to have some new auxiliary data structure to keep track of the multiple root nodes (and perhaps note that in the inverted tree each node only has one outgoing link, so if we are inverting in place each has room for two, so we can use that now available second link space to make a linked list of the roots, so we don't need any extra storage for the new root list data structure).
Be that as it may, if the interviewers hand you a reasonably detailed specification of what "invert a binary tree" means and you can't whiteboard it, I don't see how you can expect the job.
If you're expected to know what it means, and get no hint, then that is just stupid. "Whiteboard up an implementation for something we refuse to specify, beyond citing its name."
[1]: http://stackoverflow.com/questions/9460255/reverse-a-binary-...
Note that in the original tree each node needs storage for two pointers. After inversion only one is needed. You can use that now unused storage to link together the multiple roots to solve the leak problem.
But note that only consumes the second pointer storage in the roots. Interior nodes end up with some free storage after inversion.
Since inversion is reversible, the binary tree and its inverse contain the same information. Does this mean we should only need one pointer's worth of storage in the binary tree nodes?
Is there something for binary trees similar to Knuth's xor trick for doubly linked lists?
The binary tree inverse specifications/implementations I have looked at preserve the information by selectively using the left or right links. For instance, given:
P
/ \
L R
L R
\ /
P
FWIW, I find Facebook turning down Jan Koum in 2009 and then spending $19B to acquire his company even more ironic.
many projects i have worked on over the years required very little knowledge of algos.
Docs/Drive does a lot with operational transforms and CRDTs, ever since Wave was merged into it (c. 2011). Remember that all Google Docs are collaborative.
All logs analysis & experimentation relies heavily on statistics, as well as online algorithms for reservoir sampling, quantiles, and a number of other statistical quantities.
Machine-learning is pervasive, throughout Shopping & News (it's all over these), GMail (spam, priority inbox, categories), Search, Ads, self-driving cars, Brain, and many other areas of the company.
Chrome and Android both do a lot of traditional OS & systems programming.
The main product area that didn't use algos heavily was Google+, although even this had some interesting stuff in stream ranking and search.
The real reason, as near as I can tell, is because it allows management to not think as hard about allocating talent (the mythical "fungible engineer" nonsense), but whether it even works is another thing entirely.
I don't want to take away from his accomplishments as a programmer - I use Homebrew too. But my point is that it's very easy to see "Good programmer, of course he should get hired" from the outside, while the reality is that it may not be all that similar to the tasks he'd be doing.
Macs account for something like 8% of the total marketplace of all PC's. For developers, they account for something like 20%... another 20% on Linux, and remainder on Windows or other.
So even if somehow having a paid Google employee work on Homebrew seemed advantageous, it would only benefit 20% of Google's staff, and 0% of the company itself (all Google servers are Linux).
Not sure what's the status of that ban (nor I care), but will skew the numbers enough to invalidate your point.
It might except ex-googlers in this thread of stated Google "banned" use of Homebrew internally. So the net benefit to the company and/or employees remains small to zero.
This is off topic though, since he was not being interviewed for homebrew development.
I assume that the dev was referring to Googlers' personal laptops with the "90% of your devs use Homebrew". It's not unreasonable to estimate that 90% of Google engineers use a Mac as their personal machine, although 90% using Homebrew is probably an exaggeration. A lot of Googlers use their corp machine as their personal one (they're not supposed to, but it's hard to police when all you do is surf the web), and the majority of them don't have side projects outside of work. I didn't install Homebrew until after I left the company, since before then all my code belonged to Google.
Objective - the process avoids human bias (this guy was in my frat, therefore is awesome).
Inclusive - someone doesn't need extensive past knowledge in a particular area of coding to do well in the interview.
Risk Averse - Avoids false positives.
Relevant - it properly tests the abilities needed for a coding job, and not irrelevant ones (like the ability to write an impressive resume).
Scales - this will be used for tens of thousands of interviews.
Easy-to-do - this will need to be done by hundreds/thousands of engineers, who would probably rather be coding.
It's easy to poke fun at what is perceived to be a flawed process. It's much harder to propose a solution that satisfies the above requirements. Google has done extensive research on this topic and has done remarkably well with it compared to other companies of similar size.
How can you make the interview relevant without knowing what the position is? I was asked the typical CS-type questions in my interview, but the team I ended up on required no theory.
How do you define false positives before you know what the candidate will work on? A superstar in one team will be a dud in others.
And let me add another bullet point to your process wish-list: gives the candidate a sense of whether they want the job. This is impossible when the interviewer is a random engineer from an unrelated team, unable to speak to what the candidate's work life will be like. A Google style process gives candidate very little information.
I would instead propose something very old-fashioned: teams hire for themselves. The usual reply is that this results in an "inconsistent hiring bar", but so what? Teams have different requirements and need engineers with different skills, so why shouldn't the hiring process reflect that? We are not fungible engineering units.
edit: really good english skills
Inverting a binary tree is pretty easy. It is not quite as trivial as FizzBuzz, but it is something any programmer should be able to do. If you can't do it, you probably don't understand recursion, which is a very basic programming concept.
This isn't one of those much-maligned trick interview questions. This is exactly the kind of problem one may have to solve when writing real software, and though you may never have to do this specific thing, it is very related to a lot of other things you might do.
I run a small software company and I very likely would not hire a programmer who was not able to step through this problem and express a pseudocode solution on a whiteboard.
If so, they were consistent in disqualifying him in spite of his awesome track record, fame or "street cred" but if not, I can't really justify or defend this type of ceremonial or bureaucratic gotcha interview questions as they are just useless and can be abused by the interviewer to reject applicants they dislike for any reason or whim they might have at that moment.
But my point is this is not a bureaucratic gotcha question. If you can't do this task, you don't really know how to program well. Sorry but that's just how it is. It's like failing FizzBuzz.
There is this culture of crappy software that has happened lately, especially in the Web world, and it is really quite lamentable. I believe that a very large positive impact would be made on the world -- due to the extreme prevalence of software these days -- if more people would take seriously the idea of software creation as a craft with a very high skill ceiling, and work diligently to improve their understanding and their skills.
It is even kind of reasonable to just substitute a call to some black-box is_divisible_by() if you can't figure out how to do that test ...
I, and apparently many others in this thread, spent some time trying to ferret out an answer to what it would mean to invert a binary tree: at first thought it would imply a collection of nodes all pointing at their parents, which seems not very useful (and would require more than 10 lines of code to whiteboard reasonably).
Much of the discussion is about whether or not the problem was described in sufficient detail or not.
No, I can't do it (don't even understand the question) but I certainly understand what recursion is, and can solve problems and make things work far more reliably than many of the more academic programmers I have worked with.
Here's an example to think about: You're asked to write a method that, given a word and a book, will return the number of times that word appears in the book. A logical way to continue would be to (after clarifying what a book is represented as and things like that) write a simple method that iterates over the book and counts how many occurrences of the words there are, and returns that. Is that a good answer? No, because you failed to ask questions like whether the method might be called many times for the same book (in which case it'd be better to build a map of words to occurrences). Or maybe the interviewer would say that it'd only be called once per book, in which your solution is perfect. The bottom line is that coding isn't as important as the way you approach the problem to the interviewer.
The best analogy I've heard for this: If you wanted to judge whether a person was a good chess or card player by seeing just a couple moves or hands - you wouldn't just look at the couple moves/hands played - it'd give you very little information. But if you asked the player to explain their whole thought process for the moves/hands, you'd get a much better indication of how good or mediocre that player was.
Although I do see this as something you could possibly do "for real". Sometimes you'd rather change your existing data once than continue carrying forward what amounts to hacks, which is what inverting the output would amount to. Reversing a binary tree would be what you'd run in a tool to migrate all of your data over from the old format to the new format.
Pretending you understand or blindly assuming what the interviewer meant is more of a red flag to me than requiring a few rounds of clarification.
In my opinion, an ideal interviewee would communicate that they don't fully understand the requirements and possibly offer a brief set of possible interpretations. Once the interviewer has clarified, my ideal interviewee would restate the clarification to demonstrate competence and verify their new interpretation of the requirements before proceeding.
If this was really about "reversing" the tree, as you mention, the question seems more likely to address how the candidate approaches the situation. Like, he should start by making sure they both agree on what the question actually means.
Once that's out of the way, it seems relatively easy to come up with a naive solution, without having memorised any algorithms. It seems more like a case of brainfreeze to me, which can be sort-of fixed with practice (which in turn many candidates refuse to do: the dreaded "If I have to cram for the interview, I don't want the job" statement.)
So maybe he really wasn't a good fit for Google, despite apparently being a rockstar developer. Hey, startups need rockstar devs too.
> Oddly enough, Google shows me interview questions where "inverting a binary tree" means something quite different — for example, flipping it upside down, and making the left leaf the root, the right leaf the left leaf and the root the right leaf.
Whaaaaa? I can't find this, but it seems like such a weird operation. Got a link?
The guy writing the tweet (not necessarily the interviewer).
https://wwwx.cs.unc.edu/~duozhao/entry/2014/12/binary-tree-u...
They are seemingly less interested in seeing how you solve problems and work through the process of software.
I would be more than happy to see this process change, I am just not sure what it entails.
It isn't a difficult problem, even having never seen it before. This is one of those warmup problems to test how comfortable a candidate is with basic concepts such as recursion.
I think "CS Fundamentals" are really just "abstract concepts used to teach programming", and calling them fundamentals is disingenuous.
If you're writing scripts, or JS code for web pages or something like that, then maybe you don't use CS stuff, but ... are you able to write a web browser if you had to? Are you able to write an operating system or navigational software for a spacecraft? If not, then maybe just see this as revealing sectors of your skill set that could be beefed up, rather than presuming that none of that stuff is important.
Wow. Really? Most serious people I know use other people's implementations that have already been highly optimized and well tested because they have better shit to do than reinvent the wheel.
I suppose if you want to write your own red-black tree from scratch, that's your prerogative. The last time I did was 20 years ago and not only will I never do that again, I will laugh at anyone who does it without a damn good reason.
Ditto. Those who decided to reinvent even basic data structure stuff left me a huge string of bugs to fix, which I eventually got so fed up with, that I started replacing their code wholesale with off the shelf solutions to stem the flow at my last job.
Aside from fixing an untold number of implementation bugs, the replacement caught several usage bugs as well, due to actually having some error checking built in.
We had just plain broken hashtables, "lock free queues" that didn't use memory barriers... or interlocked intrinsics... or even volatile, if my memory is correct - and not a debug visualizer to be seen before I got my hands on them, of course.
> I suppose if you want to write your own red-black tree from scratch, that's your prerogative. The last time I did was 20 years ago and not only will I never do that again, I will laugh at anyone who does it without a damn good reason.
Besides laughing, I'll tend to -1 the code review as well.
For example, your data structure is on the GPU and your data is in a texture in a certain specific format because of other reasons.
If you wrote the above reply without considering this kind of case, it probably means you haven't been exposed to very much of this kind of case ... ... which was my original point.
In any case, how would you even go about knowing whether a certain piece of existing code that you blindly pulled in from some random place is optimized for your particular use case without first being able to conceptually understand it - which is all an interview is. Nobody expects candidates to create production ready code on a whiteboard.
The idea that "serious programming" means abstract data structures or coding in assembly is weird to me. I would consider a lot of people serious programmers who, while they may know how to do those things, don't actually do them almost ever.
Some companies give take-home assignments (small projects that can gauge how well you learn). I tend to find that these are smaller companies. You can always try there.
Also, the most important thing that they don't teach you in school is that getting a job is all about connections. Email some people that you've interacted with to see if they know of anything that might be suited for you.
I reached into my way-back machine and tried to derive some approaches where I simply didn't remember the answer (and I was very open about it). I made it to call backs both times, but they declined to move forward, probably because they wanted a younger person who remembered their big-Os off the top of their head, but I was okay with it. I told all the interviewers I had fun and I did.
Even if I had made it in, I'm not sure I would have taken the job at those times. So my lack of motivation ended up turning what could have been stressful into a fun look at their hiring practices.
However, I can see for people who are really dead set on working there, it can be a harrowing experience.
Serious answer: Companies that pull this crap deserve to starve and die.
How would you interview software engineering candidates?
The main problem with interviews [imo] is it grabs stuff the interviewer is familiar with and throws it at the person in the more stressful position of being the interviewee. Its also [often] a very non-standard way of communicating. I don't know about you but all of my communication is via text or graphic, not whiteboard.
And, honestly, do you care more about the fact they can invert a binary tree from memory? Or that they can learn to in the next hour and implement it?
My guess is for 90% of jobs, its the second.
I usually ask questions until the person is out of his comfort zone, and if he is completely clueless on how to proceed at that point it's a red flag.
If you're not at Google's scale, a take-home test that involves them producing a working solution to a problem in their own time, that you then expect them to be able to discuss, reason about, justify, and so on is a much better tool; even at Google's scale, where you might be worried people who copy answers off the net, have them bring the laptop they used in, and then get them to expand upon their solution.
All that said, I'd be surprised if that was the only factor at play. The Twitter comment itself seems to indicate a - uh - cultural fit issue. Additionally, and with no actual knowledge of the insides of HomeBrew, that a piece of software is widely used isn't evidence that the author is an excellent (or even good) developer.
I've never attempted to do this before, and my compsci is real weak, to the point where I'm going to make an educated guess about what a binary tree even _IS_ so... Let's start with what is a binary tree? Presumably we have a tree where every node has zero or two child nodes. Clue is in the name. In fact, that's an implementation detail, so let's say we have a tree where every node has a place two hold a left and a right node, where either can be null. Guessing that in order to be useful, the nodes need to hold a value. We need to choose a language here, and I know a miniscule amount of Go, and surely the cool kids at Google like Go, so:
type Node struct {
Value int
Left *Node
Right *Node
}
n := Node{ Value: 50, Left: &Node{ Value: 30, Left: &Node{ Value: 10 } }}
n.Right = &Node{ Value: 80 }
Input Output
50 50
40 60 60 40
35 45 55 65 65 55 45 35
We can either iterate or recurse here, right, because it's a nested data structure, and I once read almost the first two chapters of SICP, and I'm pretty sure those are the options. So, let's have a quick think about the memory / performance implications of that. I cannot think of a way we can avoid visiting each node here; perhaps there is one, but cards on the table, I can't think of one. So let's assume we have a runtime cost that will grow linearly with the number of nodes. If that's the case, I'm pretty sure that's what the cool kids call O(n) - maybe that's O(1), but I'm sure Wikipedia can tell me.
So the mechanics of switching any given Node looks pretty easy:
nr := n.Left
n.Left = n.Right
n.Right = nr
func (n *Node) recursiveReverse() {
nr := n.Left
n.Left = n.Right
n.Right = nr
if n.Left != nil {
n.Left.recursiveReverse()
}
if n.Right != nil {
n.Right.recursiveReverse()
}
}
Let's take a second to think about the memory consumption of what we've got here. At the moment, we add a new ... thingy (are they called frames? I have no clue) to our callstack for each child, and our worst case scenario then is the maximum depth of the tree. No-one has told us if our tree balances, but I suspect that if it does, you could then describe the max-depth using a formula involving `log` and the total known nodes. Wikipedia would know.
What else could we do? You could keep an explicit stack of nodes you knew you still needed to be visited, and go depth first. My gut feeling there is that you could write this where you get a worst-case scenario of your stack getting as big as the deepest node, plus one or two. If you wanted a truly iterative approach, you would need to store a pointer to the parent in each Node; there's a memory (or storage) cost to that too, so how often do we actually need to reverse the binary tree? That's a question to think about.
Given all this, then, one comes down to: what's the relative memory pressure that each frame in the call stack exerts, as opposed to pushing a pointer on to the end of an array in Go? Oooh, Go arrays are immutable, so actually you need to think about slices, unless you already know the size of the tree, and can preallocate exactly as much memory as you need. I wonder if there are memory implications there? With my commercial hat on, again, how much does this actually need optimizing?
Anyway, those are the considerations that come to mind; it would be worth checking Wikipedia for a nice iterative approach here, or if there's a sensible known algorithm. I'm going to stick with the recursive version I had above as my whiteboard version.
Did I get the job?
type 'a tree =
| Node of 'a tree * 'a * 'a tree
| Leaf
let rec reverse = function
| Node (L, x, R) -> Node (reverse R, x, reverse L)
| Leaf -> Leaf
struct NormalNode {
int value;
struct NormalNode *left;
struct NormalNode *right;
};
struct ReversedNode {
int value;
struct ReversedNode *right;
struct ReversedNode *left;
};
struct ReversedNode *reverseTree(struct NormalNode *root) {
return (struct ReversedNode *)root;
}
A similar effect could be done in an OO language by having a base class NormalNode with two accessors, getLeft() and getRight() work normally and then a derived ReverseNode where getLeft() called super.getRight() and getRight() called super.getLeft(). With a proper compiler, the whole thing would simply be optimized away.
Repeating what I said earlier - You can't build a plane by opening a physics textbook. Starting with a LARGE pool of fundamental and higher order building blocks allows you to fit them together using whatever skill/talent you have at solving problems, in a MUCH shorter time than someone who doesn't.
Someone who constantly google's basic stuff is a net drag on a team's productivity. Rather than operating at their skill level, they're constantly being distracted by having to task-switch and waste hours researching stuff. The internet isn't going to magically know the exact problem you're working on and whether certain data structures or algorithms are appropriate for solving it, or if they need further tuning depending on what your priorities are, or what have you. Getting questions answered, and googling/learning about CS stuff is great on its own, and should be encouraged as much as possible, but it is not a substitute for actually retaining that knowledge.
Doing a blind and "surprise me with a clever (or traditional) algorithm" whiteboard interview is a teaching skill, not a coding competency. You have to illustrate, explain, make your self vulnerable while understanding the problem, and work in front of someone while being judged confrontationally. It's not sane. (and glob help you if you don't wrap your entire brain around the problem in the first 5 seconds, then you're clearly subpar and are worth less than dirt even with your 15 years of experience shipping products to millions of users.)
Google is happy to have 1,000 nerd projects and only one thing that makes money. As long as they continue to interview smart people on technicalities instead of well-roundness and ability to actually contribute or move the company forward, they have no need to fix their insultingly irrelevant interview process.
Well, if you wanted to pull that card, it would have been nice to mention the sorts of problems you've worked on in those 25 years.
>What I would do if I had to invert a binary tree: look it up.
Unless you're already good at algorithms, it would net you a mediocre solution.
For e.g. You would only know that there are multiple ways to implement an algorithm when you've actually done the work dozens of times and noticed that some implementations won't be appropriate for you needs. Like with many things, its about having experience, knowing whats a good fit, and knowing why something similar isn't a good fit.
Certainly - every single time you choose an algorithm, and then decide on a particular way to implement it - you could in theory, implement each algorithm in 10 different ways and then choose the best one after benchmarking. But that would be a huge drag on productivity. And if you had to choose 5 algorithms for 5 different tasks then it quickly becomes a quadratic complexity time sink. It would be far better to just know via experience. Its kind of like in chess - the better players just KNOW that certain paths lead to less favourable winning odds. Well, novices do take those paths and eventually find out the hard way !
Quite a range of stuff, unsurprisingly. I started on an HP3000 mainframe in 1976, if you want to go back to the very beginning, writing BASIC programs on a teletype or one of the two early CRTs, and storing my programs on paper tape.
Since then I've worked in DOS, 16-bit Windows, 32-bit Windows, 64-bit Windows, Ubuntu, iOS, and Android, using Pascal, 8086 assembler, C, C++, C#, javascript, Python, and Java. I've worked on applications in multimedia, telephony, banking, insurance, pharmaceuticals, cosmetics, health care, and I'm sure a few other things I've forgotten.
All of which will mean jack squat if, tomorrow morning, the most important thing I have to do is invert a binary tree. But I'm fairly certain I would be able to figure out what I needed to do, and I am fairly certain I could manage to implement it well. It's what we're supposed to be able to do, and if you think that having studied up on it so that you could pass a Google interview means that, a few years down the road when you actually need it you'll just whip it off the top of your head, then I think life may hold some surprises for you.
>and if you think that having studied up on it so that you could pass a Google interview means that, a few years down the road when you actually need it you'll just whip it off the top of your head, then I think life may hold some surprises for you.
That is a misrepresentation of what I said. I said that _RETAINING_ basic and higher order CS fundamental knowledge is much more useful, in a way that simply looking up an algorithm on wikipedia would not be.
Sorry it was not my intention to misrepresent you. I was assuming that we all agree that simply looking something up without having any fundamental basis for understanding would not be of much use, and that we further assume the person doing the searching is in need of a refresh, and not basic education in the craft. In that context it is the idea that you might be called on to go up to a whiteboard and trot out something you haven't done in three years, and then be judged competent or not based on how successful the trotting is, that gets people worked up.
He also would've nailed it had he been given 10 minutes to do some research to refresh.
Silicon valley (and the numerous companies that mock the interview style) are testing for the wrong thing when they hire, then complaining about not being able to find good engineers.
Anecdotally, at a previous company, we tried an "open book" (i.e. you can use the web) interview policy for a few interviews. It was a train wreck.
Begs the question, doesn't it?
>> Binary trees are MOST ASSUREDLY on that list
Sure they are... and my intro to CS textbook was ~500 pages. I'll bet I could find something to trip up just about anybody, especially when coupled with the pressure of doing it in front of the class ^H^H^H people that have $$$$/power over you.
Google's insistence on this preparation thing is doubly weird to me when I consider that literally nobody "prepares" for work every day in this manner; knowing what a developer can do after studying his brains out regarding things that will be ejected from the brain again immediately after the interview doesn't really send a meaningful signal.
I prefer not to ask code questions at all, though.
If all someone needs is help remembering the name of a specific function, then I'd rather provide them with that if I remember it or just spot them it as something trivially Googleable. I wouldn't hold that against someone unless it seems like they're completely unfamiliar with the language they've chosen to use. There's no need to give an otherwise good candidate the chance to self-destruct over trivial details like that by obsessively Googling everything to make sure it's exactly correct when that's not what I care about, and in so doing they aren't paying as much attention to overall algorithmic details, which I do care about. If they need more help than trivial Googling, then they can't solve the problem anyway, and me watching them thresh around on the Internet isn't helpful to anyone.
A good analogy is laptops in classrooms. A lot of students will attempt to use laptops to take notes, even though they inevitably serve as a distraction, and study after study has confirmed that students actually learn better when they take handwritten notes. Somewhat paradoxically, studies of classes where professors have banned laptops and forced handwritten notes show higher student happiness; they're aware on some level that they're more engaged with the class and are learning better, and appreciate that the choice to distract themselves is removed, because if the choice were possible, they might not be able to exist. Well, providing someone access to Google in a coding interview is very similar to letting students use laptops in classes, in a lot of ways.
So I gave him one of the spare logic boards (open design anyway). And wrapped my hand around his. And squeezed. And asked him, if it doesn't exist, why is it making you bleed?
He watched impotently as the people who had been invited for the presentation played with my robots and ignored his as two of his guys tried to get it to work.
I finished the two projects I was doing with Google and did not call them again.
(Before you downvote: Yes, there is some video, and I consider the small amount of pain I inflicted that day a kindness compared to the much greater amount of pain that an engineer is in danger of enduring if he says things like "This thing that is in front of me, it does not exist", especially if he works with big machines).
I was doing something to remind an engineer that the physical world is of paramount importance.
A bit of bleeding and embarassment in 2010 as a reminder that things that are there, are indeed there, is better than being pulped by heavy machinery that "shouldn't be there" but is anyway in 2018.
i.e. "You want to buy us because of this, and a decent part of this is on GitHub, running every day, but you'd rather not look at -that- code but -this- arbitrary homework assignment".
From his perspective, I bet he's frustrated for the reasons he point out. If he never talks about it out loud, how would any one know? Keeping people silent about this issue is in the company's interest because they'll not have to really change, and they'll hire someone eventually.
I mean, I know I sometimes choke on problems I'm trying to solve in the privacy, quiet, and comfort of my own home on projects I really care about.
I don't know what went on in the interview but given what he said, I think I would be upset, too.
Too hard to say, tbh.
Edit: typo
I think everyone knows that this is not a problem exclusive to Google.
On the other hand, if it was a complex and reasoned decision that he's summarizing badly, then yeah, good call on their part.
Context is king here, and we have little.
If you were a corporation trying to keep PR positive, would you hire someone that has shown themselves as an outspoken public mouthpiece and tip-toe around them to keep them happy, or just avoid the problem?
And I'm pretty sure this is EXACTLY the type of guy you want to hire, I would strongly suspect writing mild mannered complaints about not getting hired on twitter has very little correlation with job success.
Interview with Lazlo Bock on Google's hiring practices:
http://youarenotsosmart.com/2015/06/08/yanss-051-how-google-...
Some of the claims Lazlo makes:
As large organizations grow, their workforce trends towards mediocrity. Google * takes special care to counter-act this effect. * researches their hiring/interviewing practices just as much as their machine learning. * publishes their methodologies: https://www.workrules.net/
The algorithm in question is discussed in Coding Interviews by Harry He. http://www.apress.com/9781430247616
I feel the original tweet conveyed a bad attitude, was emotional, reactionary, and ultimately a bad career move on the part of OP.
In my younger days I suspect I would have done something similar. I'd like to think I would see the experience as a learning opportunity and be able to react with humility and maturity, but who knows? Hopefully I can think of OP and not tap the tweet button.
Really? Pretty much everyone recognizes that google style interviews weed out perfectly good people. What Max went through is just an example of one such obvious case. It's very much a case of the google hiring algo failing. Lot's of people would have no doubt that Max can cut it when it comes to iOS dev.
That's all it is. Now, if you are going to read "tweet conveyed a bad attitude, was emotional, reactionary," into a perfectly human tweet, holy crap you are judgmental.
>> ultimately a bad career move on the part of OP.
Not at all. I've done a lot of interviews and basically none of them required us trawling twitter. I think it would have to be something pretty heinous for me to not hire someone based on their social media crap. Definitely not something as mundane as this. This sort of hilarious cowardice about expressing feelings just makes me angry. At what point do we stop acting like these trivial, humanizing glimpses into a person are somethign that is a bad career move?
If this is how success and failure are determined at google, it's no surprise how many of their products that people actually use come from acquisitions.
I interviewed once for Google (at their request) and failed. For some reason they interviewed me for a networking position instead of a code one, so questions about TCP internals were not really my forte. I was just launching my second start up at the time and would have declined the job had I gotten it. I admit it does sting a bit to be declined - not just for Google, but for any position - even those you would decline yourself.
Just quibbling: I don't think anywhere near 90% of engineers use homebrew? Google development is done on Linux, and Homebrew is a Mac thing AFAIK. I have never used Homebrew.
Android development can be done on Macs but I doubt they use Homebrew. Certainly not for anything important.
Its just good ol' hyperbole man. No small children will die because its not exactly right, it just helps make a point.
Now, I have no idea what the code quality of Homebrew is, but just because he built something popular doesn't mean he should get a green light in every company. If Google is looking specifically for top-notch software engineers, they are probably filtering them very well with their practices.
Maybe they are only good on paper at that moment and don't have something like "Homebrew" on their Github, their knowledge is sufficient to perform work at Google. So why pick someone who has fame to his name, probably wants to get paid accordingly and thinks he is a hotshot because of his Twitter and Github follower count over someone who proved himself in an interview?
The first is not necessarily better than the latter.
One problem is that only experienced developers can do these kind of interviews, because you need wide experience, be able to talk about various technical topics and tell whether the other person is telling you stories from their own experience or some quickly learned facts.
It's funny, but the best experience I had interviewing at Google and Amazon was talking with the managers.
That's usually what they say, but I've found that if you don't solve the puzzles, you don't get hired ever.
That is the killer right there. The 23 year old recruiter responds to your rejection with: "The interviewer thinks you'll be better if you get another 6 months experience. Keep practicing! We'll loop back around to you next year!"
Thanks, I've been doing this for 15 years. I'm sure another 6 months is all I need to meet your delusional standards of ineffable ability.
What does bother me is that other companies, who are not even in the same league as Google, start to copy their hiring process.
I remember interviewing with a digital ad agency a few years back and I swear, these guys thought they were Google. The number of academic trivia questions that came up, it was ridiculous.
In a way, I think Google has done a lot of harm to the industry in general by making others believe that everyone should have a hiring process like Google's.
On the exit meeting it was relayed to me that they were having trouble finding people because no one could pass their tests and I was beside myself because I couldn't understand how they would expect someone with that technical ability to want to bang out Wordpress sites all day while there are 100s of people who would love to do that job and be very successful without ever even knowing what basic recursion let along the stuff they had in their test. Bizarre.
interview was brutal - lots of whiteboarding very artificial problems totally unrelated to the business and i just couldn't get excited about it and didn't end up getting an offer
hiring is tough, these things happen
This is like knowing math and/or stats and applying it to a word problem.
I mean imagine the amount of extra work someone has to put in to start interviewing. Take a break from your current work, Prepare your CS basics again, Prepare from interview question dumps online, read up / analyze everything the new company is doing and form reasonable opinions, practice white board coding / coding without an IDE, allocate time for any homework projects given, Psych yourself up if you are introverted etc. The alternative is just to stay in the current role and hope stuff gets better. 90% of the folks I know choose the alternative over the dehumanizing process of interviews. So many folks I know are good engineers get chewed up in interviews (both in my current company and elsewhere) because the process is pretty cooked. We are trying to see how this can be improved, but yeah - I just keep going back to my pet theory :)
I do agree with one of the commenters here though: At one point your resume should speak for itself. These are the kind of problems I would like LinkedIn to be solving instead of finding ways to spam me with recruiter deluge.
I'm sure Google is still a great place to work, but its reminding me more and more of 1999 Microsoft. In fact the similarity is spooky.
Secondly, I don't think this is a hard interview question; it's certainly fair. Did you expect to be asked knowledge-based questions that Google knows you're already good at? Questions specifically geared towards you? Or questions where Google can watch you solve a problem and be comfortable with the fact that you are able to solve coding problems? Did you think Google would hire you to write Homebrew? Or solve problems on teams Google has?
I think this person is just being unreasonable.
It is just a figure he made up to make a point about how popular his software is. Using his software outside the context of our jobs is no means to justify a hire. He should go through the same interview process as 90% (much higher than that actually) of Googlers.
Edit: as pointed out by others, the hiring decision probably does not take a few hours, but under an hour. Still, the point is valid.
It's not because candidates aren't worth it, it's that if you can't come to consensus in that time period, you are unlikely to be able to :)
(I think what they do now makes sense if they want the top of their funnel to be comprised mostly of recent CS grads, and I think that industry people with really high profiles, or that internal people really want, get to skip a lot of this evaluation.)
I have a lot of respect for Google (how could you not). That doesn't mean they can't be a bit clownshoes with recruiting.
I would agree ... if they were actually having trouble hiring from industry.
IE despite everything everyone ever writes here, google simply isn't having trouble attracting enough of the right people that they want to hire (despite arguments to the contrary that they aren't hiring the right people or whatever).
So i can't see an argument that they do it wrong as long as it's actually working.
I expect, if it's truly broken, at some point they will, but
But yeah, I may not have been clear enough before: I do believe that what Google is doing is working for Google.
About the worst thing you can say for Google's recruiting as a business process is that it's somewhat inhumane, and pollutes the industry with faulty evidence (there appears to be a small cottage industry of consulting "porting" Google's recruiting process to other companies). But I mean, look at the rest of Startuplandia and Google's sins pale in comparison.
It seems like this tests a) how much you want to work at Google and b) how good you are at memorize things.
Homebrew is at least one of the best package managers for Mac. I would be very surprised if it was not at least near the 90% mark...
I mean - what's the point of spending 3-4 years in an Academic environment that proceeds to then test and grade students on exactly how good they are, at the time - then only to perform the whole process over again some number of years down the road, with fuzzier results?
Seems dumb to me.
I've worked with people who could likely do very well on algorithmic tasks - (of which most software projects require precisely zero) - but actually deliver something of use... not so much.
I get the puzzlers or whatever for a phone screens (quickly weed out people that are obviously unqualified), or if you're hiring someone junior who doesn't have work experience, but if you're at the point where you're bringing someone in you probably think they're minimally qualified, so it should really be about determining if goals are aligned IMO.
I hate whiteboard programming questions and I don't give them when I interview someone - I give them a laptop with 10 different languages on it, and some data to munge -- I think it's a pretty decent thing for both parties.
From my own experience - people that do well in such interviews are good generalists. On their own they will start discussing performance improvements and ways to parallelize the solution, it's a pleasure to have such an interview.
It's about enjoying problem solving and willing to keep your brain fit. It has nothing to do with memorizing solutions to some existing set of problems.
"We don't look kindly on trademark genericization."??
GEORGE: You know what I do at the Yankees, when one of these old guys is breathing down my neck?
ELAINE: What?
GEORGE: You schedule a late meeting.
The people vilifying Max or saying "duh you didn't get hired, Google requires awesome people" seem to have a totally warped sense of exactly what Google engineers do day to day.
Blows my mind that there are so many people defending this (well-known and pretty much taken as a trade-off) lapse in the Google hiring algo and instead making it seem like Max's fault.
Maybe we are jealous. We wish we had the brains of the people getting into Google. I can say personally that I envy these people.
The fact that they don't mind being treated as numbers maybe says something about these people too. They are cold. Their ego must be pretty big too if they make it into Google.
Someone should do a study...hehe.
Alumni's response that it is exceptionally tough. He shared an anecdote about one of his best hires.
The alumni wanted to test the interviewee's knowledge in different areas. He asked a question on diodes - it might not have been diodes, but for the sake of the story let's stick to diodes. The interviewee replied: "Hold on, I am not one to know about it."
I have not worked at Google and I do not think I'd pass its interview process. It is unlikely that I be diligent enough to make Homebrew. Nevertheless, I am inclined to the idea, that being knowledgeable in all tested areas would not reveal the personal fit necessary to make a great team.
The recruitment process (at least for experienced engineers) should be little more then "can I work with this person". The 6 month probationary period that follows the hiring process should be used for "can this person do the job well". But that's just my experience, and it seems to have worked well.
Regarding the same academic questions everybody gets asked in every development interview, I feel Einstein said it best with "[I do not] carry such information in my mind since it is readily available in books. ...The value of a college education is not the learning of many facts but the training of the mind to think."
If they try to invent something new or in different market they might need different type of people but as of now ads business is cash cow and they would be crazy to try change it.
I think the stated philosophy of interviews like this one is that a false positive is worse than a false negative. Every single one of the responses to that tweet either misses that point or sounds like little more than defensiveness in the wake of a bruised ego.
You might disagree with that interviewing strategy, but you're not addressing it directly.
I don't doubt that the engineers who manage to jump through all those hoops are sensational. Personally in the end it just dawned on me that I didn't want to work at Google that badly.
The whole Twitter exchange is a pitiful sour grapes circle jerk, and I'm surprised that it's provoked such a massive response.
I agree with your overall point (why not both?) but how does the interviewer know they can do both if they are only tested on their ability to solve problems on a whiteboard? I don't have a solution (that scales) to this problem but I tend to agree the types of whiteboard problems typically seen in interviews are a bad way to identify good developers.
First a little deduction. Let's try to be explicit about the theory behind this technique.
It's safe to assume that the job will NOT consist mainly of cranking out binary tree inversions on whiteboards while being watched over. So obviously we're hoping to make a correlation with something else. Assuming the candidate was not tipped off and learned this particular puzzle, perhaps we are correlating to an ability to rapidly create novel solutions of long-solved algorithms without reference tools.
But is that what the new hire will be doing? Probably not.
We could continue down this path, identifying ever more removed correlations until we get to something that the job actually demands. This probably involves solving hard problems like naming things. [1] But by now our theory stands on pretty thin ice indeed.
In any case, all of this deduction is theory making. It's not knowledge until we attempt to falsify [2] it via induction. The human mind constantly induces hoping to verify our deductions. We reason, observe, conclude and repeat. We're good enough at it to survive, but that's about it. Lucky for us, science came along. Today's technical hiring is at best alchemy.
A interesting company called TripleByte [3] is trying to apply induction (first for YC companies). They specially shun on white board coding and puzzle solving tests in general. I will be interested to see how they fare and whether their learnings are adopted more broadly.
[1] http://martinfowler.com/bliki/TwoHardThings.html
Without discussing too many details, I believe the issue with Google's recruiting process is it was designed when the company was smaller and it follows the philosophy that anyone that goes through the hiring process should be ready to be thrown into any of the many Google projects and be able to function immediately. That's not strictly true anymore.
You have some divisions that are extremely hardcore or require very good knowledge of a particular field (think Google Cloud Platform vs. Android Kernel vs. Chrome vs. Search, all completely disparate projects), but there's also work for people that don't need to hold a PhD from MIT (think front-end development.)
On the other hand bad engineer will stay at the company, lower standards, damage morale and set bad precedent to other engineers.
Being engineer myself, I feel much more motivated working in an environment where you can just assume, even before meeting, that the other person is intelligent and motivated. You trust hiring process to filter everybody else so you don't have to subconsciously distrust every person you meet.
This comes at the cost of situations like that.
If you're spending 45 minutes writing 5 lines of code, it is not definitive, but certainly a red flag.
If you have a definition, please elucidate.
Being a front-end guy, I proactively request for hiring manager or a senior front-end engineer from the team.
Would love to hear people's thoughts and feedback!
i also had some wierd binary tree questions when i interviewed at yahoo, was a total turn off as i was interviewing for web/front end related job. which probably would not have used any of this type of logic, ever.
Not to mention for a qualified professional candidate, it feels an awful lot like you're asking them to work for free.
> The code is reviewed by the interviewer > but probably never run.
This is what surprised me during an interview with Facebook. I tend to write a little code, run it to test it out, write a little more, test again, and so forth, you know, rapid iteration or whatever the fancy industry term is these days. The interviewer gave me these silly little shell scripting problems (silly in that they ought to have been easy and clearly had nothing to do with real life work) but instructed me _not_ to run the code. How the heck am I supposed to know if I've solved the problem? How will I know if I even have the right approach? I don't consider myself an expert software engineer and so probably not a fit for the position, but that style of interview definitely hamstrung me.
Another reason not to have candidates run the code is that they tend to get really hung up with debugging trivial errors. I've conducted a fair number of interviews either way, and the white board interviews were usually more pleasant experiences for all parties than the interviews where the interviewee was expected to execute the correct solution in front of me. The latter almost requires giving them some kind of web or library API access, which then just makes the distractions worse. I don't want someone worrying about what the exact name of a sort function is; that's not what I'm trying to evaluate in an interview.
http://www.developingandstuff.com/2015/05/why-i-dont-do-codi...
[1]: https://twitter.com/markmcerqueira/status/608914346706657280
Would they manage to pass the process?
who does dynamic programming on a white board during a work day?
it's like you are recruiting an army, but testing people's gymnastic skills. they should test street fight skills.
Jobs come and go, great work is always great work, but friends are what I remember most.
At a certain point, you learn that there are other jobs out there, and maybe the one with the biggest name isn't the best one. I certainly wouldn't be the least bit offended. Not when the market is flush with high-paying-jobs-a-plenty, particularly for someone with his background.
Walking through the cafeteria made me feel like I was back in CMU CS again, in a bad way.
Really smart people who are really insecure about their intellect?
Sometimes it's the reverse - the exhaustion of needing to always be "on" intellectually, because someone will point out the smallest inconsistency or fuzziness (again, in non-technical contexts where it doesn't matter)[2].
Sometimes it's because I'm super bored by conversations that mostly consist of geek-culture or technical references, where I get the reference but don't laugh because it's not that clever or not that funny, and people assume I don't get it (maybe that one's on me, though).
I've never been to a Google campus or event so I don't know what it's like there - I'm just describing my own experience at a place like CMU CS.
[1] An absurd example was the guy who claimed the song "Right Here Waiting" was by Bryan Adams. I told him it was Richard Marx, but he insisted that it was Bryan Adams. I didn't care so I just let it go. Later that day he emailed me a link to a mislabeled Youtube video as "proof" - when really it was just an ego salve for him to prove to me that he was justified in claiming that it was Bryan Adams. I offer this anecdote because it illustrates a common personality flaw of smart people (especially ones who spent their formative years as the smartest person they knew)
[2] Think comic book guy's xylophone comment from The Simpsons
That is hilarious and quite appropriate in this context :)
Condescension. Condescension everywhere.
It is people who are at the top of their game that are the nicest. My old CEO, who is head of a company everyone here probably knows about, was such a chill and friendly person.
In my nearly 3 years here, that is my experience as well. I've also spoken to many engineers who have left who lamented the quality of their fellow engineering talent at their supposedly 'hot' startup, compared to their former team at Google. Or having to deal with way more unprofessional (or crazy) management, prima-donna teammates, etc.
The process, on the other hand. Ugh. I have zero respect for Google as a company after that.
It starts with the standard phone screen/day-long onsite/hazing ritual. Then come phone calls with teams, and teams saying "yeah, we want you", and me saying "sure, sounds good". The recruiter basically said "time for the higher-ups to rubber stamp this, and here's the $$ to expect". Someone up the chain said "well, we're not so sure, let's haul him back here for another round of onsite interviews". Which I did, it went through the same process, and the response was "well, maybe not you for this role, but lets set you up with more teams".
All of this finally goes through, and I get an offer. Then there's a negotiation that goes something like:
Me: I'd like 4 weeks to think about it while a couple of other applications come back (keeping in mind they've dragged this on about 2 months longer than it needed to be).
Google: You get 2.
Me: I'm at a conference week #2, but I'll do what i can to get a decision to you Friday.
<Fast forward to monday of week #2>
Google: Have you made up your mind yet?
Me: I'd like to look at my options, and I'll get back to you COB Friday
Google: It's really important that we hear beginning of day Friday
<Fast forward to wednesday>
Google: Have you made up your mind yet?
Me: No. One of the options I was looking at is now off the table.
Google: So WTF are you waiting for.
Me: The other options
<Fast forward to Thursday>
<Phone rings while I'm at the conference>
Me: You can't pay me enough to deal with this.
Maybe no individual involved is a prima-donna, but the ego showed by the company as a whole through the recruitment process is stunning. It felt like I was dealing with the star quarterback who never considered that when they asked someone on a date, they might get turned down.
Google: "Just checking in to see if you have an update for me? Can we set a time to speak on Friday?"
Me: "I decided to pass on [OTHER OPPORTUNITY]. I let my manager know about the offer last Thursday. I am out at a conference this week, and we're going to discuss options on Friday.
Does end-of-the-day Friday work for you? 5? 6?"
Google: "I need to speak to you in the morning on Friday.
if you are passing up [OTHER OPPORTUNITY, misspelled] why are we waiting till Friday. Can we talk now?"
It just felt... off, though. Like, it would have felt normal if it was 7 years ago, and I was straight out of school, and as a process to separate amongst the folks in a given CS class.
As someone who has worked as an engineer, though? It was disconcerting. Writing 20 lines of code is very different than writing 20,000, and the process that:
(1) selects for the best writers of 20 lines of code, while thoroughly ignoring the skills required to write 20,000.
(2) is exhausting. Like, none of the questions were "difficult", per se, but it was a draining process. I know the grades I got probably got worse as the day went on, and it wasn't because the questions got harder. It was just a matter of "are we done here yet"?
Anyway, I guess it was also surprising, b/c Google is relatively known for having "the best" hiring process, and it just felt flawed, arbitrary, hostile, and not quite designed right. Basically, it felt more like a hazing ritual that existed because "we went through it" than anything that pertained to building a good team.
One reason you might have gotten this impression is that Google picks interviewers who have a lot of overlap in expertise with your resume so they can ask questions tailored to you. For example, if you have a PhD in computer science, many of your interviewers will, too.
I'm a Googler, and I don't think Google is a monoculture. Okay, maybe everyone does dress the same. But I've had teammates from wildly different cultural backgrounds—including Irish, Romanian, German, Hungarian, Israeli, Iranian, Pakistani, and Indian, and Chinese. Even among teammates from the US, it's not all the same schools. I went to the University of Iowa, which probably isn't one you had in mind. I've had teammates without a college degree (and I don't just mean interns).
I do think the interview process sucks. IMHO, it's the most bureaucratic part of Google. Sorry for your bad experience...
If you wanted to work for an organization where everyone likes to show off their skills to one another in the interviews, you'd have gotten the job, and you'd be one of them.
The best interviews I find are like a first day of work (but unpaid). Your experience and skills are established by resume and portfolio. The interview shows whether you can work with the team and wrap your head around the org's problems. If you're having to show off- well, that's how your work will be, too.
Google's interviews convinced me, years ago, that there's no way I'd ever want to work there. And that feeling hasn't changed one bit. Much like FB, it's an org whose coding needs are really pretty trivial and the real work was done and finished a long time ago (but there's plenty of need for debugging, egos especially). If you want to work there and surf the gravy train, cool for you.
I was with you till riiiiiight there. Google isn't just a search company anymore they're working on lots of very interesting non-trivial things all around the company.
Right. And I really doubt that people like Andrew Ng or Geoff Hinton or Sebastian Thrun were asked to invert binary trees....
[0]: http://en.wikipedia.org/wiki/Command_key#Origin_of_the_symbo...
Petty, but fuck em. They don't want him, they shouldn't get the fruit of his work.
So, I've interviewed with Google twice. Once was 3 years ago, the other was like 2 weeks ago. They contacted me. The way I see Google's interviews is like referee in Football(soccer or "Futbol"). Sure, you need a certain amount of skill to play in the World Cup but you winning or losing can, and often enough does, come down to a controversial referee call. You end up losing out to a team that did a handball - https://www.youtube.com/watch?v=-ccNkksrfls , but that's just how it is. What makes me like watching soccer so much is the same thing that excites me about Google's interview process. Yes, there is heartbreak and anger. Just like World cup fans get angry when their team loses because they were denied a point for an off-sides call even when the player was nowhere near off-sides.
--- Is Google's interview process fair? Nope.
--- Would I subject myself to their futbol-referee style of judging candidates again? You bet.
--- Do I think they should make their process more fair? Nope, let the drama and _justified_ rant posts continue. Just like I want the unfairness in futbol to stay as is. I was one of the people against putting the microchip inside the ball to know for sure if it crosses the goal line. I want the drama of a ref having to call it and sometimes getting it wrong.
I know people, especially on HN, love reliable & repeatable. I do too, except when it comes to dealing with humans.
But how about Google didn't give them a job because this is how they handle failure, embarrass an entire company on Twitter to punish them.