Google Hasn’t Changed Their Interview Questions
blog.geekli.st
blog.geekli.st
Deductive reasoning? Logical thought pattern? Creative thinking? Seems like there are a lot of better ways to find out if someone is capable of such things
"Correct. You're hired."
Alternatively, the bug could fly out if it's that kind of bug. Or it could knock the bottle over by rocking it.
Clearly I'm Google material because of my "bug in frictionless jar" degree.
However, brainteasers are indeed banned and they always have been. The problem is that what you and I call a brainteaser isn't what everyone else calls one. Thus, a brainteaser could still be asked, despite the ban.
A brainteaser is something like "A man pushed his car to a hotel and lost his fortune. What happened?" That's a brainteaser -- and nothing Google would ask.
So, do I get the job?
This boggles my mind. Where? For what purpose? Is the expectation that the candidate googles that question?
I would google lots of different things until I found some answers (car manufacturers sales reports from car magazines, news articles that may have the answer, literally for the phrase "number of cars sold in a year" etc, etc).
If you want me to talk through it, it's just mind numbing making asinine assumptions. Market sizing? Obvious assumptions? What?
I can't understand how anyone can say with a straight face that that question is straight forward and not ambiguous.
The ambiguity is what makes these google interview questions brain teasers, to me at least.
But that's probably the last item in a list of reasons why Google wouldn't hire me :)
The point is not to get the right answer. It's to deduce a reasonable answer that's in the right ballpark.
How did he do it?
Consider the Wolfram Alpha engieer who implemented the same feature.
How did she do it?
(It becomes an interesting question, eh?)
It's so ambiguous at the google search engine can't answer it.
Usually for this type of questions you don't really even need to say the numbers, just clearly outline your though process.
What does a user mean if they enter [how many cars sold in a year?]
The ambiguity shows that the specific answer is less interesting than showing your working; having the discussion about how to get an answer.
I wonder what internal Google has explored along the lines of prompting the user with clarifying questions. Like Google Suggest and Spell Check exhibit the level of intelligence needed I think.
Some coding questions as asked in interviews are actually riddles in practice because the interviewer is unskilled. Those 'coding questions' are worse than the obviously bad riddles because it is harder to tell that the feedback is slanted.
Took the interviewer a while but they finally figured it out.
So if 1000 cars are sold in a year, maybe he'll see plate #267 from Year 0, and plate #1796 from Year 2. 1800 - 300 / 2 = 750 cars per year, close enough.
Software Engineering interviews will focus on your standard coding, algorithm, and system design questions...
Why are algorithm questions still being asked in a high-pressure environment? Very few people actually work on algorithms once hired and never in my experience has it actually been a good indicator of actual development competency. As DHH states: http://37signals.com/svn/posts/3071-why-we-dont-hire-program..., unless you're hiring someone to code algorithms it's not useful.
As a Director at AppNexus I've done my best to reverse this trend by asking what I consider competency questions. Such as "On a scale of 1-10, 1 being novice, 10 being creator of said technology, how would you rate yourself?" Then based on this answer I'll ask a question at that level. I find that most people screened don't actually know the basic fundamentals of the technologies they list.
After you've gotten the basics down you can then get into system-design, or thinking questions. In the case code analysis is necessary I think it's much better to present a sub-optimal pre-written function and ask the candidate what the function does, and whether and how it can be improved. In this way I know whether they understand code, and whether they're competent enough to improve.
The faster we move away from these algorithmic questions the better.
[1]http://en.wikipedia.org/wiki/Dunning%E2%80%93Kruger_effect
See, and I hate questions like that.
If you're going to ask candidates to rate themselves in a technology, don't ask for a number. What I think a 7 is might be different from what you think it is. Just ask the candidate how comfortable they feel in a language. Words work better than numbers.
Also, you should know that many candidates are really scared by someone asking them to rate themselves on a scale from 1 to 10. You're kicking off an interview with a degree of intimidation that's probably unnecessary.
You're right though that many people don't know the fundamentals of the technologies they list. This doesn't mean that they're bad engineers; there's just a lot of confusion around at what point you should list a technology on your resume.
Remember that a lot of companies aren't particularly looking for competency in a particular language. Thus, drilling into the specifics of a technology doesn't assess much for them.
http://www.joelonsoftware.com/articles/HighNotes.html
I've found that in my work, a tricky algorithmic problem comes up on average about once a month. It's a tiny fraction of the total work I do - but if I couldn't solve algorithmic problems, then there are whole projects that I simply couldn't tackle. We'd either do something substandard for users, or we'd throw bodies at the problem (which still happens a lot, unfortunately), or they'd need to bring in someone else to do my job.
Moreover, I've found that there's a cultural shift that happens when enough people are familiar with algorithm design within a company. A lot of other places I've worked are basically "Make the user fill out a form, dump it in a database, format it for display." When a significant fraction of your employees are capable of hitting the high notes, this becomes "Make the user interact with your system in the manner they're most comfortable with, extract meaning out of their ordinary behavior, compute interesting results, and show it to them." The complexity moves from the user to the software, and as a result, users would rather use your software. There are whole subsystems within Google Search - synonyms, spelling, refinements, authorship, snippets, ranking, translate, voice search, etc. - that would never have happened if there weren't a critical mass of people willing to dive into difficult problems.
The latter makes it clear that this isn't some sort of brain-teaser, it's question about process. You could also phrase it as "what information would you need to estimate how many gas stations there are in Manhattan?" This in particular would be a good question for people who will doing a lot of work that involves some speculation, such as long-term product roadmaps, long-term expansion planning, looking at new markets, etc.
BTW, I've been to Manhattan and noticed that there are remarkably few gas stations there for a city of its size, so whatever estimate you come up with will probably be very, very wrong.
"how many gas stations are there in Manhattan" actually makes sense as a question for a venture capitalist firm employee expected to make investment decisions, as the leap from "statement" to "this is something I need to estimate" is absolutely key to the role, and hence something you'd want to test for. I'm sure others can think of much better examples of this kind of thing.
Rather than massive re-wording of the question, you can instead change the framing of the question. The most scary aspect of such questions is that they might come out of the blue, or immediately after a number of completely different questions. Mental inertia will then dictate you stall massively and find the question completely confusing.
Imagine instead that the interviewer stated "We are going to ask some questions that will test for reasoning and deductive skills that you will use on a regular basis in the role". This is similar to changing the wording, but more like real situations. You know you have a meeting with clients (so you're prepared), but they'll ask questions in a difficult and something obtuse manner.
Yes, and that's precisely the sort of thing you're supposed to take into account. Again, you're looking to get in the right ballpark. They're not looking for precision.
Asking "what steps would you take" wouldn't be quite the same question. They want you to actually take those steps to come up with a number.
Ultimately, I'm not necessarily defending estimation questions. I'm just saying that they are, in fact, asked because interviewers don't consider them to be brainteasers.
According to Laszlo Bock, senior vice president of people operations at Google, "We found that brainteasers are a complete waste of time." How did they determine that without asking candidates any brainteasers? Clearly, they did at some point--probably before the author's direct experience.
This article is really just a means to promote the author's books and prevent readers of the original article from presuming that that her books are obsolete now (they're not).
(2) I've confirmed with a bunch of people currently at Google that, indeed, nothing has changed. This study was done 5+ years ago.
(3) My books don't focus on brainteasers. Thus, if people believe these changes at Google are real, then this would actually make my books seem more relevant, not less.
(4) If brainteasers are banned, this doesn't mean that no one has even asked them. Some people break the rules (because they're unaware of them or because they don't feel that a particular question is a brainteaser). Thus, Google could, theoretically, study how effective brainteasers are even while they are banned.
(5) What Laszlo is saying is provably incorrect. He's saying estimation questions are brainteasers and that they no longer ask such questions. This is false. If he wants to define these questions as brainteasers, he's welcome to do that. However, he would then be wrong about Google continuing to ask brainteasers. By his definition, Google absolutely does ask these "brainteasers" frequently.
(6) What I really suspect is going on is that he misremembered the study (a reasonable thing to conclude, given #5). It was, after all, done 5+ years ago. I don't think the study ever actually looked at brainteasers. I read the results, and I don't remember anything about brainteasers. (They did look at interview scores and job review scores though.)
(7) Huh? It's "just" a means to promote my books? That's a huge leap. My books aren't even mentioned anywhere except for in my bio. You could argue that it indirectly promotes my books, but then basically everyone ever writing anything is promoting their stuff. And, even so, you couldn't say that it's just to promote their stuff. This article is a means to counter a lot of the myths around Google hiring practices. I don't like candidates walking into interviews misinformed.
They aren't about getting the right answer, but rather about seeing how one breaks down a seemingly difficult question into simpler pieces and determines what can be reasonably estimated. They really don't fall into the category of "brain teasers" which involve some sort of trick.
However, the statement that software developers are not asked these types of question is not universally true, nor should it be. In and SDE interview, I was asked an estimation question that was both interesting and relevant to the position (although I signed an NDA, so I'm not going to give specifics).
In my case, there were several pieces of required information that I realized I couldn't reasonably estimate on the spot, so I gave a description of how I could ballpark them from quick measurements.
How many plumbers in New York?
Or more practically that they would do a roll-out release (as they did) and increase capacity as they go. All questions for devops, the product manager or some type of an "Architect" working with analytics to estimate demand and volume.
To imply that a single engineer in a room with no information about Gmail's infrastructure is supposed to know that without (any information about: technologies, storage, indexing, user volume, mail volume, etc) seems absolutely fucking stupid. If they want to hear me think aloud about these things and know that I can be cognisant of them... well... there are better ways of doing that than asking me stupid questions that it's stupid for me to even try and answer.
As someone else said, simply changing the question from "How many gas stations are there in NY" to "How would you estimate the number of gas stations in NY" makes it entirely different, IMO.
What if, increasing capacity as they go, they discover that they're going to need 140 billion dollars worth of hard drives?
Before the rollout can happen, someone has to decide whether or not to greenlight the project. And that person needs to come up with an estimated cost. Before the rollout begins.
> To imply that a single engineer in a room with no information [etc]
Who implied that?
I'd prefer questions and problems and challenges that I'm likely to face and address as an engineer.
If Google hires engineers to answer "brain teasers" that either require a dozen exceptions/qualifications or some vague random nonsensense answer, then more power to them.
(But I doubt they do, they wouldn't be where they were if their hiring practices didn't work to some degree.)
And it was a reasonable question that may or may not have knocked me out of the job, as long as by reasonable you think it's "fair" to serve some people the bottom half of a cake and other people the top half of a cake.
But I of course don't know if that's what knocked me out.
The big problem with that question is "cake". Mappings change our entire perception of a scenario (The mass-grave-filling problem of Tetris being a classic example). Cakes have tops, bottoms, fillings, decoration, and so on. This makes a perfectly reasonable question into something much more nasty
This is the same issue I have with the egg drop problem. It can be logically deduced and so it's sort of a fair question for a software engineer. But, given the abundance of more relevant algorithm questions, there's just no reason to ask it.
Ultimately what it comes down to is this: brainteasers are banned and have always been (or at least for a very, very long time). However, since everyone defines brainteasers a little differently, you could still get a question that you feel is a brainteaser.
Or that there are other parts to a Google interview besides the teaser that can determine your employability at Google.
Sorry but this article is false.
In part, it's because it seems to have come out of consulting, where a common task seems to be to size a market for which there is no commonly available data. So, understanding how someone might go about doing that seems reasonable on its surface.
But in fact, I've met too many consultants who seem inclined to build "castle in the air", spinning market details about theoretical markets without properly defining either the product or service or the buyer of that product or service. And since they are for markets for which there is by definition no solid data (since that is why the consultant was asked to size it in the first place), their estimate is never validated (or if it is, it is so long after they have left that they never hear about.)
At the same time, no one ever seems to mind if the answer to such questions is off by a country mile when the actually size of the market is known, so long as the thought process was rigorous and shows the right kind of logical decomposition of the problem. But this justification for the value of the question also bothers me, as it seems to be testing more whether you can come up with convincing-sounding bullshit than whether you can correctly estimate a given value. This, again, may be an accurate measure for whether someone can become a consultant, but I never liked the notion that we should judge people on how well they can spin bullshit.
Finally, the question seems to also be gauging the interviewees willingness to "play along" with what an interviewer is asking. The questions are often on seeming random topics (piano tuners in New York or gas stations in Wisconsin), and are often not directly related to the domain. In the real real world, you would actually probably find a list of such things on the internet, or conduct some basic research that can be done to answer such a question. Or the correct response might be something like, "We can make a rough estimate, but without more solid data than a few random facts which we've rubbed together to come up with a market size, maybe we shouldn't be pursuing this market". In which case, the contrived example serves to allow the interviewee to demonstrate that they are the type of person who will enthusiastically pursue whatever random intellectual exercise they have been assigned by the interviewer, so long as there is a chance of getting the interviewers approval.
All of the things that the question tests for, then, seem to not be characteristics you would actually want in someone if they were to answer the question in the real world. In the real world, you might do a back of the envelope calculation, sure, but you would also do a lot of research on the internet, conduct a lot of interviews and surveys, and/or conduct evaluations of competitors to understand and size a market.
I agree though, that overconfidence in one's ballpark estimate is not to be desired. But, that's more useful information gained from the question, not less.