Hacker News Reader

Comment by bnoordhuis | Hacker News Reader

I interviewed with Google for a SRE position earlier this year and their screening process sure is tedious: Lots of grilling on obscure UNIX trivia and data structures/algorithms but nothing on the broader picture, like how to write good, maintainable software. I like nerding about graph traversal algos as much as the next guy, but not for 90 minutes on the phone after a long day of work.

I'm sure the screening process filters out duds but it's bound to generate a lot of false negatives as well. People that would otherwise be a perfect fit but who get fed up with the interminable interviewing or simply don't do well under that kind of stress (for me it was both).

View on HN

False positives are way worse for most companies than false negatives. Interviews processes are generally designed around this principle.

They've actually said as much. They understand that their process generates false negatives but they don't want any "crud."

Lots of grilling on obscure UNIX trivia and data structures/algorithms but nothing on the broader picture

They understand that their process generates false negatives but they don't want any "crud."

But quizzing on random trivia is irrelevant to the process of eliminating crud. It's quite likely that a cruddy performer will be able to rattle off all known sorting algorithms or all the option flags for GNU grep -- that has absolutely no bearing on true intelligence or any of the other qualities that Google professes to be interested in.

Having some insider's knowledge of Google: the process does not appear to be working. There is a lot of crud in there now, and the crud only seems intent on hiring more crud. Lots of power struggles and posturing; less and less innovative thinking.

Ah, but this is not your normal unix trivia, the questions I were asked were stuff like what is an inode, where are inodes stored, what is a scheduler, what is the difference between /dev/mem and /dev/kmem and other questions along those lines.

This is not just flags to give to a process to make it do something, this is system internals. When I mentioned that I knew FreeBSD fairly well I was asked questions specifically about UFS and the likes.

I never did get asked about flags to grep and other UNIX tools.

the questions I were asked were stuff like what is an inode, where are inodes stored, what is a scheduler, what is the difference between /dev/mem and /dev/kmem

Precisely what I mean by unix trivia. What's the difference between /dev/mem and /dev/kmem? I think it's memory vs kernel reserved memory, but in my 16 years as a unix admin I have never once needed to know that. If I did, I'd go look it up. That is why I consider those questions trivial: they're not fundamental, although you can make them sound like they must be.

Exactly.

They've also said that their high false negative rate is why they clear out their records every couple of years. If you got rejected a couple years ago you can apply again and they won't be looking at your previous attempt's performance.

This is the first time I've heard of this. I interviewed back in 2007 for SRE, and again in 2010 for SRE. I was told "we'll keep your resume in our database and let you know if anything comes up."

It kind of goes against Google's entire corporate culture to ever delete information like this. Sometimes I wondered if their interview process isn't designed more as a data mining process to find out what interesting things technology leaders are working on.

What Google should do if they want to hire really good candidates is to go with a contract to hire approach. Find good candidates, do one phone interview and a second in-person interview in the space of a few days (not 4 or 5, stretched out over months). If they pass your initial 2 interviews, hire them as a 6 month contractor.

See how much they accomplish in 6 months. If they are contributing and integrating effectively, hire them full time. If not, just wish them the best and don't renew their contract.

This is the way it works in most effective large companies. The cream that rises to the top gets offered a full time job within a 6 month to 2 year timeframe. My current job is like that. I've worked there for 4 years and started as a contractor. It became so expensive to pay my hourly rate that they were begging me to become full time after a while. I was doing a valuable job for them and was very effective at it, so I got the full time offer.

Not to mention that in a couple years you can become a much stronger hire anyway, especially early in your career going from 0 experience to 2 years. Experience = Time * Density [1]

[1] http://stevehanov.ca/blog/index.php?id=105

I see lots of assertions to this effect. Is it really true?

One point that no one seems to consider is that frustrating interview processes are a big contributor to smaller applicant pools; that is, a would-be good employee has applied a couple places, been filtered out by false negatives, and is thus disincentivized from even continuing to look for a better programming job.

Personally, I don't like my current job but am growing frustrated with job hunting. I'm very close to just looking for a bartending job to go along with the part-time valet work that I picked up between jobs, and then maybe doing hobby programming in my spare time for pleasure and profit. I'm serious here: ridiculous interview processes are making the quality of the programmer pool worse by nudging programmers out of it altogether.

Why is it so unreasonable to bring people in on an intern-like trial basis?

Maybe programmers nudged out of the process is a feature not a bug. People who self select out of a hard hiring process may meet not Google's criteria for certain attributes they want to see in their candidate.

Reminds me of property management: When screening possible tenants, merely saying that you will conduct background and credit checks will deter some people from even applying, most of them being people you wouldn't want to rent out to anyway.

> Why is it so unreasonable to bring people in on an intern-like trial basis?

We do that, in addition to an interview process biased towards false negatives. We still find it expensive to make hiring mistakes, enough that we are still circumspect about candidates.

Not only does it take a lot to build up someone new, it's emotionally costly to tell them at the end of the evaluation period that they didn't make the grade -- and still give them full support and a real chance the whole way, in case they can turn it around. It does happen; sometimes it just takes time for ambient culture values and priorities to sink in before people "get" what's important to succeed.

It's much harder to do this trial period with option to buy, than for interns who are going back to school, where the expectations and time frame are clear. It also seems that the most capable candidates aren't as interested in being hired on a trial basis, since they have other options willing to commit more fully to them.

These are a few reasons it doesn't work as well as you might hope.

(* note, not talking about google here)

It is really true. As you would know if you'd been on a team that needed to fire the wrong hire.

As for why not to try people before you hire them, a lot of experienced people already have jobs. They aren't going to want to accept a "maybe yes, maybe no" position without a really strong incentive.

That said, a number of companies do have policies of liking to hire contractors, then make some of them permanent. This kind of works. But the best contractors they'll find generally are contractors from choice, and so aren't likely to want to make the transition.

To me, this is one of the big unspoken about benefits of working in a startup. There is no room for slack and if someone isn't doing their job it's immediately apparent. Nothing kills my motivation more than working with someone who is technically incompetent and/or generally unpleasant to be around.

Are those interviews so frustrating that you are willing to take 2 low-paying jobs instead?

As a bartender and a valet, I've easily averaged $15-25 an hour. Right now I'm making $25 an hour when I'm billing, and once I get home I'm not in the mood to do much computer work. Additionally, you can find service jobs anywhere, and start up very quickly.

I'd rather have the flexibility to move anywhere, the stimulation of constantly meeting new people, and some leftover desire for personal projects when I get home than to have a programming job I don't like.

Fair enough. Note that I'm not looking down on your choice of jobs (recently I've been thinking it would be cool to get a part time job as bartender but it's not feasible now) but I suppose they inevitably come with some hassle and that will be worse over time than the one-time hassle of the interviews. Besides I assumed your programming job would pay much more (well, it should).

Want to work in Chicago?

Our interview process mostly consists of asking you questions you should know the answer to. If your answers make sense and you seem pleasant, welcome to the team. Oh, did I mention you'll probably make twice as much here than at Google?

Incidentally, more people want to work at Google than want to work at Bank of America, even though there is plenty of interesting code to write at the Bank, and plenty of boring paperwork to do at Google. It doesn't make sense to me.

I love Chicago... that sounds good to me.

Could you please give some examples of interesting code at Bank of America?

Fraud detection.

So, BoA wants to detect itself then?

What language(s), tech, framework is used for the software (web-apps? desktop? etc) at BoA?

There are something like 70,000 developers and 300,000 total employees, so it's hard to say anything other than "all of the above".

I mostly use Haskell and Perl. The rest of the department is mostly Perl. Sometimes I do C. Other groups are Python-heavy. There is a big Java and C# presence, and of course a lot of people that think C++ is the only real programming language.

There is some initiative to standardize on C++ and Python, but I doubt that is ever actually going to happen.

You forgot to mention that you would be working for a giant corporate behemoth that is evil to its customers.

How exactly are they evil? They borrow your money at a certain rate, and loan it to other people at a higher rate?

Without getting into the evil/not evil argument: This is not quite correct. A bank can lend more money than they have/borrow. The bank must only maintain a specific ratio of equity and loans. For more details see Basel I, Basel II and the upcoming Basel III, which (should) regulate the loans of banks.

I recently went through financial difficulties that involved problems paying 5 different credit cards. Bank of America was by far the most helpful company, actually recommending that I enter into debt management to allow me to pay off my cards at a markedly lower APR without taking a huge hit to my credit.

This is asserted as a truism, and perhaps in Google's case it is, but if the difficulty in the hiring process dissuades enough people who have other options from applying I can see it hurting the job pool.

If you were to get 100 applicants, and you were only interested in hiring the top 5% (5 people), and 3 of the good people decided to accept decent jobs they could get without giving blood samples and their first born, and 20 of the less capable people decided not to bother (Incompetents don't know they are in competent and have less choices), then now you have 2 good people out of 77.

Now what if one of those two is a false negative?

Google is attractive enough that good people go through all the trouble anyway, but is this always true? Since we are able to fire people easily in this country, is a bad hire really so ruinous?

I don't actually know the answer to the question, but I am skeptical of your statement. Maybe the assumptions in my numbers are wrong (they almost certainly are), but what are the real numbers?

Your argument holds weight in general but Google is a special case.

I'm sure they knowingly turn away bright & competent engineers simply due to the large size of the applicant pool.

Smaller companies would be wise to heed your advise though.

When I interviewed with them for the SRE position I was extremely nervous (I was right out of college, and Google had contacted me rather than me submitting my resume) and the guy on the other end of the phone for my first interview seemed uninterested, had a really bad quality phone (I could hardly hear his questions), and kept asking the same questions over and over in different terms, and I really couldn't figure out what he was looking for as the "right" answer.

The Unix trivia was not nearly as bad, that was easy as I had done a lot of reading on it, however some of the data structures I was asked about I had never learned in school and had just read about on Wikipedia so I didn't really understand them well enough.

The second interviewer was absolutely fantastic, and was genuinely interested in me as a person, had checked out my resume and looked at some of the previous projects I had done (Near Space balloon launches were discussed). I felt much more at ease, but I did seem to have somehow mixed many of the different programming languages I know (C++, C, Python, Java, F#, Scheme) thus my code was less than clean.

Ultimately after 5 different phone interviews I was unfortunately told that I was not a fit, that they would keep my resume on file and that I could try for another position when I had gained some more professional experience.

The Google interview did help me get through many other interviews I had the pleasure of going to. I was less likely to be caught off guard, especially since it was my first MAJOR interview with an extremely large company. I do wish that Google had not been my first interview as maybe then I would have gotten further into the hiring process and maybe just maybe I would have been working at Google.

That is an awful lot of maybes and it is not worth brooding on what could have been. I am extremely happy with my current job, I love what I do and the people I work with at a small new startup. I feel with a small company I can make more of a difference.

I interviewed two years ago and I think it was pretty reasonable: one 45min phone interview and two 3h on-site interviews. I interviewed for a stat position and the questions were mostly a "see how you think on your feet" problems, talking through how you would solve some problem. I thought it was quite a pleasant experience, the whiteboard-discussion style was quite similar to what we do in academic research.

Perhaps the position they were interviewing you for actually involved writing efficient software, and thus required thorough knowledge of the fundamentals of computer science (data structures and algorithms &c) and systems programming?

And certainly your day-to-day work would involve answering questions about those data structures and algorithms in real time, with your future career depending on it, without external references.

If you know something well, you can reason about it in real time.

Pressure is an artifact of the interview process that is controlled primarily by the attitudes of the interviewer and interviewee regardless the subject matter.

External references are useful for things you don't know cold. For something basic and fundamental to your job (eg. basic graph traversals, the suitability of a hash or a balanced BST to some lookup task, etc), you shouldn't need them; for something more arcane which is accidental rather than essential to your task, or something relatively advanced (eg. specifics of tries for incremental string operations), a competent interviewer will help you out and supply some information so that he can evaluate your ability to reason, design and code.

None of these things have any bearing on whether or not it makes sense for Google to focus on knowledge of the fundamentals of computer science and systems programming in their hiring interviews.

I think not. Your day to day work involves sitting in front of an editor of your choice and writing code that may or may not compile or interpret properly, may or may not contain bugs, and may or may not have the correct logic. Simply having a computer at your disposal with the Internet handy, a step debugger, or the ability to run a shell or interpreter in an interactive mode puts you light years ahead of someone struggling in front of a white board trying to remember the arguments and proper order for some obscure function they haven't used in months.

How many people actually solve problems in their day to day jobs without every looking anything up online? Or at least checking to make sure they are correct. Just having a syntax highlighting editor is light years ahead of a white board and dry erase marker.

edit: Sorry, my sarcasm detector doesn't work. I think we're agreeing.

> I think we're agreeing.

Yep. Sorry if my sarcasm broadcaster had a weak signal

SRE (Site Reliability Engineers) according to Tom Limoncelli (a Google's SRE) is a kind of sysadmin, I doubt that this would ever be the case.

Right. I interviewed for SRE at Google. It's basically a sysadmin job. You're expected to be proficient at scripting in some language (Python, Perl, bash, etc), but I would assume you're not expected to write code for most Google applications.

I would think the senior SRE engineers know enough about the code base to poke around and make suggestions ("found a bug in module.py at line XYZ that causes bad performance when thousands of users hit it"), but I got the impression that most of their top developers probably would be bored in SRE.

The thing that is ironic is that most of the people interviewing me were not in SRE, but were pure developers. How many companies do you know that let the developers interview sysadmins? I had guys asking me CompSci questions I hadn't heard since college. If the tables were turned, do you think I'd be asking a prospective developer a question about how to configure DRBD replication and heartbeat from one Linux server to another? Even though some devs might know these things from hobby-like experimentation in their free time, they are not going to be doing it on a day to day basis.

It just strikes me as a very broken interviewing system when you have developers interviewing sysadmins.

An important distinction that's missing in this discussion is that you can be either an SA-SRE or a SWE-SRE. You can figure out which is which from how the SRE job postings on google.com are phrased.

SWE-SREs have to pass SWE interviews. I work down the hall from Tim. Nobody keeps track of who came in through which side (SA or SWE), but some of the SREs I work with wrote large parts of the services they support.

> It just strikes me as a very broken interviewing system when you have developers interviewing sysadmins.

Sounds like your recruiter messed up.

I had the same issue.. was kinda like, wtf..?

And how many systems does a google sysadmin administer? :)

(Answer: a lot.)

Not entirely true. There are two sides to the SRE positions. The sysadmin side is indeed what you say they are, they work on the systems keep them running and do hardware related tasks as well as UNIX level tasks.

The software engineering SRE's work on products to make sure they are ready to scale or if something fails to fix the point of failure. They work alongside the product development teams to make sure the product is stable in production.

The two positions require two different skill-sets (I was interviewed for both sides of SRE). One requires very much computer science, data structures, programming languages, and all of those, the other sysadmin requires more knowledge about Linux, its internals, and the command line interface.

Maybe they didn't have people readily available to interview you on topics more relevant to your position.

Or maybe they care more about IQ and the sort of interview you got is better at estimating IQ.

Reply on news.ycombinator.com