Hacker News Reader

Comment by Tarrosion | Hacker News Reader

I would consider it a strong negative signal if a company used this kind of question for anything more than a very quick initial screen - a sort of "data science FizzBuzz." Firstly, most of these questions are things that a reasonably smart person with the right technical background could just look up on Wikipedia or in a textbook, etc. - why base your interview on whether the candidate has bothered to memorize a bunch of easily at hand knowledge? More importantly, none of these questions focus on the parts of a data science job that--for almost all data scientists--are actually the most difficult and subtle: understanding your data and where it came from, disentangling confounds, communicating results, understanding the right question to answer, etc.

View on HN

For example,

> What kind of CNN architectures for classification do you know?

Is rated as an expert level question, but reads more like "what are some of your favorite pokemon?".

I can almost guarantee you that this is because "deep learning" is considered "advanced DS" in most corporate settings. DS has gotten so broad that, if you try to fit everything into a single category (rather than breaking it out into sub-areas like ML Engineer, Data Engineer, Research Scientistic, etc.), this is what you end up with. What "advanced" here really means is "relatively new field."

You can't ask someone just out of a stats program about CNNs because they probably have never touched them, even though for someone doing a lot of DL work in CS may consider this trivial, so you mark it as intermediate. And you can't ask someone with 10 years of pre-DL experience in DS about CNNs because they've been doing forests and regression and clustering and such, so you move it from intermediate to advanced.

In other words, it's "advanced" because HR wants to make sure it's "optional".

I like your suggestion about understanding data and where it came from. You are right that there are more interesting and complex questions than knowledge base questions, and I have no doubt that many people here agree with you. Knowledge questions in interviews seem to be a popular punching bag these days.

Let me offer a few thoughts to maybe soften the extreme point of view that knowledge questions are any kind of negative signal.

First, please don’t assume that the existence of knowledge questions means that the interview process of the author is based on them or primarily about them. Most, if not all, companies have multiple interview modalities, and basic knowledge questions is just one of them.

Given that there are multiple phases for an interview, the most important thing a candidate should do is avoid being rejected for something simple. It’s a gauntlet, and if you only think about the boss stage and forget to do really basic things like spell check your resume or choke on how to summarize what gradient descent is, you might not even make it to the hard interview questions.

From experience, it certainly is tempting to think that because you can look up knowlegde questions that people will just game it and ace them, but that’s not what happens in practice at all. In reality, people who don’t have a background in the subject might be able to recite something they’re heard or read, but have a very hard time answering any followup questions, dealing with terminology they haven’t heard, or connecting topics together. It’s very easy as an interviewer to detect BS by asking very straightforward followup questions, by having a conversation. Plus there’s a much bigger risk in trying to memorize answers to things you don’t know; when it’s quickly discovered you were trying to fake it, that is grounds for being quickly rejected.

Knowledge is part of the background you need to get to the interview, and while it might seem like you don’t use it directly on the job, it’s part of a foundation of exposure to the topics that connect the pieces together. There’s nothing wrong with exploring bounds of a candidate’s knowledge to get a sense of how much they covered in school, how much they exercised during internships or other jobs, how much they’ve thought about it on their own.

It’s perfectly okay for there to be easy interview questions (you’d be surprised how many get answered incorrectly!), and for there to be questions that are broader in scope than things you’ll do day-to-day. Keep in mind, it’s a conversation where you’re trying to convince the employer that you’re already trained in the subject, and also well-rounded and have the potential to grow.

All very fair points. Thanks.

I have never seen confound used as a noun before (I am not a data scientist). Is it the same concept as a confounding variable?

That's the sense I meant it, but I think it's an abuse of terminology. i.e. confound is not obviously a noun :)

I like to question about the models the candidate has used. I'm fine if someone has never used a CNN or another method. For me the signal is whether or not they care to know about the methods they use. If not it usually means that they are happy pushing data in and getting some numbers out.

Yes, of course - we should never ask questions where the answers can be found on wikipedia

Why does someone always say this? What are you supposed to ask these people, "hey, are you good at communicating?" "is u good with data?"? You need technical questions somewhere, and if you ask someone specific technical questions then you're being a pedantic dick, but if you broaden it (like these questions), you're only asking easily looked-up knowledge. I mean, come on.

Many of these questions are foundational or semi-foundational questions, and if you're not asking those then how can you possibly tell if someone understands the basics? I don't care that someone can look up what an ROC curve is, I care that they can explain why it's used and the relationship between TP, FP, recall, precision, etc., and why these distinctions are important.

And yes, your last sentence is correct, but it's ONLY correct when the individual actually understands the foundational parts of ds/stats. If they don't, then it doesn't really matter what questions they ask or how good their communication is.

For some reason there is this widespread notion that in DS, because the data is so important, and so tied into the business logic/processes, that you don't need to know the math, and that's just not true. That's how you end up with "data scientists" working in excel and putting out meaningless reports that no one ever uses because they can never get to any real, actionable results because anything more than linear regression is an impossible task.

I have to agree with you there. It's not nice to have to ask such questions, it feels like the equivalent of just hammering the candidate with questions about the time complexity of a bunch of random algorithms.

Coding or maths interview questions are quite nice because they can be treated as puzzles that the candidate can tackle in different ways, and the interviewer can guide them without giving too much away if they get too lost.

DS questions on the other hand come in two flavors: pop quiz type of questions like the ones in this post, or the "pretend you have data, how would you write a recommendation system" ones, which also suck.

I try to mitigate this by inserting questions when the candidate is talking about previous projects, so I can gauge if they have a deep enough understanding the stuff that they've used in the past, and also to give more context for the questions.

Reply on news.ycombinator.com