Amazon data science interview questions
mldatageek.herokuapp.com
mldatageek.herokuapp.com
By a very large margin, for most of the time you are actively developing as a full time employee at a large tech company, you're working on integrating systems, either through a build language or extending your software to use an already existing API.
What are the most useful skills for these tasks? Being able to communicate effectively, feeling comfortable asking questions, feeling comfortable admitting when you don't understand something, and being kind and friendly with those you interact with.
Therefore, when interviewing new hires, I do not ask many technical questions beyond a general competency question.
I've never liked this idea or thought it was true. At the end of the day it doesn't matter how could of communicators people are if they don't have the skill or talent.
Give me a genius programmer on the spectrum over someone who smooth-talked their way through the interview but doesn't know shit any day.
I think either way, a completely non-technical "good communicator" or a "genius programmer" non-communicator make it difficult to get these tasks done.
Still, besides that issue - which all novice programmers have, but Data Scientists come in at a much more senior level - it was a great market advantage, one that I think is largely gone today, as companies understand the need for statisticians and economists who really and truly understand modeling and math.
There was a company that did this a while back, I don't even think they actually had a product, they just knew that one of the big four would pay millions to acquihire them.
Anecdotally, I've worked in places where this is a major source of contention - The Data Scientists treated as talented individuals (who produce broken solutions which work for cherry picked data sets), and anyone else is just a monkey who maintains and fixes the broken code.
This is very much the opposite of my experience.
The go-to graduate level machine learning text "The Elements of Statistical Learning" was written by 2 statistics professors (one, Prof. Hastie, a fellow South African! :)
Granted, neural networks are often taught as a bolt-on lecture in statistics course machine learning modules but topics like regularization and a rigorous study of overfitting were born in stats departments.
That being said, there are few of them and they congregate to the big tech and finance names. It can look like everyone or noone has a formal education, depending on where you personally work.
For example, going through all the issues with the question and implicit underlying assumptions, talking about what you'd need to give a good answer, or the characteristics a good answer may have might not be exactly what the interviewer expected or was looking for - but such an answer could demonstrate your competence in the field and be a good answer.
On the other hand, trying to make up an answer, or brazen your way through with jargon while hoping the interviewer didn't notice would likely be a bad answer.
It's not that these companies are looking for a single kind of thought. On the contrary, I believe they value different kinds of thought. Instead, they believe they can sort kinds of thought into desirable and not.
That's what people always say. Like, as if they actually tell from 1 or 2 of these goofy whiteboarding sessions.
It seems the most they can actually glean from these sessions is, "Huh, looks like their problem-solving approach is different from mine. Plus, they come in not having the advantage of having the solution already ready-baked and laid-out for them, in whatever interviewing guide it was I found this problem. Doesn't that suck for them? Not to mention being under pressure from doing way too many, generally carelessly orchestrated interview tracks (in their non-existent spare time) while being stuck in some job they're anxious to quit. And getting distracted by me diddling with my cell phone while they're doing headstands for me. Gosh, why is it so hard to find decent candidates, these days?"
When I mean "right approach", I mean that there are several possible approaches, but only one of them is doable in a few hours in the afternoons, where as the others might take a research team and several weeks. Being able to recognize the right approach, and to ask the right clarifying questions, is important for data scientists, in the same way that being able to choose the correct algorithm or data structure is important for (certain types of) software engineers.
What rubs me the wrong way is when, applying for more or less "standard dev" jobs, one gets asked research-grade questions (i.e. "Solve this cute problem I was fed the solution to recently, either in grad school or in some handout my manager gave me just now. But which was actually an open problem in the literature for 5-10 years before a reasonable solution came down the pipe.")
For a job that ends up being, "Congratulations on making it through our interview process! Now would you mind debugging this {{horribly written X}} that no on here wants to touch with a 10-foot pole? Everyone here has to do 9-12 months of shitwork before moving on to anything interesting here, you see."
I don't think commercial confidentiality makes that a viable generic method...
A lot of these CS professors are themselves maths grads.
now take a statistics major, every class is relevant, and you can still take machine learning in your electives. win win.
I came to this conclusion after I noticed more of my classmates in the mba program (wharton) as data scientists than people in computer science who took machine learning. in fact, _all_ of the CIS majors in machine learning who really wanted to be a data scientist ended up as engineers.
so then I started doing a small search on linkedin, only looking at the big tech company data scientists. selection biases aside, out of 12 profiles: 5 statistics majors, 5 business, 1 biophysics, 1 IT major.
I have also done some looking into interview questions via glass door, and you get grilled on statistics questions. this matches my one interview with uber in 2016. I only got asked 2 ML questions: what is random about a random forest, and in KNN, what happens to bias & variance as K goes to 1
if you want to be a data scientist, you need to learn stats really well or getting past the interview process is going to be very difficult.
Modern ML is applied, layered statistics.
season to taste.
One of the big pushes in Bayesian statistics recently has been to try to figure what the hell all these neural nets are actually doing. It's certainly not the case that the stats have been in the driving seat there.
For anyone that wants to get the background for any of this I highly recommend ISLR (and ESL as a reference). http://www-bcf.usc.edu/~gareth/ISL/
> How do you treat colinearity?
Throw away the redundant part of the data
> How will you deal with unbalanced data where the ratio of negative and positive is huge?
This is very problem-dependent, but it's got the potential to wreak havoc with your learning algorithms. You might get seemingly good results by e.g. always predicting positive. Think carefully about your loss function.
> How will you decide whether a customer will buy a product today or not given the income of the customer, location where the customer lives, profession and gender? Define a machine learning algorithm for this.
This is a messy combination of continuous, categorical, and binary data. I'd encode the data in a vector [salary, x-y co-ords, one-hot encoding of profession, and binary indicator for gender]. Something like a random forest will probably get you most of the way there. Unusual professions could mess with the algorithm, so consider grouping by industry or averaging with a model that omits the profession data.
> Is it useful to apply PCA to your data before SVM classification?
Probably, but it could be data dependent. Semi-supervised learning often improves machine learning models, and it'll lower the dimensionality of your inputs, which will make training / hyperparameter search far more efficient.
> How do you compare a neural network that has one layer, one input and output to a logistic regression model?
Not entirely sure I understand the question, but a NN is basically just nested logistic regression, depending on the activation function.
> From a long sorted list and a short 4 element sorted list, which algorithm will you use to search the long sorted list for 4 elements.
To be honest, I'd use binary search and call it a day. I realise this isn't the answer you're looking for.
> How will inspect missing data and when are they important for your analysis?
You might try to impute the missing data from similar datapoints. Maybe use something like K nearest neighbours on the non-missing components to deduce the missing values. Got to be careful doing this though, since it could massively bias your analysis: consider using a special encoding for "missing data" too.
> Estimate the probability of a disease in a particular city given that the probability of the disease on a national level is low.
This is a tough one, since there are so many ways to answer it. I think I'd start by writing down a list of factors that might allow a high prevalence of the disease locally despite a low prevalence nationally. Weather? Local wildlife? Proximity to major transport hubs? Then I might suggest a simple model like naive Bayes.
> How do you treat colinearity?
Use an LASSO as a feature selector or regularizer.
> How will you deal with unbalanced data where the ratio of negative and positive is huge?
User F1 score instead of accuracy as a performance metric. Try using cascade classifier.
> How will you decide whether a customer will buy a product today or not given the income of the customer, location where the customer lives, profession and gender? Define a machine learning algorithm for this.
More or less, the same as what you said.
> Is it useful to apply PCA to your data before SVM classification?
No, since kernel-SVM optimization is dependent on data (similarity) and not features. Also, we will be projecting the data into a basis expanded space anyways, so the dimensional reduction would be redundant.
> How do you compare a neural network that has one layer, one input and output to a logistic regression model?
An MLP-NN is the same as a logistic regressor, if all connections as 1-1 (no basis expansion), no activation function in the data-1st layer connection and Sigmoid as activation function in the 1st layer-output connection.
> From a long sorted list and a short 4 element sorted list, which algorithm will you use to search the long sorted list for 4 elements.
I am dreadfully bad at core CS questions...throw some AI questions at me. XD
> How will inspect missing data and when are they important for your analysis?
You might try to impute the missing data from similar datapoints. Maybe use something like K nearest neighbours on the non-missing components to deduce the missing values. Got to be careful doing this though, since it could massively bias your analysis: consider using a special encoding for "missing data" too.
Love your solutions. I might also try using cascade classifiers.
> Estimate the probability of a disease in a particular city given that the probability of the disease on a national level is low.
tough question. Would probably involve me asking a lot of questions about what data I have to play with. Very interview specific.
Please correct me if I'm wrong, I'm still just learning this stuff.
PCA isn't lossless either. We are definitely trading off some richness in the data for efficiency. When the gain in efficiency is negligible (as is the case with SVMs) the trade off doesn't look as tempting.
I am student in the area as well, so take my words with appropriate caution.
Colinearity can be handled either through variable selection, or dimensionality reduction. The right approach depends on the data. If you're trying to identify the root cause of some phenomenon, L1 regularized models will pick out variables (though they might pick the wrong one). If you just want to prevent colinear variables from taking over your model, dimensionality reduction is the right choice.
> How will you deal with unbalanced data where the ratio of negative and positive is huge?
This is one area where bayesian statistics really shines. The variance of the posterior predictive distribution for a bayesian model is inversely proportional to the amount of data you use for inference.
> How will you decide whether a customer will buy a product today or not given the income of the customer, location where the customer lives, profession and gender? Define a machine learning algorithm for this.
Random forests are a good answer to this in terms of predictive accuracy. The problem with random forests is that they don't give you any information about how certain your estimates are. A better technique to use would be Gaussian processes (GPs). GPs are a kernel method somewhat similar to SVMs, but they have the marked advantage that they provide a posterior distribution over their estimates, rather than just the maximum likelihood value.
> Is it useful to apply PCA to your data before SVM classification?
The right answer is that it depends on the shape of your data and the specific SVM kernel/hyperparameters you're using. If you have a square matrix and you're doing linear SVM, PCA first will be slower. On the other hand, for a highly asymmetric matrix and nonlinear SVM, PCA done by stochastic approximation of the SVD, followed by SVM will be much faster. From a prediction accuracy standpoint it isn't going to make a large difference in either case since the kernel trick effectively projects your data into a high dimensionality feature space.
> From a long sorted list and a short 4 element sorted list, which algorithm will you use to search the long sorted list for 4 elements.
To answer this properly you need to know if the list has repeated elements. If there are no repeated elements then a binary search on the first element of the 4 item list, then scanning forward from there is the correct answer. If you admit the possibility of (possible massive) repetition of elements then this is no longer the case.
> How will inspect missing data and when are they important for your analysis?
This is yet another case where bayesian analysis shines (can you tell what I specialize in?). With bayesian methods you don't have to impute a value, rather you can treat unknowns as coming from a specific distribution (which can be the exactly distribution of that variable, via empirical bayes). This prevents you from introducing bias in your model.
> Estimate the probability of a disease in a particular city given that the probability of the disease on a national level is low.
This question is kind of bad given that it doesn't put a bound on what data is available to you. In terms of model, this is just a classification problem in disguise
> How do you treat colinearity?
Look at feature interactions when using decision tree based models to discover if some interactions between seemingly similar features could improve the performance of the model.
> How will you deal with unbalanced data where the ratio of negative and positive is huge?
Don't use accuracy or other "averaging" evaluation metrics. When looking at unbalanced data you are likely interested in the minority class so I would use metrics that only look at this class for performance evaluation.
I recently graduated from a masters of business analytics program and would have humbly assumed that the breadth and depth of knowledge asked for in these questions would be beyond my capacity, but it was actually pretty spot on with what I had learned. Strictly speaking, I am far from an expert in statistics/math/comp sci. But these seem like reasonable questions. The best one is
"How will you decide whether a customer will buy a product today or not given the income of the customer, location where the customer lives, profession and gender? Define a machine learning algorithm for this."
This is a difficult question because for someone who has played with data science before, the answer seems to be obvious: throw the data into your favorite predictive algorithm and see the result (worst case: deep learning). What they are probably looking to see is a discussion of feature engineering, hypothetical exploratory analysis, and an effective way to present the answer (straight probabilities, weighted loss functions, outlier detection, etc.)
- A big company on your resume has no outside value to your prospects.
Seriously. There's not "hey well he worked for amazon so we know he's good" free pass in the future for having Amazon on your resume.
There might be a "well, she wasted 3 years at amazon and we all know that's a hellhole so we can get away with abusing her" signal though. (Amazon is a hellhole, I spent 2 years there, in retrospect I would have been better off walking the day it became clear they were using cult methodology.[1])
It took me many years to find my calling. Once I did then my career direction was set. I'll only be working for companies in this industry going forward, because it really is my calling.
Take a job that maximizes your creativity and ability to contribute.
Don't chase big money or big names. The two almost never go together, anyway.
[1] Magical Phrases are a key indicator of cult methodology. Also known as "thought terminating cliches" they are multi-purpose sayings that are used to invoke company ideology and shut down dissent. At amazon there are things like "It's day one!" and the like. Every time I've seen these cult signals at a company it turned out to be a toxic work environment.
A good question to ask in interviews is if they company has any slogans all the employees know.
Do you have evidence on that? Virtually all data I've seen from salary sharing threads, and all compensation discussions I've had with startups suggests the contrary.
I respectfully disagree. Recruiters and hiring managers, and even individual interviewers take me far more seriously after having "Google" on my resume than they did before. It's not enough to land a good job automatically, but most interactions now at least start from the presumption that I'm competent, which makes a job search less frustrating.
I know some well-known niche companies with highly-coveted positions that appear to make new hires almost exclusively from AppAmaGooBookSoft alumni (last time I mentioned this, a user correctly guessed one of the companies I was referring to from just such a vague description).
The reality is that most people hate hiring and have no idea whatsoever how to do it well (in the sense of bringing on competent people, not from the HR perspective of minimizing legal liability). Big names, in either education or employment history, help a lot. It's probably easy to underestimate how much they help if you haven't gone without them.
I would also venture that ex-BigNameCo is a lot less of a big deal if you work somewhere that's a hotspot for that type of company (Silicon Valley, Seattle). There it may be seen more as a sign of rejection than success, like "Why don't you still work at Google if you were Google material?". But even still, most companies are going to be swayed significantly by a big name on the resume.
As for name recognition, it always helps. No employer is going to hire someone solely based on where they went to school or where they worked previously. However a well-known school or company can be the deciding factor between two candidates who are otherwise equal. My wife went to Purdue and did her pharmacy residency in Colorado. She later found out that she was competing with three other candidates for one position who were equally as good in the eyes of the residency program, and she won because she graduated from a well-known, top 10 pharmacy school.
First, because I think you're factually mistaken - for a lot of people, a big name on your resume is a huge, positive flag.
Second, because there are other effects of working at a large company which you're not considering. Most importantly - they're large! That means you're going to interact with a huge amount of people. In 3 years at a BigCo, you'll likely have contacts not only in various departments, but probably in several other companies as well, both because of work with outside companies, and because of interacting with employees who left.
Not to mention, you can learn a lot about the industry that you might not necessarily be exposed to while at a small startup - via interactions with those outside companies (customers, partners, etc). You can also work on seriously big things, which you won't be working on at a startup, for example.
None of this is to say that working at a BigCo is the right move. I actually tend to think the opposite, depending on various factors. But it's definitely not an all-downsides proposition.
I might not have worked on my passion in specific there, but I had a great time, learned a ton, and IMO set myself up for a great trajectory in my career.
Like most other large companies, YMMV depending on where you're at.
The point of a CV is to get you an interview. A person who graduated from an Ivy/Russell, did a year or two at Google and a similar spell at Goldman Sachs, will be able to get an interview anywhere, for the rest of their lives.
They probably won't get an easier ride in the interview, it may even be harder as the interviewer's egos insist that they best one of the "elite", but they will never need to worry about passing a screen.
Nowadays I am looking for a new permanent position (have been freelancing for a couple of years). My wife happens to be a headhunter and knows how most HR and internal recruitment teams think and work. She tells me with Amazon in my CV many more doors would open. I don't regret turning them down, I know for sure had I accepted to work with them I would have never met my wife, who brings me more joy than any job would ever do... But I can't help to wonder...
Me: "Hey how come you guys don't use source control?"
Manager: "Are you being an asshole?"
Right, because BY FAR they are making the most progress, in the most promising technologies today, and they pay egregiously well.