Hacker News Reader

Comment by ryndbfsrw | Hacker News Reader

I’ve worked in the field for 7 years now so not that long but long enough to build some heuristics. The best data scientists are just people who try to understand the ins and outs of business processes and look at problems with healthy suspicion and curiosity. The ability to explain the nuances of manifolds in SVMs is not something that comes into it outside these contrived interviews. I prefer to ask candidates how they would approach solving a problem I’m facing at that moment rather than these cookie cutter tests which are easy to game and tell me nothing

View on HN

>> I prefer to ask candidates how they would approach solving a problem

Word. Totally off topic, but: I work in the field of information technology since more than 20 years now, more or less. Not always the same focus, not always full time, but always IT related. I consider myself a good problem solver because of my self learning and analytical skills.

I recently applied for a job as a BI developer. The interview consisted of 10 questions about SQL. I more or less answered them, just 1 or 2 wrong. Not wrong as in "not correct" but rather "Not what we exactly expected" or "you did not see the little traps".

Comes out they didn't take me because of my lack of SQL skills. I do not understand how this kind of recruiting process will help anyone getting skilled people and how this is still common practice. It's frustrating for people like me, who do not have the complete SQL syntax in mind, but are flexible in choosing their problem solving approaches. A couple of years ago I started in a big data company, never heard of MongoDB before, little skills in Bash. If they would just asked me questions about that, hiring me would be a total no-go. They did hire me. I improved process, like measurable, and mastered MongoDB. Nothing, that one could expect from a questionaire.

A second interview, same outcome. They not even asked me detailled questions, just wanted to know what my SQL skills are. I answered: Immediate, but I'm good in learning. The did not take me, too.

Although, I understand that it's hard to evaluate this kind of skill, I'm really frustrated, when I face those "hiring techniques". Or maybe I'm just not good in SQL, and they anticipated it.. ;)

I can try to help you pass the SQL round. I’m going through DS/Analytics Eng interviews myself. I’ve passed all of my technical rounds, but failed the final rounds so far. Email me at aok1425 at gmail.

I think this must happen in all computer engineering fields. It's happened to me numerous times when applying for DevOps positions.

Interviewers don't seem to realize that possessing knowledge and fluency are a trade-off. If I'm amazing at SQL, I'll have a gaps elsewhere and vice-versa. There's just too much to learn, and stay on top of.

My takeaway when I fail an interview due to nonsense like this is that these aren't places I would've been happy working at anyway so they did me a favor by not hiring me.

> Comes out they didn't take me because of my lack of SQL skills.

Good. You don't want to work there. It's not a good place to be in. That signal is very clear.

There's another possibility but only because you mentioned SQL specifically. People often use SQL as shorthand for "understand how to manipulate data," and if a new hire doesn't have this skill it can really set a data team back, so interviewers are touchy about SQL. It would be helpful if they clarified if it was the SQL syntax skills or the data manipulation skills they had a problem with. But generally I agree with you that companies should prioritize problem solving skills over technical minutiae.

> But generally I agree with you that companies should prioritize problem solving skills over technical minutiae.

Unless, the company's goal is to hire someone with technical minutiae that they can pay the lowest wage possible :)

The - to my mind - most absurd version of this was a well-known Swedish fintech I interviewed with where the one barrier they had set up was one of those online IQ tests (rotating, mirroring etc. blobs on a grid).

I was interviewing for a senior data science role and the other points of contact I had in the process were surprisingly non-technical conversations with a VP of data and another senior data scientist.

Alas, my skills in rotating blobs on a grid failed me that day so that was that.

Yeah, I instantly know which company you're talking about...

I know a couple of developers there and as far as I understood all applicants have to do the IQ test. They also thought it was ridiculous, but the CEO really really likes them, so it stays.

The way I see it, an interview goes both ways. They try to assess your fit in the company, and you're also trying to assess if the company is something you want to spend your time on.

I call this the "game show style" interview. You know the answer, you get $10k. Next question. You didn't know the answer? You're out of the show. Next contestant.

To me this is very disrespectful of your time and your capabilities. Once I was pumped to go into an interview with a very well known company in Sydney, but was dismayed to learn that they do this game show interview (with IQ test, no less!). I wouldn't want to waste my most precious resources (i.e. time) for a company that doesn't respect me.

I completely agree! I run a data science department at a corporation and I give applicants real world data and ask them to answer real world questions. "Here's a dataset of the dates we sent an email and the open rate. Does one day of the week have a higher open rate than the others?" People have to know how to turn a string into a date, turn a date into the day of week, and then apply statistical reasoning and then give me a one sentence answer. That kind of thing is far more common at work than explaining the kernel trick.

Agree. In addition to curiosity another quality I think is very important is persistence. A surprising amount of being an effective data scientist comes down to being able to learn new things quickly and being able to make the computer do what you want it to, especially when the first few things you try don't work.

Success if often a trial and error process, involving slowly building a deep understanding of the problem you're trying to solve (business and technical), hitting lots of problems, and not giving up too easily (at least, not giving up due to surmountable technical hurdles).

This often means spending many hours banging your head into a brick wall; but in my experience these are often the times I'm learning quickest even if it doesn't feel like it at the time.

Diving deep into a domain and interfacing with a subject matter expert goes a long, long, way. We've been building custom data products for enterprise for about seven years, and a lot of the work is listening and grokking large amounts of content about whatever domain we were helping with in general, and the specifics of our clients. Retail, banking, telcos, energy, communication.

Having a background in acoustics, reservoir characterization, telecom networks, opens up clients because you 'get it' or at least you work hard to get it, which improves buy-in of the experts to sit down with you and answer your questions. You did your homework.

If you don't and just storm in talking about something something neural nets, they'll see it as a waste of time, won't bother explaining nuances, will delay sending data you desperately need. You won't have their cooperation even if you have executive support. There's no data in CSV form or an API to hit in most real world projects, so you need their help getting data, and their expertise to understand it.

Another major point is specifying the metrics. The real world metrics, not AUC or F1 scores. You need collaboration to get there, too.

There's so much to be done before there's data to work with, let alone good data. And there's so much after the model building step.

It can drive people to quit. One reason is that when you storm in and consider that people are morons, you get frustrated rapidly.

>> If you don't and just storm in talking about something something neural nets, they'll see it as a waste of time, won't bother explaining nuances

>> Another major point is specifying the metrics. The real world metrics, not AUC or F1 scores. You need collaboration to get there, too.

>> you need their help getting data, and their expertise to understand it.

All this x100. And it's made worse as this humility isn't taught in schools and rarely asked in data science interviews

Humility is discouraged in software engineering with job ads for ninjas and rock stars.

It should be taught. Lack of humility almost cost us a project. In that instance, humility unscrewed the project and unblocked 400k by simply sliding a sheet of paper and a pen to the client across a table, and asking them to draw the dataflow they thought we were going to build. A couple of boxes and arrows made it clear what the problem was. We drew the actual dataflow. The security person said "Oh, I thought... OK.. if it's like that then we're good to go". Legal said they're OK with that. Data people said OK. A dozen people were relieved.

The previous person on the project, although brilliant technically, thought they were "idiots who didn't understand crypto", as if it were the end goal. All it took was to keep quiet for a second and listen to what they had to say, and let them talk about what was problematic, instead of snark.

I am actually starting to believe that (too) smart developers can actually be a hindrance in the wrong circumstances. We have a prima dona on my team and while smart, he doesn't make things easy to follow for the next dev.

> The best data scientists are just people who try to understand the ins and outs of business processes and look at problems with healthy suspicion and curiosity

I guess the unsaid part of this is “... and curiosity AND are often able to leverage this in data-driven solutions to bussiness problems”. Because nobody cares whether you’re curious or not if you don’t bring any value to the company. With that said, the part you mentioned almost seems like an innate ability, while the part I filled in could probably be trained.

Suppose you were still in college and wanted to become a data scientist. Based on your current knowledge and values, how would you go about it, being a student? Which skills would you hone and how?

Yes, it’s a given you need to understand what the different techniques are doing and when to apply them, but rarely to the level of depth expected in these white-board interviews that are often just thinly-disguised ways to grill and belittle nervous candidates.

If I could redo my twenties, I would tell myself to choose some topic I cared about, find publicly available data on that topic, and just start exploring the dataset with basic pivot tables and graphs. Ideally write up what you found and publish it somewhere. As you find interesting things in your data you'll naturally start asking more questions and you'll learn modeling techniques as a function of those questions and writing about will help you become a clear communicator (this matters far more than technical knowledge)

This is why I'm switching from DS to SWE. The communication hurdles with the business people are so hard for me. I've talked with other nerds my whole life and struggle to connect with and dissect the other side. That and the pay is better.

SWE has its own problems. Business will want X done by Y time and doesn’t speak Agile/sprint/tech debt/etc. Also, you’re going to deal with resume-driven developers who want to use some shiny new tech when there’s no valid reason over something plain and boring that works. The grass is never greener.

Reply on news.ycombinator.com