Hacker News Reader

Comment by dannykwells | Hacker News Reader

I mentor individuals looking to transition into data science. I can say without a doubt that those who do the best (get the best jobs) are those with domain expertise and who are able to leverage that expertise in a data science realm.

Of course, domain expertise is hard to come by and takes years to develop. I look forward to data science leaving far behind the notion that "you too can become a data scientist with 3 weeks of Python + scikit-learn".

View on HN

>"you too can become a data scientist with 3 weeks of Python + scikit-learn"...

I don't think any serious practitioners were out there thinking in this manner. But there is a gold rush right now, and any gold rush will attract its share of charlatans.

There are a whole ton of students coming out of one year masters DS programs, who don’t know that much more.

Obligatory xkcd https://xkcd.com/1831/

I see it all the time in my domain (search). Novice data scientists getting into a new field quickly get frustrated with existing practitioners. Existing practitioners want to use tried and true methods; data scientists want to reinvent the field with ML. The existing practitioners don’t know how to manage the usually black-box ML solution. Data scientists don't know the domain enough to be effective. So it doesn’t go far.

It’s a dynamic that sadly burns out overly optimistic, but smart data scientists and sadly leaves a negative impression on existing practitioners on the promise of ML. Those practitioners stick their head back in a hole instead of innovating.

All to say with more explainable ML (and more humility on everyone’s part) more progress would be made.

> I don't think any serious practitioners were out there thinking in this manner.

But a lot of thought pieces / YouTubers are pushing it, which is a problem.

In fact, many quantatitive researchers in their domain are already data scientists without using modern tools like machine learning. Being able to use these tools is only the beginning of a new practicing and advancing.

I consider it the responsibility of computer scientists and engineers to make the tools better and easier. But unfortunately, it's that one has to become a good computer scientist first then a domain expert.

We are not doing enough on innovations of tooling.

Computing people in general seem to have forgotten the whole reason our field exists is to build tools for other people, the ones who do the "real work". With the possible exception of video games nothing we do with computers is useful or interesting on its own, its only point is to assist other fields. So we can have all the advances in the world but if we cant make them into useful tools, what was the point.

well, as long as your serious practitioners aren't trying to get hired they should be just fine

Statistics expertise is also very important. Stats may be less sexy today than "data science", but they were doing "model explainability" before it was even a thing!

>"but they were doing "model explainability" before it was even a thing!"

Do you have an example?

Linear Regression

Check this: https://news.ycombinator.com/item?id=19353204

To an extent, CART.

But more classically, any structured statistical model with (eg) terms for variable interactions, measurement noise, and hierarchy.

Prior to model explainability was experimental design.

I don't understand what it can possibly be without statistics expertise. If you take away that, what's left? a A not-very-good programmer?

I always took data scientist to be a rebranding of statistician. (To be clear there's nothing wrong with rebranding.)

Well the common description of a data scientist as "better at stats than a programmer and better at programming than a statistician" can also be reframed as "worse at stats than a statistician and worse at programming than a programmer" in most cases. I think it's typically a rebranding of PhD in a science field realized that market for that educational track is very small so rebranded as "hottest job of the next millennium"

I wasn’t familiar with that formulation. Are contemporary stats grads really any weaker in programming than data science certificate holders?

I can buy that there’s a shortage of statisticians and so industry needs other people to pitch in, but it seems like the floodwater of incoming students should be directed to stats programs and not data science ones.

Statistics is boring and uncool.

ML is hip and profitable.

Hoenstly, it's mostly garbage. The original notion of data scientists was invented by FB for a very specific set of skills (social science PhD's with Map-Reduce and experimental design), but it's a cool title and thus it got spread across multiple roles.

It's super weird though, despite doing data sciencey work for about a decade now, when I changed my title on LinkedIn to be data scientist, I started getting offers for jobs that paid a lot more money, so there's an incentive on the candidate side to re-brand.

But yeah, ultimately all the job is is some stats, some code, and some communication. Don't get me wrong, its a great job and its hard to find people who are good at all of this, but in my experience the limiting factor is definitely the statistics and the domain knowledge rather than the code.

Honestly what should your salary be if you have domain expertise and are good with data science tools (and tools surrounding them like git, CI/CD, etc.)?

Reply on news.ycombinator.com