Hacker News Reader

Comment by vinay_ys | Hacker News Reader

Good data scientist described here seems to have unrealistic expectations at super human level of know-it-all/do-it-all.

I think there are more well-established job architectures like business intelligence analyst, data engineering, user experience designers, product manager, software engineer etc - these roles in combination serve to do a lot of what is described here as data scientist. These roles are easier to hire, have well defined career paths and good ways to get job satisfaction and can scale well as the business-problem-space/orgs grows.

I think the scientist label should be reserved for those who actually do the scientific mathematical research – specialists who have done deep research in specific areas.

For applying pre-existing sciences to solve practical business domain problems, we need lots of engineers, analysts and managers etc who are all trained with AI-first software development practices and just a few specialist data scientists.

View on HN

Agreed. The second point about pipelines stuck out to me:

> [Good DS] will often build these pipelines themselves. Bad DS thinks it is someone else’s job.

In a small environment, sure, do the job so it gets done! But in larger more corporate settings the 'cowboy' approach to pipeline building is not sustainable or even feasible. Am I a bad DS because I can't provision VMs, open firewalls, replicate production DBs and build hooks in other teams' services to expose data? No, its not my job. A good DS collaborates with other teams and sysadmins to build a pipeline that is maintainable and monitorable, and doesn't do it all themselves.

Completely agree a DS shouldn't be responsible for all those things you mentioned (all the infrastructure, networking, security, etc.). In today's age, "data pipelines" can look like plugging into tools & frameworks that abstract away a lot of this complexity. For example - fivetran for extraction & dbt for data modelling. So once those engineering/devops heavy things you listed are abstracted away, I think it's fair for a DS to be responsible for building the data transformation logic in the pipeline. Of course, not all companies will have such tooling available, in which case it's not fair to place those expectations on a DS.

Spending all your time on the clock acting as your own database admin, data engineer, app developer, sysadmin, etc. is a great way to accomplish very little data science and burn yourself out in a hurry.

Source: personal experience.

Are these useful skills to have? Yes, especially if it gets you out of a bad situation and makes a customer very happy.

Are these requisites for being a "good" data scientist? No; arguably, they are distractions and wasted room in your brain.

> Good data scientist described here seems to have unrealistic expectations at super human level of know-it-all/do-it-all.

Hmm. Know-it-all/do-it-all is a useful standard to strive for, though, even when, in practice, one will often fall short in one area or another.

One of my personal frustrations is that I have invested heavily in trying to be well-rounded and it doesn't quite pay dividends because of how often I find myself confronted with prejudice of the form "because he's good at X, that probably means he's bad at everything else". For example, if the first impression I leave on someone is that I'm good at math, they'll often jump to the conclusion "because he's good at math, that probably means he's bad at databases". If the first impression I leave is that I know a lot about finance & economics, they'll assume "because he knows a lot about finance & economics, that probably means he can't do projects in a technical domain" and so forth.

> seems to have unrealistic expectations

Well, the expectations aren't unrealistic - if you were to grant the "good data scientist" a reasonable amount of time rather than demand that everything be done by this afternoon, which is what most "real data scientists" are up against.

"This is what you need to be able to do to call yourself a data scientist and become employed. These are the table stakes."

vs.

"Here's some ways to become better, if you want to be better"

Reply on news.ycombinator.com