Hacker News Reader

Comment by collyw | Hacker News Reader

That's what I am thinking. Any half decent software engineer could probably produce the results they need from a few SQL queries. But "data science" is hot at the moment.

View on HN

Comments like this reinforce the idea that data science is too broad a term. Everything data-related is thrown into the "data science" bucket.

You describe what I'd call business intelligence or even advanced analytics. SQL jockeys (not a pejorative) really slicing into the data and presenting it in meaningful ways. There's usually exists some science behind these exercises.

Then there's machine learning, which also gets clubbed into the data science camp. Try capturing a convolutional neural network in SQL - you might go bald from pulling your hair out. It's totally not the tool for the job.

While perhaps any half-decent software engineer could maneuver his/her way around the great ML libraries out there to produce a CNN (or something of the like), I would hesitate to give them a project that involves true science (bring out your pitchforks). Experimental design, hypothesis testing, etc. While being a SQL expert could help turnaround time and enable you to test more interesting hypotheses, I wouldn't leave most DBAs to develop and carry out an experiment.

So I guess my point is "data science" is too broad and overused a term to be meaningful.

True, but many people jump on the hype wagons and assume that they need machine learning, or whatever the flavor of the day is when in fact they need something far simpler.

A lot of "machine learning" is the more traditional statistics-based data science, not deep learning using a CNN or something...

I understand that, but my point is good luck doing a lot of that "statistics-based data science" using just SQL.

There's a really good video here, seperating what "data science" means to different companies

https://www.youtube.com/watch?v=xC-c7E5PK0Y

TLDR

- some smaller -mid-sized companies call datascience are actually referring to software engineering / data engineering

- Smaller companies with datascience might be doing software engineering as well, if no data pipelines / collection datasets available (they are using the term wrong here)

- most companies calling for AI/deep learning don't actually meet the requirements for it.

- You don't need a data scientist until you have SWE's / data engineers

- Machine learning is more narrow smaller defined scoped problems / dataset, for things like text classification and analysis. Things like identifying what number is drawn out on a piece of paper using MNIST, etc.

basically, data-science could mean a lot of things and is pretty ambiguous. Its the new software-engineer vs web-developer terminology.

A half-decent software engineer can easily and quickly produce a lot of data products.

If you divide the responsibilities of a data scientist into two broad categories:

* Can I get the answer I need/build the product I need?

* What is the question I should be answering? What do I need to build?

Software engineers tend to do quite well at the former, but are not necessarily guaranteed to be successful at the latter.

In my experience its the software engineers bosses that are poor at the later.

That really depends. I'm working sometimes on problems that physician researchers have been cranking away on for decades. We want to do better than their state of the art. That's not something you can throw a software engineer at. There is a pretty wide spread of problem difficulty out there.

Reply on news.ycombinator.com