Comment by classybull | Hacker News Reader

Comment by classybull | Hacker News Reader

Parent Full thread

Becoming a data scientist on your own is exceedingly difficult because, despite their purported adherence to objective data above all else, the practice of data science is full of people who consistently appeal to authority via educational credentials. You can see it in this thread. They regularly make the mistake of thinking that because the skills necessary to be successful in the field correlate highly with advanced degrees that means that only people with advanced degrees should be able to participate in it. They generally make it very difficult to objectively evaluate an individual's skills because their injection of bias into the candidate evaluation process.

Its regressive and completely out of step with the supposed meritocracy we like to think we follow in tech. Its also the path towards cartels. I get the feeling a large portion of data scientists would like to create the American Data Scientist Association, with credentials and bar tests.

Yeah I kinda get that feel as well. The thing that makes me suspicious is the amount of unnecessary and obfuscating jargon that gets thrown about. They've even invented new jargon to replace perfectly confusing old jargon (e.g. your model's "error residual" is now your "function cost"). I've spent the past couple of weeks doing a bit of ML vision stuff. Most of the terminology was lost on me, at least until today when I discovered "Machine Learning is Fun": https://medium.com/@ageitgey/machine-learning-is-fun-80ea3ec...

I think I learned more in an hour than I did in the past week, thanks to this series. The author actually bothers to explain concepts (that turn out to be fairly simple btw) like 'gradient descent'. Highly recommended read if you have the time and interest. Just to whet your appetite:

...current machine learning algorithms aren’t that good yet — they only work when focused a very specific, limited problem. Maybe a better definition for “learning” in this case is “figuring out an equation to solve a specific problem based on some example data”.

Unfortunately “Machine Figuring out an equation to solve a specific problem based on some example data” isn’t really a great name. So we ended up with “Machine Learning” instead.

First of all, errors and residuals are different things.

Second, "cost" is not new jargon. What is relatively new is thinking about probabilistic modeling in terms of abstracted cost functions, but only relatively.

There are dozens of tutorials, courses, etc that are clear and don't introduce unnecessary jargon. Nobody is trying to keep you out of data science.

As for the term "machine learning," it's because what we today call ML gree out of actual AI research. It so happened that a lot of progress was made very quickly by the ML researchers, so the ML-oriented terms became popular as some older statistics terms were subsumed.

This is true of most domain. The jargon often gives the impression things are really complicated, but often time it's just that you don't know the domain language. Even with math, 60% of the challenge is remembering the notation and substituting it in your head with the real construct.

Now, as you get into advanced topics of any field, you'll start to find difficult concepts, but 90% of the time, the foundation just appears hard because of the language you don't know.

Also, keep in mind Machine Learning is not "Machine Figuring out an equation to solve a specific problem based on some example data”. Machine Learning is a subset of AI which focuses on teaching the computer how to perform a task without explicitly programming the task execution logic. Currently, the known practical technique of doing so can be described as "Machine Figuring out an equation to solve a specific problem based on some example data”. This might not be true in the future, as better ML techniques are researched and discovered.

Also, it's good to keep in mind ML is a software engineer discipline, and data scientist just benefit from modern software, the same way Excel created jobs, this technique of ML created jobs. In the future, different ML techniques might create more jobs or replace the current ones. This puts the data science field at the mercy of ML research. Already I think it's been hard to keep up as a data scientist, since ML research is being financed greatly and advanced quickly.

One of the main aspect of Data Science is... Science.

I know some people who I would qualify as scientist without a PhD, but they are extremely rare. On the other hand, I've seen countless unqualified people apply and get (lousy) data science job because the "title" itself is very vague. I'd be more inclined to hire a PhD (who's been through a fairly painful scientific training) in, say, material science, then train her to data/programming, than get a good programmer and train her for science.

I'm not for a "Data Scientist Association"[0], but I'm not for diluting the value of all the effort I made to effectively become a data expert AND a trained scientist.

[0]Rant: there is a world outside America.

I have the opposite problem: I constantly feel inadequately prepared for my job as a data scientist because I never got a PhD. I know mid-career doctorates are considered risky but I still have my mind made up to get one.

The PhD is irrelevant. What is relevant is the experience of doing science in a rigorous fashion. Today's "data scientist in a box" lessons don't give people the basic understanding of how to approach a problem or even what a null hypothesis is. Some of the videos on Youtube make it seem like any correlation is relevant when they don't even know what they don't know.

I can't agree more: PhD shows that you went through the pain of scientific training, you can however train yourself to science in different manners.

Start from the start though: epistemology is grossly underrated, but the scientific method (aka: calling bullshit) is my most valuable tool.

Your comment is really refreshing, thank you

Reply on news.ycombinator.com