Radical Change Is Coming to Data Science Jobs | Hacker News Reader

Radical Change Is Coming to Data Science Jobs

forbes.com

29 pointsideaoverload10 comments

Interesting discussion on this article over on the machine learning reddit https://www.reddit.com/r/MachineLearning/comments/awu86b/dis...

I particularly agree with the comment saying:

no software is capable of trawling through the bowels of the organisation to find out the correct interpretation of the `Extra2` field on the `Sales` table that takes three values: "TRUE", "Error" and null.

This, data cleaning, and understanding how best to store the data for better insight are the true bulk of data science work, very little is the shiny model building work. I guess this is close to the Industry specialist as outlined in the article though

That more likely falls in Data Engineering part.

While some data science tools may be `excel-implementable` (or some equivalent) in five to ten years, there's significant risk of mis-implementation of results given how much of a mystery many methods are to the people who'd use them.

I'd compare data scientists more to CPAs. You can have software like TurboTax and Quickbooks, but CPAs don't seem to be going anywhere. Similarly, anything that's more complicated than cookie-cutter data analysis will require someone who knows how to develop, build, and debug the algorithms themselves.

Use cases of data science are often too specific for most data science to turn into button pushers. Look at the vast array of ways a neural network can be implemented. Which one of those implementations will be in the `excel package`?

I think the counterargument is that there will be some standardization of these methodologies such that they don't require experts to implement. Similar to how it used to require a programmer to develop any webpage at all, but now most common use cases are covered by simple off the shelf solutions like square space or WordPress. Moreover, the skills will disseminate quickly to other fields. Originally only accountants used Excel and now it's an indispensable skill for most professional jobs. So worker bees of the future will know the differences between various types of NN just like the modern worker bee can write vlookups or create charts in Excel even though that would have been a mysterious skill thirty years ago. Meaning your theoretical Excel package would cover many different types of NN, with users expected to understand the difference.

Pretty uninspiring content marketing piece.

Radical change is coming technical jobs in all fast moving fields, whether in biotech / software / hardware / ... all the more reason to spend enough time learning new things.

Man that's a nothingburger article.

>Over the coming years, I foresee data scientists dividing into at least five types of workers

Yeah, sorry my dude; that happened before "data science" was even considered a profession.

The other tools; trifecta is a helpful thing, but I doubt it's helpful enough people will actually pay for it. Auto-sklearn/DataRobot was the result of a DARPA request a few years ago, and doesn't even vaguely solve the right problem. It's also just R-caret which has existed for 10 years now.

My prediction: data science in 5-10 years will look pretty much the same as it does now. Just like aircraft in 5-10 years will look pretty much the same as they do now, or did 5-10 years ago.

Open source packages such as Uber Ludwig (https://eng.uber.com/introducing-ludwig/) - let alone the commercial stuff - makes you wonder the same thing. Such tools should automate a large portion of the mundane parts of the data science workflow. While there's a lot more to the job than building/training a classifier, I wonder what the field will look like once that part is commoditized.

Here's one way to read it - I just removed the word "data" to see if it made sense.

> "A radical change is coming to science jobs. Similarly, I believe the job of a scientist as we know it today will be barely recognizable in five to 10 years. Instead, end users in all manner of economic sectors will work with science software the way non-technical people work with Excel today. In fact, those science tools might be just another tab in Excel 2029."

Not just data science jobs: I think most ‘knowledge worker’ type jobs are going to go through large changes: AI assistants that will make people very much more productive in their jobs, and fewer jobs available because company’s will get more work per employee. I expect salaries to soften, with the exception of people who are at the very top of their fields.

That's funny, didn't expect to see that my old boss Nate Oostendorp wrote this.

Anyway, the sooner enterprise stops putting data science in an ivory tower the better off we'll be. The number of Fortune 500 clients I have who can't seem to grasp that an algorithm or model by itself is not a business solution is disheartening.

Reply on news.ycombinator.com