The actual process of collecting, aggregating, cleaning and verifying data is a hugely important skill, and not one I've really seen typical data scientists possess.
And I would warn you from my experience teaching statistics to undergraduate engineers...they are not going to be much better. Regularly get 'hey we have this data what test can we run?' 'what are you trying to show?' 'we don't care we just need to run a statistical test' conversations.
Then they are not scientists. They have a label "scientist" but lack of rigor of actual science.
I don't see why changing the label to "engineer" would suddenly make them have rigor.
This is sort of the meta failure of the argument. They are arguing that people's data skillsets are wrong. To make that argument they are analyzing based on the wrong variable in a data set.
Data in the classroom setting is pristine and beautiful; data in the real world is messy and buggy. You have to get burned by buggy data a few times (or maybe a bunch of times) in the real world to learn to look for bad data smells -- I don't think schools effectively teach this kind of intuition, regardless of whether the students are training as data engineers or data scientists.
If data scientists are spending more time in school getting advanced degrees, they're not getting as much exposure to buggy data, whereas data engineers with a BS and a few years of industry experience would already have built up this skill.
I got to take over our department's undergraduate statistics course a few years back.
The first change I made was all homework, tests, and projects used real data set. I intentionally have them collect bad data (they don't know its bad before hand). First day of class we collect data using the board game operation...I give basic instructions and then halfway through ask everyone to stop and agree on how they are entering data for the variable of 'success or failure' of the surgery. Oops...
In my experience teaching the course, the reason the students (engineers) find statistical reasoning hard is:
* They have never been given anything 'broken', everything is curated to avoid things not working. The result is they think data has inherent meaning. A right answer.
* Their entire learning experience has been stripped of context and the need to make decisions with information. They can give me a p value but are terrified (not unable, just unwilling) to interpret it or give it meaning.
* They have never encountered the concept of variability...everything is presented as systems with exact inputs and outputs.
When I work with postdocs, I sometimes (less frequently) encounter many of the same challenges. Data is treated as sacred and external and inherent. It's wild to me.
If there is a question mark here - it's really how much value are we deriving from all of these data people?
Where is all the ML that's changing our lives? Search, Alexa and TikTok, I can see it.
In the future obviously vision systems for autonomous cars etc..
But I'm really wary about the heavily decreasing marginal returns after that.
It will surely change the world, but I think in specific areas. Most of the entire field seems like an optimization on something rather than anything new.
Washing Machines feed up immense amount of labour and toil. Alexa telling me the weather is not.
If the article is trying to make a point about skill development and diversification, I'm totally on board. Bifurcating the roles instead is going to be less effective.
To the value point...my sense has been we are seeing the Webcommerce 1.0 bubble Machine Learning edition. Lots of uses of it, not all of them have value. I am excited for where we will be in 10 or 15 years, but I suspect the difference will be huge. If you put me to a guess, I would say better data handling practices and ethics will likely be the linchpins of value creation vs. using tools for the sake of tools.
Most software isn’t consumer facing but just because you don’t see it doesn’t mean it’s not changing things around you. ML tends to be overhyped but your assessment is too pessimistic.
However, finding defects in aluminum parts that involves using computer vision, would absolutely be a ML solution.
Everyone is buzzing about the latter, and few even realize what is the former.
You need architecture, you need backends, you need a front end, you need product design...all with data.
Why are computer scientists computer scientists not engineers? Why is computer science about the code side? Why did computer engineering end up being more on the hardware end of the spectrum?
Words, especially newly coined terms are pointers to meaning. That meaning is socially mediated, it is not inherent.
You're saying this (adn I think the author is too) because there is a need for this group of people to look beyond titles to skillsets, and the existing titles carry linguistic baggage of the difference between science and engineering that has existed for decades.