Right now our product has accumulated a lot of technical debt on the data validation side because data scientists designed the test code in a way that dramatically slows the development process.
Many "data scientists" (not all, but many) have little to no ability to do anything other than apply "recipes" of algorithms or classification methods or logistic regressions, etc. Asking them to develop a "novel" method would be fruitless. Asking them to clean and scrub the source data set is like telling an amateur pie-baker the store was out of pie crusts, you'll have to make your own from scratch -- it's not going to happen, they just don't have that skill, the instructions on the box don't account for that possibility. As soon as the task diverges from the simple step 1, step 2, step 3 that they were originally taught, you realize they have very little ability to adapt. YMMV of course.
This is because they rarely hire people with scientific thinking ability. They just hire people who can code and program from set recipes. Once you hire such people you can not expect them to do non-recipe work. If you don't want recipe work, don't hire people will recipe skills. Do not have job interviews that select for recipe people. But, that is exactly what most companies do.
I think you've been working with conmen/conwomen. I've never seen a data science project that doesn't involve data cleaning or wrangling of some sort.
I meant a data science project in terms of a project completed by data scientists. In my experience, all data scientists are accustomed to doing extensive cleaning etc.
In all fairness, it’s basically impossible for a new grad to have those skills. 4 years of a bachelors in any field isn’t enough to cover such a wide area. Even for people with graduate degrees it’s a stretch.
If your four year degree didn't give you the ability to learn and expand your knowledge on your own, its a colossal waste of your time and money.
It's very easy for an average engineer (like me) to start using ML using these tools, but a lot harder to explain how it works, or exactly which type of models to use.
In my mind a DS would be really useful to just point us in the right direction and check work. Like a super specialist QA...