I'd say this. I've become convinced that the bottleneck in machine learning is the willingness of organizations to make training sets that are representative of their business problems.
If you are downloading data sets from Kaggle you are going to be "always a bridesmaid and never a bride."
So I am convinced that the royal road to automation is "break the task down into little bits, build something that people can control like a puppet to do the job, use the inputs to the puppet to train models that partially automate the task -- someday you need only 5-10% of the inputs that you started with"
Thinking about it that way I am very happy to be mastering (and I do mean mastering) user interfaces with React and similar tools.
I had a great opportunity in the past where I could have gotten management to take the initiative, formulate the problem correctly and build the right kind of training set but I didn't have the courage and I made the same mistake that almost everybody else makes to depend on generic algorithms and training sets -- something that we all knew would fail and it did.
I won't be fooled again.
My concerns aren't so much about organisational problems but more about how to leverage existing experience when switching to a new problem domain.
I am not doing any ‘data science’ at all in my current side projects which all revolve around art, images, manufacturing, etc. because that is my obsession now. But I guess I should take that back because I am building image selection pipelines again and these are building up training sets that one day will go into a model of some kind. (But that competes with colonizing my office walls, making digital twins for my supergraphics, making a persistence of vision display for my car, and devising a lightweight mirror I can install to perform with a pepper’s ghost, …)
I think ‘data scientists’ themselves have more problems than the organizations they work in. Many of them think they are too good to have to think about reliable builds and the many details you have to attend to go from ‘I make a report’ to ‘I make a system that makes reports’. Some of them just won’t take orders when it comes to standardizing the way they do things.
In web dev knowing how to use version control, issue tracking, etc. are table stakes. Discipline in software dev is important, just as important as knowing how to map click event coordinates to objects in a web UI, what to do about margin collapsing, where to keep state in a React app and not go insane,…
That's a really insightful view I hadn't considered but I think I agree with you and will likely spend more time with React and Vue now to have that card in my pocket for the future.