I've seen two companies in the last year start twisting themselves into circles to worry about 'productionizing' a data science project.
The intent is right to focus on end use case and business value, but it feels like the communication is exclusively in terms of overbuilding systems.
The general factors I've observed during scale-up are common across all projects.
1. Very few people know how the underlying model works, or why it works - many of the people you would expect to know do not.
2. The business value of marginal improvements to the model or extending to new use cases in the business is much smaller than you would hope. Or more advanced methods introduce unfortunate tradeoffs which make them impractical.
3. Whenever you add new modelers to the mix, they want to use radically different technology - reducing the effectiveness of platform efforts. Many of these divergent efforts do not produce net gains and instead come down to the old use X instead of Y type debates.
4. Few modelers want to touch anyone else's code. Most believe internal tools will inherently be garbage (which they often are)
5. Platform efforts tend to spiral into cost pits.
6. Investment in the product area is largely dependent on leadership buy-in for ML related efforts, usually with a big initial thrust - moderate gains, then slow wind down.
This reminds me of systems engineering and DevOps pre-cloud, and tells me that companies are going to want to outsource this tooling as fast as possible. I'd also expect that there will be a good market for directly offering specific customizable platforms for things like ASR, Recommendations, Search, Computer Vision, and others - but the challenge in 1&3 will make this a tough sell.
https://www.coursera.org/specializations/machine-learning-en...
Even the free videos are valuable if you do not have experience deploying real world systems.
https://cloud.google.com/architecture/mlops-continuous-deliv...
It covers differing levels of maturity, with the Maturity Level 0 being what the op was mentioning. Our team is trying to dig out of Level 1 to Level 2.
For example, it can be as simple as wondering how to organize, collaborate on, and versionize lots of Jupyter notebooks! Or how to keep track of and compare different models/hyperparameters/datasets, not only individually but also across a team. Both are not discussed in typical data science courses.