But do you ever get good enough to make a contribution that's worthwhile? And also is your knowledge flawed because the knowledge of the data used to train the model is flawed?

Whilst I do see your point and I do see the value for prototyping, I don't quite agree that you can learn very much from it. Not more than the many basic intro to..... Articles can teach.