Machine learning is about approximate reasoning but with no real guarantees of the approximation. Even if the approximation is usually very good, being occasionally bad can be "deeply problematic" when we don't have control over exactly how exactly the bad recommendations are created.
I think the biggest takeaways are...
1) Machine learning in large software projects is complex because decisions made by one algorithm can influence the data of other algorithms, creating massive biasing.
2) Simple cross-validation / hold back methodology is limited as we expand what we want machine learning to be able to handle. Reason: big data is too big and "correctness" is difficult to evaluate for things like Q/A systems.
Plus read the article/presentation. It goes into the issue with great depth and clarity.