It can be argued that some algorithms like Random Forests don't impose a generalization penalty as you increase the number of parameters (forests).
Tradeoffs, tradeoffs everywhere. It's almost like traditional mathematical statistics has something to offer them fancy machine learners. (Breiman was a professor of statistics, after all... ahead of his time, but no less a statistician.)