ML is ill-defined. But take most textbooks on ML, and you will find regularized linear regression, decision trees, cross-validation and bootstrapping all in there.

In my view, the main difference between ML and plain statistics is that with the latter, you come up with the appropriate model apriori, and then make sure the data satisfies the assumptions so that you can draw conclusions from the in-sample fit of the model. You control for overfitting by choosing the simplest model that is reasonable - often univariate linear regression.

Whereas with ML, you let the data dictate how complex a model you should use. You choose the appropriate model complexity using techniques like cross-validation, and verify the effectiveness of your model empirically.

ML is often used interchangeably with ANNs which I think is a mistake. Take structured data problems on Kaggle and you would very rarely see ANNs as a major predictor in the winning models.

Not necessarily. Cross-validation can give you a valid estimate of out-of-sample performance even if you have more dimensions than samples, and even if some of the features you have are (sporadically) perfectly correlated with the target variable. See https://stats.stackexchange.com/questions/295626/does-cross-...