And the reply also leaves me with a similar impression:
> your training objective can be different at train and test time
But this is not overfitting, this is concept drift, a different and well-defined thing in ML.
> the datapoints you use can be different at train and test time
Both train and test data came from the same population. They are just different incomplete random samples.
I guess what I am getting at - overfitting happens because we know we are training a model on an incomplete representation of the whole. But that representation is not a proxy, as suggested in the article - it is not slightly different to the goal. It's an incomplete piece of the goal.