As Yann Lecun said, “in high dimension, there is no such thing as interpolation.
In high dimension, everything is extrapolation.”
It’s trivial to come up with a prompt that doesn’t exist in the dataset. To generalize, the model cannot memorize.
It’s trivial to come up with a prompt that doesn’t exist in the dataset. To generalize, the model cannot memorize.
No comments yet.