What do you mean by: large language models cannot generalize beyond the training set?
That’s one of the main reasons they’re so impressive, they _are_ able to do this, to a limited extent.
That’s one of the main reasons they’re so impressive, they _are_ able to do this, to a limited extent.
When researchers dig in, they find that the neural network is learning wrong things. Word matching between answer and question, learning to model the annotator who asked a lot of questions because his Q&A answers are predictable, and stuff like this.
There was a great review of the problem, but I can't find it, so I will have to link to this article [1], which gives an overview of issues with current NLP models.
[1] https://thegradient.pub/frontiers-of-generalization-in-natur...
Don't know exactly what is meant by "generalization" in this context. I'd argue that it's very unclear what the difference between "interpolation" and "extrapolation" even is in a high dimensional and sparsely sampled space.
Would a NN trained to recognize "fur" trained on different kinds of dog fur also activate when it encounters wolf, cat or even bear fur? This seems quite plausible, and seems like a kind of generalization.
I agree, it's very hard to fidn the difference. Especially for GPT or BERT, where the training set is basically the whole internet.
It's a very good question about fur. I would suspect that it would correctly recognise all kinds of dog-like fur and sometimes fail on different furs, like bear furs. But in general NN's are very good at textures, so maybe it will just be good on all kinds of fur. One problem that might arise is that you will show it something fur-like but not really fur, and it will think it's fur.
I agree it's some kind of generalization. Here I don't have enough background to draw the line, perhaps a more theorically oriented person could, but I can't.
I guess the most important judge is that if you devise a benchmark like commonsense Q&A, a neural network fails it. Or how a Tesla will recognise a truck full of red stop signs as a real stop sign, while a "generalizing" thing like a human would definitely know that a core property of a stop sign is that it should be installed near a road. So there is a real problem.