It feels like to improve the quality of the model, we need to be more selective of training data.
Why do you say that this is a common belief that used to be true, but is now no longer true?
GPT does not attempt to learn the average "text-generating system". It attempts to learn all of them. In superposition.
Just imaging how bad it would be at predicting a scientific paper if it spoke (and reasoned?) like the average person. Such a model would not survive training.
Training GPT does not make it behave more like the average, but instead widens and diversifies its probability distribution over all possible follow-ups.
To make this clear, ask it to take on a persona. It can pretend to be all sorts of people, invented or sufficiently catalogued. I've had fun having it play multiple roles from podcasts I enjoy. Or it can pretend to be a fictional FTP server at Disney where poor authors have stored their unpublished screenplays. You can ask GPT to run dialogues with itself, playing both sides of the argument.
There likely is some level of global persona at play, either through fine tuning or there being a general attractor basin of "helpful assistant" that we're all reinforcing. But there's no reason to believe that this looks anything like consensus or averaging.
So it's weird, but I expect LLMs to give me better answers in narrow well discussed fields than broad but highly argued fields. I guess it's still really important to know how to ask the right question.
It will act “smarter” if the prompt indicates a smart person wrote the text.
Interesting point - I'd never considered that.
IQ is a proxy for intelligence. People often use them interchangeably, but they’re not the same thing. Your statement is more valid for intelligence than IQ.
A person with say math degrees will not take the same test as a person with a highschool diploma.