It covers a variety of times people might use it. Sometimes it's genuine, other times it's flattery. Some people use it all the time (and in the episode they talk about calling someone on it). One guest says it's a bridge, to go from the question asked to the question you want to answer instead.
I don't see it in this transcript, but I also thought I remembered hearing/reading that it can be a sign the speaker doesn't know the answer offhand, and needs to consider the question to formulate their answer. I think I'd classify that as a genuine usage: the question is something the speaker hasn't considered before, and thinks is worthy of consideration.
[1] https://freakonomics.com/podcast/thats-a-great-question-rebr...
I remember being advised to do this ~20 years ago when I was going to be answering questions from a group of people. I was told that it's good practice to say something like "that's a great question" every time someone asks anything, as a form of social lubrication, to encourage others to ask questions. I can't say whether it works, and it was advice for a spoken context rather than written, but I don't know how to finish this sentence.
When I go to research lectures, I sometimes hear that in response to audience questions, although not especially consistently. Some speakers do this more than others, I don't think anyone does it all the time.
The next question (which is a great one, from what I understand) is: Why do LLMs use these phrases so much if humans rarely use them in written form? Maybe a fair portion of training data comes from lecture transcripts, where such responses are common when responding to direct questions? And/or system prompts are just instructed to be like that?
As far as I understand, it's due to RLHF. The reviewers the AI companies use don't necessarily know what kind of question is a good one, so when the LLM answers "That's a good question!", they tend to rate the answer higher because they like being flattered. Proxy models that are themselves trained on RLHF inherit this pattern. Similar effects contribute to sycophancy.[1]
That's a great question! => I can tell you understood what I explained and used that understanding to reach the next step of reasoning just like I did.
Validating their ego and effort and social position can fulfill their social desires regardless of an answer that dismisses, say, buzzword soup as both inappropriate to the current context and incoherent to people who know what those words mean.
And then, somehow, he kept turning the dumb question into a deep and insightful one.
So I've adopted that technique but if I say "that's a great question" think for a while and come up with nothing, it's probably a moronic question. I'm not perfect.