I can't agree with you here. While GPT-2 isn't good at filling in particular details (like the language someone from Boston should speak) it is astonishingly good at recognizing the _kind_ of answer that should be produced. The fact that it usually answers with a language here is a reflection of what it understands. And it behaves similarly for a whole range of different tasks. If you write a sentence that should obviously end with the name of a person it will give you the name of a person. If you write something that should be completed by the name of an event it will give you the name of an event. If you write text that should obviously be finished by the description of a person's appearence it will give you a description of a person's appearence. GPT-2's grasp of the appropriate context of a wide range of situations is nothing short of incredible. Dismissing this as "Eliza on steroids" is just short-sighted.