I think the big innovation here isn't in its grasp of language, but in the way it's able to access data. Comparing ChatGPT against ELIZA or GPT-2 isn't the right comparison, you should compare it to something like SQL or Google.

Against that comparison, it stands out as very innovative, but it also seems like it has big potential to improve. If you could perfectly index a large database and carry out reasoning tasks, this is enough to replace a lot of jobs. ChatGPT is in this regard just on the boundary between a toy example and useful in prod.

It's also worth noting that, up until now, people who have been saying this ("I think this is about where it peaks, it'll never do X") have always been proven wrong by the passage of time. So I think we should assign lower credibility to the skeptics, insofar as their past predictions were incorrect.