It may be possible to create a machine that flirts with actual intelligence, but this is simply not it. There's not even room for doubt about this.
You seem convinced that ChatGPT will never have "actual intelligence" — care to make a prediction about something that LLMs will never accomplish? We know they can write code, write essays, generate artwork, and play chess. What's a task that requires "actual intelligence"? Parenting a child? Running for President? Making a steak sandwich?
So... creating a new language, maybe?
My bet would be that in ten years GPT would get pretty good at it, but eh, I dunno.
But it's not 100%.
Are those two responses enough to say it's invented a language? Probably not. But it's already farther along than I would have gotten if you asked me to do such a thing.
If I kept prompting it for hours, I bet it would start to contradict itself and lose track of the rules it had already established, but so would I. If I were actually inventing a language, I'd take months and keep extensive cross-referenced documentation on grammar and syntax and vocabulary. We don't _let_ ChatGPT do that, it has no mechanism for persisting its ideas like that. But like... neither would I if you took away my notebook and the parts of my brain that persist long-term memory.
I guess my interpretation here - I see differences in _capabilities_, but not differences in actual _intelligence_.
How do you know?