I believe where I disagree with you is the basically "understood" part. I don't think ChatGPT and the current models understand anything at all. Not in a way thats not fundamentally flawed and would require a complete new approach. I am hopeful that it can evolve into an incredible knowledge indexing tool but I am very hard pressed to believe it can do anything other than that at the moment.
ATM, im not sure if intentional or because it just can't do it, it just gobbles it all and spits out the mashup without knowing from where things got there, so it cant give me references.
I still think it can be very useful as it can point me to the general direction where I can copy and google stuff about its responses on subjects I know nothing about, though im not sure how much better that would be than googling it directly.
The way I see it, it still a very good simulation, just like a GCI lighting before ray tracing, very good at looking convincing but nothing underneath is actually reproducing the real thing.