The "inside" of an LLM is a black box in that we don't know how an input arrives to an output. We know some general ideas ("it predicts the next thing"), but not the detailed route to get to the output.
Similarly, we do not know how human minds work. How does a thought arise? How does a new idea arise?
I would not argue that an LLM is identical to a human brain. But I do think there's some interesting discussion to be had on that topic.
It's as though we have two black boxes, and so many folks are willing to say, "One of these is definitely, certainly not like the other black box!"
And I don't see where that confidence comes from.
That comparison, to my silly mind, is astonishing to think about. And I've yet to see why I should (or could) dismiss the notion of similarity.
Or, simply using GPT-4; that's quite astonishing to me. I'm not a stellar coder by any means, but I've quite literally 5x-ed my productivity with GPT-4.
"Well, you must not be working on novel problems." You're right! But is "solving novel problems" a requirement for discussion now?