One of the things that always gives me a little hit of hype is when I tell it to do something ridiculous and it just dutifully starts spitting out the result without complaining or questioning lol
I think LLMs inference training should include teaching it to ask questions back before starting full-fledged generation. You know, make it a little more Socratic.
Right now the approach is: ChatGPT starts answering and, if it's going the wrong way, you either hit "Stop Generating" or just wait for it to finish then figure yourself how to improve the prompt. LLMs should be also trained in ranking the prompt and determining what questions would make the prompt statistically stronger to generate. I bet it would result in savings running it too. In fact, one can try this out by configuring a system prompt that tells the model to ask questions before getting started with an answer.
It does feel like it should be more commonplace or even just the basis of how these things function because sometimes it just goes off on something completely unrelated because my prompt was bad haha
Edit: It almost seems like you are anthropomorphizing it. It is just a program doing what it's supposed to be doing: to predict the next token based on its weights. Nothing more, nothing less. It does give the illusion of intelligence. Pretty soon, though, we may not be able to tell the difference.
Every time I see a comment along these lines it gives me pause: there is a built-in assumption that each of us is somehow doing something more than this.
I'm not convinced.
I've heard people refer to some of our instinctive behaviors as due to "our lizard brain", suggesting that our brains are hierarchical, or comprised of a series of evolutionary steps, a more evolved order of brain wrapping the more primitive.
I increasingly suspect that ChatGPT has more or less nailed one of those layers.
It's the same with speech, sometimes if I'm looking for something nice to say to someone but nothing is top of mind I'll just seed myself by starting a sentence "You are great honestly I was just thinking about how..." and then just hope my brain finishes it for me. Usually it does but sometimes it ends in me explaining why I stopped haha
One may argue what passing the Turing test means, but at least by definition it is mimicking human intelligence in some way.