But you didn’t ask it to pitch a new book. At one point, Pattern Recognition was actually his next book, and there’s probably a lot of text out there discussing it as such.
I was playing around with GPT-4 making it take the questions for an Economics exam the other day and it was missing a lot of context that a human would get or assume. It was still much better at economics than me, but was going down "wrong" paths that a human wouldn't go down.
If you told it to explain the the question and list assumptions you could work out where it was going off track, but it lacked the ability to step back and recognise it had taken a nonsense path.
Errr. Yes! We're at the stage where you should be amazed how capable these things are considering their fairly brute force nature. Being surprised at the limitations is a peculiarly - dare I say? - human response?
Usually the workflow involves:
1. Write a naive prompt
2. Gain some insight into AI's behaviour
3. Improve prompt and repeat 1.
Of course they lack insight. Of course there's no "meta-reasoning". What is astonishing is how often a glorified Markov Chain gives the impression that there is both.
Shows how this tech is such a double edged sword for user value, expectations and assumptions.
Even here there are limitations, which was my intent with mentioning the Econ exam.
Aside: It's also possible to get different answers by making the LLM simulate being different people (a student, or a professor).
It is very easy to slip into thinking about ChatGPT in terms of actually thinking, but then again our understanding of the human mind has led to theories that conscious decision may well be an illusion as well (if brain scans showing that the decision is made before the conscious mind is aware are anything to go by).
At the moment though, these are just tools, with varying degrees of usefulness.
We get so many context clues that aren't present in current LLM text interactions.