The root of the problem is that no matter how hard you want to believe, the LLM doesn’t actually know anything.
If you gave an idiot something very intelligent to say and he read it out loud perfectly, people might be very impressed too. That’s GPT.
If you gave an idiot something very intelligent to say and he read it out loud perfectly, people might be very impressed too. That’s GPT.
Could GPT be given some screenshots of a game you want to play and then code it up?
Could you run through a demo of some competitor’s app and have it make something similar but better?
Everyone repeats the retort you gave, yet I’ve yet to see a clear definition of “knowing”.
And when something gives the increasingly-accurate illusion of knowing, I fail to see how it matters (with regard to impact on society and overall utility).
I’m not saying GPT-4 is this amazingly accurate, near perfect model. But if you extend the timeline a bit, it’ll be able to become more and more accurate across a broader range of domains.
Furthermore, how can we prove a human “knows” something?
When I write code, I don’t just focus on solving the problem at hand, I think about things like, like: how will another human interpret this, how maintainable will this be, what are the pitfalls down the line, what are the consequences of this, any side effects, performance implications, costs, etc… things GPT does not know.
And your point about humans lying about knowledge only to be found inexperienced is quite the opposite of an LLM (albeit there is the hallucination problem, but GPT-4 is a massive improvement there):
These models do have “experience” aka their training data. And I would argue with most every one of your examples of things that GPT doesn’t know.
You can ask it about performance implications, side effects, costs. It’s quite good at all that right now even! Imagine the future just a few years out.
There is no “getting better” from this. If you gave a monkey a type writer and it occasionally typed words randomly you wouldn’t say “Wow this is just what it can do now, imagine several years out!”
Comparing GPT-4 to a monkey with a typewriter , and claiming the absolute of “there’s no getting better from this” when we’ve literally seen dramatic progress in just months?
I think you’re missing out on some of the utility this stuff can actually provide .
And it will never do those things, because it’s an LLM and there are limits to what LLMs can do. There is no “getting better”, it will only sound better.
If it’s going to replace programming, the prompts simply cannot be more laborious than writing the damn code yourself in the first place.
LLMs are just the part of a much larger looping system that can do these things you speak of. Be active and seek out stuff. Of course, it’s all illusory, but I’m sorry I think it’s no different with myself.
By the way, it actually gives ok reviews on novel code, so I’m not sure what you mean. At some point nothing is truly novel, even innovation is composing existing “patterns” (at whatever abstraction level).
So thinking that chatGTP could gain understanding is as crazy as the idea that primates could learn to use tools or type words?
If an AI reliably produces good code in response to prompts, it is a better program than most humans. If an AI reliably produces prose that's free of errors, well organized, and summarizes the issues asked for, it is a good writer. Irrespective of what we can say it "knows" or doesn't know.