I don't think LLMs are anything like an AGI. I think that a lot of people are experiencing very strong pareidolia. They're falling for an illusion.
>I think we're not even close to having one.
Why is this the default assumption until proven otherwise? The issue is determining what decisions bring the most expected utility. The default assumption should be downstream from the utility analysis. We don't know whether we're close or far from AGI, but it sure seems like the timelines are a lot shorter than any of us originally thought. We need to prepare society for AGI (or prevent its creation) well before it is realized. Banking on an alarm that will alert us to its imminent creation is looking more and more like wishful thinking. So the course of action that maximizes utility is to assume we're close and prepare for it.
How true that is depends on how you define "better". But we've been making lots of tools over thousands of years that can perform tasks better than people. You need more than that to declare a thing "intelligent" in the sense the average person thinks of it.
> To place your bet that we will never reach AGI is bone-headed.
Never is a very long time, but I think it's a solid bet that it won't happen within the next few decades, at least.
> Why is this the default assumption until proven otherwise?
In the absence of solid evidence for the existence of a thing, assuming its nonexistence is the logical position.
> We need to prepare society for AGI (or prevent its creation) well before it is realized.
We agree on this. But that's not what's happening. What's happening is some people are scaring the hell out of other over an imminent that there is no evidence exists.
We have lots of time to consider these things carefully and to react with calm prudence.
> So the course of action that maximizes utility is to assume we're close and prepare for it.
Not in the way it's happening. The way its happening will, in my opinion, actually increase the harm that AI is threatening to cause, and will distract from the actual risks that AI as it exists right now poses.
I disagree very strongly both with this statement, and the idea that there is no evidence we can create AGI. Absence of evidence is not the same thing as evidence of absence.
It would make sense to be skeptical if people were asserting the existence of something for which we have very strong prior beliefs that this thing does not exist. To say that there's an enormous green monster hiding on the back side of Jupiter, for example. Until we've looked at the back of Jupiter, we have no idea what's there, but why would it be enormous, and green? To say that there's life on Jupiter, that's a tougher one -- we could say that we are not sure, but based on our understanding of Jupiter's atmosphere, we think it's unlikely. If we did not know what Jupiter was made of, it would be perfectly reasonable to assume that it might host some sort of life. It would be wrong to assume that because we have no evidence of life there, that there is no life. If we had carefully inspected thousands of planets and never found life, it would be much more reasonable to assume there is no life on Jupiter. This is the whole idea of Bayesian reasoning.
But with AGI, it's not like we're searching the universe to see if it exists. There are a ton of smart people actively TRYING TO CREATE IT. The evidence that AGI will happen is what people have already accomplished. We've got goalposts on rails, we've been moving them so much. An AI that understands language, and can respond with perfect grammar, has a theory of mind comparable to a 9-year old, is already mind-blowing, sci-fi tech. And OpenAI is claiming that they have not reached the limit of scalability with bigger models yet. The models we see aren't even the smartest ones -- they're the ones that have been neutered by fine-tuning to be less offensive and less likely to give out dangerous info.
To claim that AGI will not happen, you need some very strong evidence that the people actively trying to create it, who believe they can create it, and who have had some massive successes up until this point, will fail. To assume that it won't be created because we haven't seen it yet, doesn't make any sense at all.
This is what's so insidious about this debate, people are completely unable to reason with uncertainty. No, you do not know that we have "lots of time". Maybe we do, maybe we don't. But we do know that acting too late is far more dangerous than acting too early. Thus we should be biased towards implementing safeguards and guardrails now.
>and will distract from the actual risks that AI as it exists right now poses.
Perhaps this is right, and I have plenty of sympathy for this. But the answer is not to pretend there is no x-risk concern at all. The answer is to make sure we're taking seriously all manner of AI risk. Once we all agree we should be having these discussions, it will be much easier to discuss the more immediate risks as well.
I entirely agree, but that's not what's happening. What's happening is that some people are focusing entirely on the most extreme, and extremely unlikely risks and completely ignoring the much more likely risks.
Between the extreme hype levels and the fearmongering from influential people, the pool is well poisoned and having a reasonable discussion about risks is essentially impossible.
> Once we all agree we should be having these discussions, it will be much easier to discuss the more immediate risks as well.
I don't think anyone is thinking we don't need to have these discussions. But the discussions that we need to have aren't the ones we're having.
Obviously you don't which is why i'm asking you to set concrete goals on what quantifies AGI. No point in saying we don't have AGI when i don't know what AGI means to you.