I’d like to remind people that these ppl have no more knowledge about AGI as anyone else on this planet since there is no knowledge yet and everything they say about this topic is as relevant as something every other random person can say.
Dario Amodei (Anthropic CEO, builder of Claude 3 Opus): "My chance that something goes, you know, really catastrophically wrong on the scale of human civilization, might be between 10 - 25%"
Consider this scenario:
- Ask ChatGPT to produce some output meeting certain criteria
- ChatGPT responds, but the output does not meet the criteria in one or more obvious ways
- Point this out and ask for a corrected version
- ChatGPT says "I apologize, here's the corrected version:" and spits out a nearly identical answer with the same obvious mistake.
- Repeat steps 3 and 4 ad nauseam.
Whatever sort of intelligence it may have, I would categorize it as not "general" enough to fall under AGI. Being able to answer AP test questions cannot be a sufficient measure of generality if a model can do it with flying colors but still fail at far more basic tasks.
You can ask ChatGPT what it means to be asked to correct a mistake, and it will give you a perfectly thorough and eloquent answer. But it is often unable to apply this concept to its own behavior. If asked, it can explain back to you exactly what correction you want it to make. You can even make it pledge to correct the mistake in exactly the manner that it just described. And then it will completely fail to do it. It reminds me of that "repeat after me" meme from Friends [1].
This makes me lean in the direction of "stochastic parrot" when I think about what LLMs are. As impressive as it is, ChatGPT demonstrably lacks a sense of self. It talks as if it understands that it's an agent in control of its behavior, but then fails to control or even recognize its own actions.
[1] https://en.meming.world/images/en/c/c2/Friends-_Repeat_After...
ChatGPT was trained on hundreds of terabytes of information. I would hope that with all that effort it could answer AP test questions. That doesn't make it intelligent.
That it's able to spit out essays makes it something, intelligence is too loosely defined to describe it for everybody, so we need to develop new words and meanings and new language for what it does. It doesn't think, but it does dot products to tensors to get create its output.
It is interesting to consider AIs that way given that almost every AI will fail catastrophically when confronted with an area of expertise that is outside of their training corpus, although I guess Corvid tool use isn't exactly out of the animals' range of expertise.
All AI, even narrow ones, necessarily have the ability to interpret novel questions in some fashion — how novel and how well they interpret being the "G" and the "I" in AGI.
This is especially obvious since opinion differ so much about the effects of AGI on humanity and since you cannot prove them right or wrong, every opinion is equally realistic as each other.
Like I could say that every AGI that becomes self aware will kill itself instantly because of boringness, and you cannot prove me wrong.
We just don’t know yet.
Feeling uncomfortable with this, will not change this sad reality.
I am indeed uncomfortable with someone saying that we can't worry about a gas leak until after someone strikes a match and notices an explosion or the lack thereof. If the sad reality is that we can't detect the gas leak any other way, I would argue that the precautionary principle suggests we ban the striking of matches until such sensors have been developed. But here we are in a room of people grinding up saltpetre with their mortars and pestles.
I have proven to you logically that they cannot have more knowledge then anyone else, but you continue to present their authority as knowledge.
Pls prove to me that they know more about the effects of AGI on humanity or that they have some magical knowledge we normal ppl are unable to comprehend.
If you cannot do this, then why are you bringing up these ppl?
I'm only asking for a standard that falls slightly short of "already having built Schrodinger's doomsday weapon and observed whether humanity dies as a result". You can set the bar wherever else you like, please!
Considering how rapidly the technology is developing, and considering how clearly a threat a superior intelligence will be to the human race, it is absolutely the time to have this discussion NOW
You've asserted it, not proven it.
Given their entire research field is "how to make the thing", it's a fair bet that they know more than me — even if what they know is, to paraphrase Thomas Edison, "1000 ways to not make an AGI", that's still more than me, and I follow some of the research and read some of the papers.
Correct, and this (your counterpart's unrealized because it is the normal, proper way to think error) is what worries me... I think once again (as with affordable vehicles powered by fossil fuels) humans may be once again walking blindly and confidently into a situation where we've over invested in certain competencies to the detriment of others, increasing the already dangerous imbalance that exists between power and wisdom.
I don't know to what extent OpenAI and their compatriots are actually trying to bring forth artificial life (or at least, consciousness) forward, versus how much they're just banking on how cool AI is as a term in order to funnel even more money to themselves chasing the pipe dream of building it, and at this point, I don't care. Even the products they have made do not hold a candle to the things they claim to be trying to make, but they're more than happy to talk them up like there's a chance. And I suppose there is a chance, but I really struggle to see ChatGPT turning into skynet.
My response to a very similar statement is here:
https://news.ycombinator.com/item?id=39612641
> That's just... not how it's going to be done.
I agree. I think there are some critical ingredients missing. Obviously the weights need to be able to update to new data in a semi-online fashion, for example. But I think there are less ingredients missing than there were a decade ago, by a much larger factor than I expected at the time, and a decade from now, the number of missing ingredients might be zero.
My uncertainty on this is not because I think GPT is more than it seems,but because it's unclear how much of the amazing cognitive capability that humans have is, under the hood, much simpler than it seems.
I really want humanity to exist even ten decades from now; I want my kids to have grandkids. So I care about this even if I don't expect AGI from a 2025 Q1 product launch. And I don't think it's too early to worry about it, just like I don't think 1980 was too early to worry about climate change.
This is a vivid bit of rhetoric to underscore this point, but if you think about it for any length of time it starts to fall apart really, really quickly. The Wright brothers and the dozens if not hundreds of inventors forgotten who came before them drew upon the physics of what they observed in heavier than air flight to create the winged shape we know today that reliably causes lift, and then set about constructing it. That's not what OpenAI is doing. We still do not have a very solid understanding of where our own intelligence emerges from, apart from having particularly large brains relative to our body's size. So, to borrow your metaphor, it is indeed like a bird saying that there's no evidence to say that heavier than air flight is possible, because the bird lives in a world without atmosphere upon which to glide.
But when you look at, say, Sora's videos... How certain are you that there's no path between that and the human visual cortex? How certain are you that they aren't solving at least some of the same problems in structurally analogous ways? Given that nature built a visual cortex (more than once!) by applying survival pressure and turning a crank, just how hard can it be? When we apply a bunch of optimization pressure and billions of dollars turning cranks and something eerily similar pops out, that tells me that there's just less magic to our own brain than we thought. Like when spent centuries wondering if our solar system is unique, and then we finally put Kepler up there and it turns out that planets are everywhere you look.
Sure we do: we're made of atoms.
To reject that this necessarily shows that AGI is possible, you would have to demonstrate human intelligence is tied to an immaterial soul granted by a supernatural being.
Some people do in fact believe this. I think that if we can't build AGI even with a perfect copy of a brain, that would instead be a surprising proof the existence of souls.
> This is not intelligence of any sort, period, paragraph.
Let's run with that. Just using your definition of intelligence: given AI can already beat us in Chess, in Go, in Mathematical Olympiad puzzles, in protein folding predictions, in poker, at real world stock market analysis, in the game of Diplomacy, … — does it matter that they're not what you call intelligent?
Do submarines swim?
> bring forth artificial life (or at least, consciousness) forward
Why does it matter if it's "life" or "consciousness"?
“Deep Blue was intelligent the way your programmable alarm clock is intelligent. Not that losing to a $10 million alarm clock made me feel any better.” ― Garry Kasparov
> To reject that this necessarily shows that AGI is possible...
Incorrect. For something to be possible in an absolute sense, it has to be possible in both the physical and the metaphysical realm, and a lack of knowledge of blockers in the metaphysical does not mean that there are none (in a comprehensive sense), it only means it appears that way, due to (at least) our cultural beliefs ("truths") and conditioning.
> ...you would have to demonstrate human intelligence is tied to an immaterial soul granted by a supernatural being.
Could demonstrating that intelligence rests upon a virtual reality generation machine (that cloaks itself from being realized as such) or something else not also work, at least maybe?
Plenty of people have interesting or valuable opinions and insights on AGI, programming LLMs doesn't have any impact into that though.
It'd be like expecting the developers of friendster to have a good idea of what the impact of social media was going to be in 2020.
AI isn't AGI.
The hour before the Kitty Hawk flight, the two most informed people in the world on the topic of heavier than air flight were a pair of bicycle retailers and manufacturers named Wilbur and Orville.
These people sure don't spot everything, but they're still way closer than the average human.
Actually, this is quite false. There were many people working on flight with massive credentials and massive funding. That the Wright brothers beat them is part of what makes the story of the Wright brothers so interesting.
That society didn't know this at the time is indeed relevant to AI development, and the analogy would be that the next big thing might come out of some minor research team that most people dismiss instead of the biggest and most well known groups… a bit like how Google's LaMDA was making headlines before ChatGPT surprised everyone.
It's certainly still possible today that some random individual has a crucial insight that gives them an edge over the big names, yet even those big names know far more than me.
Which is rather annoying, because I'd be interested in taking the opportunity of having been laid off to switch from iOS to AI. (If anyone reading this is hiring people in Berlin, contact details on my website. Plus side of hiring me: I'm relatively cheap in that field, and enthusiastic about the possibilities).
Truly no offense meant, as I was deeply into the EA movement myself, and still consider myself one (in the original "donate money effectively" sense), but the movement has now morphed into a death cult obsessed by things like:
* OMG we're all going to die any time now (repeated every year since circa 2018)
* What is your pDoom? What are your timelines? (aka: what is your totally made up number that makes you feel like you're doing something rational/scientific)
I'm deep in the weeds w/ LLMs, e.g. I probably finetune an average of 1 model a day, and working with bleeding edge models... and AI safety just sounds so silly. Wanting to take drastic measures today to prevent an upcoming apocalypse makes as much sense as taking the same drastic measures when gradient descent was invented.
It's sadly twisted how mentioning that -- the majority of leaders doing the cutting edge research on AGI think it has a significant chance that it kills humanity -- is considered being part of a "cult" movement.
Your analogy is the same as early Intel engineers completely unaware that those chips would bring on the ramifications of social media. "In the weeds" and yet unable to foresee the trajectory and wider impact. Same with the physics that led to nuclear weapons.
> Wanting to take drastic measures today to prevent an upcoming apocalypse makes as much sense as taking the same drastic measures when [nuclear fission] was invented.
Exactly! As they should be. (for both Intel engineers developing chips, and physicists developing nuclear research)
There were a billion more potential dangers from those technologies that never materialized, and never will.
I'm glad we didn't stop them in their track because a poll of 10 leaders in the field thought they were too dangerous and progress should stop. (note that no one is against regulating dangerous uses of AI, e.g. autonomous weapons, chemical warfare; the problem is regulating AI research and development in the first place)