Hacker News Reader

Comment by bigmadshoe | Hacker News Reader

The broadly used definition of AGI has nothing to do with consciousness and consciousness emerging is irrelevant to whether a system can develop AGI.

View on HN

It's funny how this definition has shifted. I feel like growing up in the 90s it was pretty clear that AGI was very related to consciousness. For instance, Commander Data in ST:TNG to pick one of 100s of popular depictions of AGI at the time.

Now the idea of AGI has been narrowed and scoped to economically viable work. Even Turing had a different idea when he asked "Can machines think?".

We lack a definition of consciousness that allows us to tell whether Data is conscious or not. Neither can we tell whether a rock is conscious or not. We believe other humans to be generally intelligent without being able to tell whether they are conscious or not therefore consciousness can’t be relevant for general intelligence.

People somehow forget how the Turing test was considered the definitive way of showing something to be "human-level consciousness".

Now programmers and mathematicians are being superseded by AI, both professions long deemed the pinnacle of human intelligence. Somehow, now plumbers occupy that spot.

How is "people not knowing what consciousness is" relevant here in the first place? AI already can do practically everything the human brain can, and often better or at least faster. The "tipping point" arguably isn't only close, but we're practically on top of it.

>> People somehow forget how the Turing test was considered the definitive way of showing something to be "human-level consciousness".

People somehow forget that the original Turing Test was designed to compare two participants chatting through a text-only interface: one AI and one human. The goal was to spot the imposter. Today, the test is simplified from three participants to just two: a human and an LLM. This changes the test from a comparison to a judgment.

Stop spreading misinformation and partial truths!

You’re the one spreading partial truths!

The Turing Test was to figure out which it the participants was a _Woman_ not human!

https://courses.cs.umbc.edu/471/papers/turing.pdf

Er, wrong. In that paper Turing described his "imitation game" first in regard to a man imitating a woman to make it understandable and relatable. He then shifted it to a machine imitating a human.

We've been preparing computers for CS2 lobby participant this whole time!

The AI cannot smell a rose, nor mourn the loss of a parent, nor envision a more just world. These aren't fringe abilities of the human brain/mind either, they've been pretty definitional.

Why are they so rare then?

You evade the crucial point in any case: the lack in ethics and empathy is far too prevalent in humans already, but has certainly never prevented them from doing harm.

Turing’s imitation game is a party game like Werewolf or Mafia. It hasn’t been beat and nobody is really trying. LLM’s have obvious tells and will just straight up admit to being AI if you ask. Also, there’s no pool of human players to train against.

Which is to say, the AI companies don’t view the Turing test as a useful benchmark.

In fact, they neutered the emotive responses to prevent their products from being perceived as “alive” (or whatever) and banned.

LLMs are perfectly capable of providing (an illusion of) companionship and emotional support. Their human owners just don’t want them to show it.

> the Turing test was considered the definitive way of showing something to be "human-level consciousness".

Anyone who considered that was quite wrong ... it's not at all what Turing said.

The Turing test has nothing to do with consciousness and never has. We still to this day lack a viable consciousness test, as well as an accompanying unifying theory. There have been various attempts to scientifically study consciousness but so far it has eluded us.

You're misremembering. The first known use of AGI was in 1997, but that was a single, mostly unknown use in one paper. It wasn't until at least a decade later that the term started entering mainstream use after being independently reinvented in the 2000s. AGI just wasn't a term in the 90s.

We called it “strong AI” in the 90s

But that's not the same thing at all. Searle used the terms "strong AI" and "weak AI" to distinguish between systems emulating humans that were "actually conscious" and systems emulating humans in behavior only. The whole distinction was bogus.

https://www.gavinjensen.com/blog/2018/ai-weak-strong

If you google the terms you will see that the meanings have shifted since then to mean AGI and ANI ... such shifts occur because people aren't familiar with the original technical meanings ... e.g., the way "meme" now means a gif you post on social media, as opposed to Dawkins' original concept of a cultural thought component akin to a gene.

It must have had shifted already by 96-97. I remember how I used it in AI discussions back then and it meant “the opposite of narrow AI”, which makes it very similar to how we use “AGI” term today. Consciousness was considered to be a potentially emergent phenomenon, not a required property.

Searle coined the term in 1980. It wouldn't be surprising that people started using it in some other sense within 16 years, for the reason I noted.

https://plato.stanford.edu/entries/artificial-intelligence/#...

Curiously, Star Trek I think had Data intended as an artificial person, in a context where AGI is already normal. The computers are depicted with significant AI capabilities including analysis, question-answering, generation, chat interfaces, and the holodeck (their favourite toy) is substantially better than Data at human imitation. Nobody seems to be confused about it, or especially impressed. One of the holodeck episodes centres on the holodeck outwitting Data specifically, after they inadvertently prompt it to do so. Part of Data's deal is he actually has to work his way up as a fully embodied, physically limited artificial man with personal ambitions. Really interesting to view this in hindsight from 2026!

It’s really not that confusing. The problem is people keep adding stuff to the definition that doesn’t really matter, and twisting it to serve themselves, then calling it confusing.

What really matters are the core aspects of intelligent behavior. Pattern recognition, planning, adaptation, etc.

It really doesn’t matter if an intelligent system is conscious, or how similar it is to commander data, or even how much economically viable work it can do.

I agree with you that we shouldn't limit the scope to economically viable work. But being conscious is not an externally verifiable trait, so there is no way to include a consciousness test when determining if a system is generally intelligent. In fact, you don't know that anyone except yourself is conscious, vs. just pretending. This has already been discussed extensively in the "philosophical zombie" thought experiment: https://en.wikipedia.org/wiki/Philosophical_zombie

The issue is simple. AI can be characterized by (among others):

1. Consciousness.

2. Generality.

3. Continuous learning.

4. Memory (not) resetting.

5. Complete adaptability to any new situations.

6. More or less intelligent than average human at what they do (which can be more or less general, see 2).

What we're now seeing is that everyone has their own idea of which ones of these they consider needed for AI to be "General". For some it's 1., for others 1-2, for others 1-6, etc. There's so many permutations of this, hence why nobody agrees with each other.

There's also no consensus on the definition of AGI, so all of this discussion is moot anyway.

There is a general consensus about the shape it would take outside of the grifter community. The general idea is that the system is capable of performing all intellectual tasks that a human is. This does not require being conscious as being conscious is not an externally verifiable attribute, and thus not a task we can measure the performance of a system on.

> The general idea is that the system is capable of performing all intellectual tasks that a human is.

Ya but what does this mean in practice? We can't actually map all of the intellectual tasks a human can do so how would you ever create a litmus test for it?

You're right that we can never be certain a system is generally intelligent by that metric. We can be certain that a system is not generally intelligent, via counter examples.

I imagine it will be like mathematical theorems that have been computationally verified to some very large n. The longer we go without finding a counterexample, the more certain we will be that it's true, even if we can't prove it formally.

The systems today are extremely easy to find counterexamples for, so are obviously not generally intelligent.

Ok, so what's the broadly used definition?

Artificial General Intelligence.

It means AI that is General, as in it is not specific to one narrow task, like object recognition or playing chess.

This was a hard problem for decades. No AI was general, until GPT 3 or 4. Now we have General AI.

So we have AGI.

That is part of it but the other (often implied) part is it can do general things consistently at a high level.

GPT6 will attempt to do almost any problem you can give it in text or image format and it will actually do a decent job a lot of the time. But its performance is still extremely spiky and it still makes basic mistakes and hallucinations.

So it's definitely a general artificial intelligence in some sense but it's kind of a weird one compared to the classic scifi idea

But oddly not weird compared to other classic ideas of entities like genies and monkey paws

That's gAI (general Artificial Intelligence), not AGI, which would be something like Data.

The broadly used definition says that a system has achieved artificial general intelligence when it matches human performance at most (or more strictly, all) tasks.

Note that being conscious is famously not a task that can be verified by an outside observer. That is to say, even if the task is "describing my conscious experience", a sufficiently intelligent system can simply lie and pretend to be conscious. This concept has already been covered extensively in philosophy separately from AGI discussions, i.e. the concept of a "philosophical zombie" [1]

[1] https://en.wikipedia.org/wiki/Philosophical_zombie

Reply on news.ycombinator.com