Hacker News Reader

Comment by lsc | Hacker News Reader

exactly... we don't even have a definition of consciousness that is better than Jacobellis v. Ohio - For that matter, I'm not sure we even have it that good. I'm not sure I'd recognize a non-human consciousness when I saw it. I'm not sure anyone would.

I mean, sure, I can totally see the idea that consciousness is maybe some emergent property of complex systems... in which case, sure, we could stumble upon it by accident. That is totally possible; we're creating complex self-replicating systems all over the place.

But as far as intentionally constructing the thing that will come after humanity? we can't do that until we know what, exactly, it is we are constructing, and as far as I can tell, we're pretty far away from any idea of exactly what that is.

View on HN

Is consciousness a necessary component for AGI, though?

>Is consciousness a necessary component for AGI, though?

I always thought so. Like, without volition, it's just a really good natural language tool, and not a general intelligence.

What would an artificial general intelligence without consciousness look like to you?

A really efficient slave. Essentially it does not get creative and is ultimately predictable, even in failure modes. It would also have no subconscious drives to do unexpected things.

Capitalistic dream.

without creativity... I don't think it would make for a very effective slave in a world that is constantly changing.

Is creativity possible without consciousness? I don't pretend to be an expert.

Probably possible, yes. Subconscious drives are a thing. You could have sentience without consciousness. (The difference is that consciousness can change or modify drives and recognize its own actions on the world - its own agency. It is related to sentience and the line between the two is thin and blurry.) I'm not sure it is possible to be conscious without having been at some point sentient.

The idea was that such an AI would never question its priorities or change them, not that it wouldn't be able to solve novel problems.

(Which is the definition of intelligence, duh. And shows how far we're from that. What we have is glorified pattern matching algorithms and some basic symbolic logic and clustering.)

That sounds more like a Communist dream. Capitalism doesn’t reward really efficient slaves.

Asimov's Three Laws of Perfect Slavery (Forgive me Isaac!)

If such beings were conscious they would be saints.

One of my favorite AI jokes:

Q: What's AI?

A: When the machine wakes up and says, "What's in it for me?"

It would be fetish to actually make humanoid servitor robots. IMO.

Like the intellectual equivalent of a beast of burden - good at solving open problems by (apparently) intuiting connections and making inferences, probably with the appearance of creative operation and experimental learning, but lacking any genuine desire or personal agency.

IMO this is much more plausible as AGI than some kind of mythical general meta-intelligence that has a virtual soul.

I don't see why consciousness or agency are at all necessary for AGI - except as a science fiction trope.

>good at solving open problems by (apparently) intuiting connections and making inferences, probably with the appearance of creative operation and experimental learning, but lacking any genuine desire or personal agency.

so, like a philosophical zombie? like something that appears to be conscious but isn't?

Besides which, consciousness and agency are two different things, damnit! I've got a github repo in which I train agents by policy optimization. I have no illusions that my Pytorch models are conscious.

I mean, we could call it an "artificial general intelligence" or we could call it a "generalized optimizer & problem solver"

So basically if we can have a system that can solve any problem thrown at it, that's essentially the AGI.

I don't think it's necessary for us to get a machine to reach "consciousness", whatever that is, before we realise it's overtaken our mental ability to combine two existing concepts in order to create a new one.

I imagine this is what's going to happen: we humans have drawn two circles (thank you dear fella for that analogy) and we are about to draw a third, combing two existing concepts.

At some point we will have drawn so many circles that the next one will not be drawn by us. It will be drawn by AI.

Who knows when we'll reach that level of machine intelligence and if it even requires them, the machines, to be conscious? But we will, without any doubt, reach a point where there have been so many circles drawn that they are now smarter than us at drawing them. Absolutely without any sort of doubt in my mind this will happen.

Tens of thousands of people comprehend Einstein's work. It used to be hundreds. We're smart as shit these days. Fuck me if it's not before I die.

>Tens of thousands of people comprehend Einstein's work. It used to be hundreds. We're smart as shit these days.

In some ways, I feel like this is the real singularity. This trend towards feeding and offering education to everyone. Like, it used to be that to be an intellectual, you'd need to be born rich to get the free time and education. I mean, being born rich still helps, sure, but a lot more people are getting a shot.

On the other hand, it's a process that levels off at some point, like everything else. The population is leveling off; as we bring education and leisure time to more and more of the world, eventually most of the people who have the capability to do this sort of thing will have done it.

I mean, I see that with computers, too... like with the leveling out of Moore's law; exponential processes, in nature, tend to not remain exponential.

If you subscribe to the theory of great and continuing, accelerating change due to technological improvements, advanced technological education for everyone is an expected ovservation during the decades or centuries it’s occurring :)

Reply on news.ycombinator.com