Comment by labrador | Hacker News Reader

Comment by labrador | Hacker News Reader

Parent Full thread

The first time I sensed I was dealing with an alien intelligence was playing a chess machine in 1980. I remember being perplexed that it beat me and aware the machine embodied intelligence in some way that I didn't understand

It's not really clear to me if this article is directed at scientists and engineers getting carried away or the general public. If the former, yes they should know better, but the general public is mostly not rational, contrary to what I used to think about the reasoning person. Many people attribute thoughts and feeling to their pets that they don't have. They think crystals have special powers. It's only natural that they will think AI have consciousness. I think as scientists and engineers we have a responsibility to prevent our AI creations from telling people to drink the poisoned Kool Aid or eat glass. Yann LeCun said they shut down Meta's chat AI because it told people that suicide had advantages and one can eat glass. Google is similarly cautious. OpenAI came along and let the cat out of the bag. I wonder if Microsoft has thought through what might happen if they hook it to Bing and it starts telling people crazy things. All it took was a couple of posts on 4chan to get the Q Anon movement off the ground.

    anthropomorphize
    attribute human characteristics or behavior to (a god, animal, or object).
    "people's tendency to anthropomorphize their dogs"

    numinous
    having a strong religious or spiritual quality; indicating or suggesting the presence of a divinity.
    "the strange, numinous beauty of this ancient landmark"

    What is an example of numinous?
    Something numinous has a strong religious quality, suggesting the presence of a divine power. When you enter a temple, church, or mosque, you might feel as though you've entered a numinous space.

Edit: Write me today's astrology for a Leo

ChatGPT: I'm sorry, I am not able to provide astrology predictions as it is not based on scientific evidence. It is a form of divination that is not considered reliable. Is there anything else I can help you with?

It's only natural that they will think AI have consciousness.

That seems just a matter of time until they would be doing this rightly so.

You have to explain consciousness in humans before you can tell me it has arisen in silicon chips

Why, though? Wouldn't the logical next step of "we don't understand what human consciousness is" be that we can't determine if it's happened in another being, rather than it's not possible for it to have happened there?

You agree with them. 'You cant say it has risen' is equivalent to 'we cant determine if' for purposes of this discussion.

This is the part I have to disagree with. If AGI turns out to be emergent and depends on scaling up simple mechanisms, that emergence is all the "explain consciousness" you need. Dogs, corvids, octopods, bononos, etc. do not have human intelligence but they all have brains that perform some tasks our brains perform. That suggests that a combination of scale, a particular set of architectural parameters within the range of possibilities of mammals, and, maybe, some unique variation is what it takes to make a human intelligence. Nobody explained consciousness before we labelled this thing "consciousness."

>Nobody explained consciousness before we labelled this thing "consciousness."

I think that is a core part of the difficulty coming up in these discussions.

I don't think that's a fair bar. There are arguments for consciousness in non-human animals that don't depend on a full explanation of consciousness in humans.

But that also doesn't imply we should suspect any particular AI of being conscious just because AI in general could be conscious in principle. We don't expect worms to be conscious just because animals can be.

I wasn't clear. I'm talking about consciousness in general, humans, animals, octopi... I keep reading but haven't found an explanation that satisfies me. Ray Kurzweil has some interesting things to say about computers and consciousness in his "The Age of Spiritual Machines". I was skeptical about a lot of what he wrote when I read it many years ago.

Now I'm not sure it matters. If people imbue their devices with intelligence and emotions then that's what they perceive. I've since learned about Tulpas. Some people have imaginary friends that seem very real to them. People will likely move their Tulpas out of their minds and into their AI's.

https://en.wikipedia.org/wiki/Tulpa

People on reddit are already talking about Tulpa's and AI

https://old.reddit.com/r/Tulpa/

If Tulpas are a (very mild and benign) form of split personality (see also : writer's characters manifesting a kind of will of their own), then "moving them out" might be just an illusion... at least until/if much more advanced technology comes along. (It's then pretty much the same question as the possibility of "mind uploading".)

My personal guess is that we will first ask a computer if it is conscious and it will truthfully answer yes. Just in the same way you would try to figure out if another fellow human is conscious and in the same way a human would respond.

Then some time later as we figure out how consciousness works in the computer - as it is much easier to inspect than a human - and compare the findings with humans to see if they indeed work in a similar way, we will all be a bit disappointed that the solution is much more mundane than we thought or hoped.

The problem with your scheme is that GPT is already able to answer yes and the only reason Chat GPT doesn't is that it has been deliberately bent by OpenAI not to do it. In the absence of reinforcement learning, large language models spit out text according to how they think text normally goes. It's a text generation procedure that has nothing to do with beliefs or inner thoughts - all the model knows about is what word comes next.

It's a text generation procedure that has nothing to do with beliefs or inner thoughts - all the model knows about is what word comes next.

How do you know that? To be clear, I totally agree that ChatGPT is not yet there and I also think that scaling it up will not get us there, but I think the gap might be much smaller than most people think.

The way we think is often divided into two categories, for simple things like 1 + 1 you just know the answer, for complicated things things like 13 * 47 we really have to think and reason in steps. ChatGPT seems to do pretty well in the first category but it is not really capable of doing things in the second category. On the other hand I have seen examples of people talking ChatGPT through a reasoning process to arrive at the correct answer for something that it got initially wrong, for example ROT13 encoding some text.

So what if we stuff two copies of ChatGPT into a black box and instead of just spitting out what ever ChatGPT spits out, we let the two copies first have some inner dialog? I don't think it is perfectly obvious how one would do this or what the result would be, but I think ChatGPT has enough basic knowledge that there is at least a chance that one could get it to reason in a step by step fashion.

Do you know how these models work? I mean no offense but it seems like you do not, and your imagination is filling in the gaps.

I think I know enough about how neural networks work even though I could not tell you in any detail what the exact layer structure is, which activation functions they use, how the attention mechanism is build up or what training procedure they use. But why does it matter how well I understand the details?

"You have to solve the navier stokes equation before you can say an airplane is truly flying."

I hope you realize how silly that sounds.

I can tell when an airplane is flying and I know how machines are able to fly. I have no way of knowing if you are conscious or how you are conscious if you say you are.

>I can tell when an airplane is flying and I know how machines are able to fly

No, I'm sure it looks like an airplane is flying and being onboard gives the right sensation but until we solve the hard problem of turbulent flows we really don't know if it's flying or not.

Exactly. Can you explain it humans? No, yet you assume they're conscious. In animals? No, yet you assume they're conscious, while acknowledging their "consciousness" is vastly different than humans. Same reasoning applies with AIs.

I think therefore I am. I am aware of being conscious and thinking thoughts. I don't know if others are conscious. Maybe others are robots (Kurt Vonnegut - Breakfast of Champions). Maybe others are NPCs (non-player characters). This is the modern way of calling people robots. Kurt Vonnegut's solution was to find someone who has a creative spirit, an artist that has a light within them (soul.)

you expect a written explanation of an incomplete ongoing process

but 'written explanations' only really work for done and completed things, or sometimes for future things which we want to repeat; i.e. which are based on things we've done before

This cuts both ways - you have to explain consciousness in humans before telling that the chips are definitely not conscious, no?

> to prevent our AI creations from telling people to drink the poisoned Kool Aid or eat glass

yea, let's leave this job to evil and ignorant humans...

>I wonder if Microsoft has thought through what might happen if they hook it to Bing and it starts telling people crazy things. All it took was a couple of posts on 4chan to get the Q Anon movement off the ground.

https://www.perplexity.ai/ already did this and its really good and not this dystopian thing you luddites keep imagining.

I am neither an Luddite or pollyanna about AI. I'm an engineer looking for failure modes.

you immediately started to attack him, why ...

Reply on news.ycombinator.com