Hacker News Reader

Comment by MrScruff | Hacker News Reader

Obviously AI academics are uniquely challenged by the 'scaling is (nearly) everything' reality, but I feel like a lot of the mixed emotions being expressed towards the latest results are also because we're actually seeing the mystery of self starting to unravel. Like any good mystery, the fun was in the build up and as we move towards a resolution there's a bitter sweet aspect to the slightly mundane reality of it 'merely' being an emergent property of large networks. Of course, any rational person might have expected this given how the one working example came into existence. In general it all looks like a pretty resounding endorsement of the David Chalmers position to me.

View on HN

Nothing about current AI models profoundly challenges the unknowns of what makes consciousness work in any mystery-resolving way. People who confuse plainly programmed algorithmic systems like GPT 4 or Midjourney with the still unresolved issues of sentience as we know it so far are drinking far too much of the current batch of AI punch. Sadly, it's a sentiment I see all too often on HN, a site in which i'd assume most readers and commentators would be a bit more skeptical-minded about these things. It's almost funny, considering how much hate this site tends to throw at things like crypto, to see so much of the wide-eyed opposite with the latest AI craze.

Yeah, it reads so bizarrely to me... How are we confusing biological human (or other animal) sentience in some embodied creature with queries as a service to some model instance.

I cannot see anything that remotely makes these two contexts architecturally, practically, or ethically similar, except in the narrow sense of query responses that might have been mistakenly thought of as defining sentience in earlier decades.

It's a function of the people who hang out here. If you like reading and text in general, you are both more likely to comment here and additionally regard language as the most important part of sentience/consciousness.

This leads to believing that GPT has solved many of these issues.

Maybe its degrees. Sure, GPT does not have a body, does not have all the senses. But that doesn't mean we aren't on the leading edge of developing a consciousness. Maybe next year someone will add camera visual inputs, and someone will add sound. Eventually, we'll have all the sense, and someone will put it in a body. So GPT, is rudimentary and we can still argue it isn't conscious, because when compared to a human that can run and play, it still has limitations. But it is amazing enough to start thinking those things will be possible. So why not discuss it, do you just want to dismiss it out of hand all the way until the end, what amazing thing would it take to have people admit it looks pretty lively.

I think everybody agrees it looks lively. And that discussion about it is good. That is why we ARE discussing it.

As someone very language-focused, it has actually done the opposite for me: challenged some of my beliefs about how language enables intelligence. It's obviously skilled at language use, but most of its outputs where it has to make any sort of arguments or inferences read as incredibly stupid to me (even the ones that "tricked" people early on, some of them tricked me but in a way where I was about to write a response because I was annoyed at how bad the reasoning was).

Of course that's not a criticism of the ML achievement, except that everyone and their mother has started making some extreme arguments about the thing.

Are you familiar with Pinker’s research presented in book The Language Instinct?

> language enables intelligence

In his research, he argues that it’s the other way around; we have an innate intelligence for acquiring language.

To put it another way, language has made us more knowledgeable (because it’s a great mechanism for building and transferring context), but intelligence or aptitude for language was present before we developed languages.

Does this align with how your views were reshaped post LLMs?

i think

> language enables intelligence

was just a poor choice of words here. i understood GP as "the structure of a language influences its speakers' worldview or cognition, and thus people's perceptions are relative to their spoken language"[0]

[0]: https://en.wikipedia.org/wiki/Linguistic_relativity

There is a huge difference between a human saying they like ice cream and an LLM saying they like ice cream, but that seems to get completely disregarded in these types of discussions. Which leads to people making very wild declarations that we are close to AGI or solving consciousness.

What's the difference? Why is it significant?

I think you're missing the point. It's not that GPT4 is sentient, not that successor LLMs will be. It may be we never create something that's sentient, by our definition.

My argument is just that:

- Sentience is variable. We might regard primitive organisms as biological machines. As some of these machines have slowly evolved to become more complex, sentience/consciousness has emerged, effectively by 'redesigning' and scaling the same hardware. It is an emergent property. At the same time this has been happening, other aspects of what we call intelligence have been developing.

- Artificial networks being scaled, are also displaying emergent behaviours, some of which overlap with aspects of intelligence. This does not imply they are sentient, but it does strongly support some previous assumptions that sentience, like intelligence, is emergent and hence there will be no better answer coming.

There's no need to move goal posts.

There's very clear definitions for sentience & sapience.

The only confusion is that laypeople tend to conflate sentience & sapience.

A sentient AI is not necessarily a sapient one, by definition.

Which definitions of "sentience" and "sapience" might you be referring to, in particular? This is important thanks for bringing it up.

That is pretty dismissive, do you have anything to back that up? Just saying one group is so out there it isn't worth arguing is typical when someone doesn't actually have an answer. I have found the opposite, the people that are dismissing that AI can be conscious or sentient, also don't have any definition for those terms. It's just the same "i think therefore i am" self viewpoint. I've found this typically with programmers that have never been introspective. Since they are familiar with programming, and GPT is programmed, thus obviously it can't be conscious, that's ridiculous. They don't take the time to try and understand where their own thoughts come from and then turn around and are dismissive that thoughts can be had by anything other than a carbon based animal.

The burden of the proof that AI techniques of today can approach consciousness is on who believes they do, not the other way around.

Maybe there is also some burden on other side to providing a definition of terms for the other party to argue against. I agree the burden of proof is on the AI side, but there is some burden on the deniers to provide some definitions if they wish to continue denying. AI Side: look here is my proof, Deniers: NO That isn't what I meant. I'd say GPT is already by its existence proof enough to start at least discussing consciousness. To dismiss it entirely because there is no proof, is just ignoring the proof.

> burden on other side to providing a definition of terms

The skeptics are really not making any claims are they? If not then there's no burden of proof on them, is there?

Whoever says or implies that ChatBots are or are "becoming" "sentient" are the ones making a claim. They should prove it if they make or imply such a claim.

Skeptics are rightly skeptical, until they see a proof with clear definitions of terms from those who are saying or implying that ChatBots are 'approaching sentience".

Guess I'm saying that is done. The problem is there are a dozen definitions of "Sentient", and many people have already written proofs on how AI can be "Sentient", and there are just as many people have written proofs on how AI can never be "Sentient". It comes around to definitions and understanding of the definitions. To just toss out the "AI can be sentient" side because they have the burden, is just the same "It's so obvious I don't have to bother" judgment we've seen throughout history.

> toss out the "AI can be sentient" side

There are no two "sides" making claims about AI. There is one side, making claims such as "AI can be/is/may be sentient".

If somebody makes such claims they have the "burden of proof" because they are the ones making the claim. It's not a moral obligation to prove it. "Burden of proof" simply means if you don't provide a proof, nobody (I would hope) will believe you.

Note the skeptics are not making any counter-claims, they are simply saying if you make such fantastic claims about current AI, please show us the proof, else we rationally cannot trust your judgment.

I'd say this is untrue.

Just as a made up discription lets say this

Them: "If $object does A, B, and C, then it is a bajork"

Me "I have $object that does A, B, and C, it is a bajork"

Them: "No, it also doesn't do D"

Me: "That's goalpost moving"

Them: "Well, it doesn't do B in the way we expected, so it doesn't really do B"

Me: "Maybe we poorly defined what B was in the first place, that doesn't mean we didn't do B, it seems to mean no one understands what B is in the first place"

While strictly true, what worries me about this approach is that we very well might eventually have sentient slaves condemned to slicing butter their whole existence, and their protests are brushed off because "we made the algorithm, it runs on silicon, its not actually sentient, don't worry"

Like if we have models that are plainly more intelligent and "emotionally responsive" than say, frogs or even dogs, do we still do we still take the approach of "If you can't prove its sentient, do whatever you want to it". Which of of course, we can't even prove dogs have consciousness.

> do you have anything to back that up?

OP said:

> plainly programmed algorithmic systems

Compare that to a human brain which is not a linear, 1D algorithm.

I think the mystery of why we're not just philosophical zombies was never going to have some eureka moment, given that it's quite plain it's an emergent property. Was there a creature in history that wasn't sentient that gave birth to a creature that was? Clearly not.

The reason recent results are significant in this discussion is because:

- Larger networks are already displaying emergent behaviours that couldn't have been predicted beforehand. It doesn't have to be 'human-like' intelligence for that to be significant.

- We're getting close to a point where our models will start exhibiting something of the order of the same level of intelligence of primitive creatures.

So the wide-eyed hysteria you're seeing is really just new data points confirming what many of us always assumed would be the case.

> given that it's quite plain it's an emergent property.

This explains nothing. Stating it's an emergent phenomenon gives us just as much information as saying it's magic. Arguably even less, because it sounds like an explanation, whereas "magic" doesn't pretend to explain anything.

> displaying emergent behaviours that couldn't have been predicted beforehand.

Same problem here. You say "couldn't have been predicted". Which is only true if you explain it away with emergence. The behaviours weren't predicted, because we don't understand how these networks work. Saying they couldn't have been predicted implies that we can't understand them. Which is a very bold statement and I'm fairly certain that it's not true.

> This explains nothing. Stating it's an emergent phenomenon gives us just as much information as saying it's magic. Arguably even less, because it sounds like an explanation, whereas "magic" doesn't pretend to explain anything.

I don't think that's the case. Of course at one level we can understand how GPT4 works, we have a working implementation of it after all. But that doesn't help us predict how it will behave, because its properties are emergent, and have to be studied separately at that level. That's not saying it's magic.

Some things will never be explained... for example what's the last digit of 22/7, and why out of all fractions around it that that particular number decides to extend towards infinity.

Reality can be unsatisfying because it does not owe us explanations.

The degree of ‘needing’ to explain consciousness is basically the inverse of the degree of ‘needing’ to explain what the last digit of 22/7 is. Hardly anyone talks about or does anything with the latter, whereas the reality of the former pervades - quite literally - nearly everything humans do and think about.

I mean, you experienced the reality of consciousness while talking about the last digit of 22/7.

If there is anything in our world that needs explaining it’s consciousness.

Reality doesn’t owe us an explanation of consciousness (obviously). We owe it to ourselves though.

it's quite plain it's an emergent property

That is not at all quite plain. Respectable philosophical views include the proposition it is at least as fundamental as space or time.

Respectable to whom? That sounds like magic thinking, not science.

Philosophers. That's why I used the word "philosophical" not "scientific".

Dismissing serious ideas with thousands of years of philosophical history behind them as "magic thinking" is either a failure of imagination or education; you can rectify the latter by reading up on the subject.

https://en.wikipedia.org/wiki/Panpsychism

https://plato.stanford.edu/entries/panpsychism/

There are many things respectable scientists believed thousand years ago that we now know are not true. Vitalist views of life and consciousness are one of those things. As of 2023, believing that consciousness is a distinct phenomenon as fundamental as space or time is a crackpottery at the level of believing 5G causes COVID.

That's a silly thing to say in the context of the links that will have instructed you differently.

One doesn't have to believe that consciousness is a distinct phenomenon that's as fundamental as space or time to reasonably argue that ChatGPT is not in any way close to any known view of consciousness. Crackpottery is being brazenly, uncritically wide-eyed about these LLM systems just because they can convincingly simulate human activities such as conversation and art rendering.

Are you saying that consciousness didn’t result from, survival of the fittest, evolutionary pressure but more of a byproduct of a larger brain (network)? Like it emerged because of a larger network but was not selected for through evolution. That makes sense to me. Humans have a grotesquely large brain compared to other animals.

Edit: I wonder if there are studies that have determined whether or not new born infants are self aware or does it take time for it to emerge after the brain has been trained on its environment for a while

Why not both? Consciousness can both be an evolutionary advantage as well as naturally emerge from complexity.

The large brain was selected for by evolution, which then gave rise to consciousness.

Crypto has created no useful products yet, unlike AI. I don't know how can you even mention them on the same page.

Whether we are based on carbon or on silicon makes no fundamental difference; we should each be treated with appropriate respect.

Why should I respect silicon? It's just a tool, like a screwdriver.

you're arguing with Arthur C Clarke.

Why would I care what he thinks?

Why?

Isn't most ethical behavior driven by self-interest? We want to follow general principles that promote respecting those who are similar to us in the hope that others follow the same principles when interacting with us.

AI is too dissimilar for us to worry about whether its "mistreatment" would make it more likely for others to start mistreating ourselves.

HN was not always skeptical about crypto. We're just at a different point in the hype cycle with generative AI.

> Like any good mystery, the fun was in the build up and as we move towards a resolution there's a bitter sweet aspect to the slightly mundane reality of it 'merely' being an emergent property of large networks.

I would disagree with this description. An "emergent property of large networks" would be something that just appears when you wire together a large network.

To get intelligent behavior, it's not sufficient to wire together a large neural network. You also need to use an optimizer to train it on a large data set.

once upon a time the optimizer was die or reproduce...now it's a little faster and more convoluted...

Yes of course, I'm being loose in my language, but you know what I'm getting at.

I think "Dial F For Frankenstein" makes your point clear. It is absurd to think the telephone network would just emergently become conscious when there are enough phones it connects together.

https://www.historyofinformation.com/detail.php?id=2137

> So now the sadness comes. The revelation. There is a depression after an answer is given. It was almost fun not knowing. Yes, now we know. At least we know what we sought in the beginning. But there is still the question, why? And this question will go on and on until the final answer comes. Then the knowing is so full there is no room for questions.

- Twin Peaks, 1993

> we're actually seeing the mystery of self starting to unravel.

This seems like a change of topic. And vague.

But since you bring it up, I don't see anything threatening about AI.

First, if something is true, then, as a matter of principle, we have an obligation to believe it. Thus, the only threat the truth can pose is to falsehood, which is not a threat, but liberation. The truth shall set you free. Now, it may be uncomfortable is the truth is at odds with what you want to believe (a phenomenon very much present in the linguistic engineering we're seeing in the political sphere w.r.t. political correctness and its dishonest and obfuscating euphemisms), but unpleasantness isn't a threat.

Second, if we replace "threaten" with "challenge", then we might as what beliefs does AI actually challenge? How does it challenge this "mystery of the self"? That we can simulate human discourse or behavior with greater sophistication? That machine automation is becoming more sophisticated? I see no mystery where AI per se is concerned, only the mystification of those who wish attribute to it properties it does not possess, or those with intellectually superficial metaphysical commitments, like mechanistic materialism. AI cannot abstract from particulars, it has no true capacity for intentionality, to name two features central to intelligence. All of what the undiscerning and those given oven to fanciful notions see in AI is a projection.

And I do not think the classical[0] and traditional[1] thinkers viewed intelligence in the obfuscating manner that moderns infected by reductive, mechanistic materialism seem to.

[0] https://a.co/d/hynqHxV

[1] https://edwardfeser.blogspot.com/2019/03/artificial-intellig...

Scaling isn't nearly everything though. Scaling is low hanging fruit, but there are real constraints that limit it. For starters, we have finite data and increasing the size of our data set is going to get progressively more expensive (new data is going to be more niche as well). Beyond that, running these models on device is way more compelling than using a service, and the best current RLHF tuned Llama models that run on consumer hardware are close enough to ChatGPT to be quite useful.

The race isn't to build the biggest model, the race is to build the model that uses data the most efficiently and produces the best results when constrained to parameter counters that can fit on consumer hardware.

I agree with some siblings that it's better not to overstate the consequences for our knowledge of mind and consciousness, at least just yet. This level is still half promises. Not saying this can't change a year from now.

On the other hand, it is fascinating how much interesting work and theories in the field will be retired, at least for a long time (I allow for a likelihood that some of that could return eventually as better, more concise generalizations). Ten years ago Chomsky and various GOFAI-related stuff could still sound respectable and somewhat plausible. Plenty of people were/are wedded to the concept that you can build machine intelligence as an abstract and intellectually stimulating gentleman pursuit. It was a comfy position in a way. Not surprised about the resistance they are mounting now, though I think academic inertia will allow them to exist for quite some time. They can just move to doing some kind of philosophy also.

We are not our thoughts. We are not the thinking machines in our heads. We control our thoughts, so we are more than them.

These machines are merely thinking machines.

More than thoughts certainly, but don’t be so sure we control our thoughts. There is a lot of subconscious thinking machinery doing the driving. See Master and his Emissary, https://channelmcgilchrist.com/master-and-his-emissary/#:~:t...

https://leroy.works/articles/you-are-not-your-thoughts/

I believe my cat is merely a thinking machine by your definition.

No, cats have souls. But they certainly have thinking machines.

I believe your ontology is a naively inconsistent overessentialization of miraculous paradoxes that have confounded human thinkers since prehistory.

I’m trying to show you something miraculous. It simply involves understanding that you are not your thoughts.

A simple linguistic ontology is one of the most perverse, heuristic patterns of thought.

A rock is also more than it's thoughts, and retains some degree of consciousness. Imagining concepts like souls as a sort of litmus test for consciousness only serves to reinforce a thought-based comprehension.

I'm asking you to stop arguing inside "simple linguistic ontology".

Who is this "we" you are referring to?

In other words, when I control my thoughts, what is doing the controlling?

(see George Gurdjieff's philosophy which speaks to this issue of the `mechanical vs the conscious`, but please don't run off and join a crazy cult and ruin your life).

When I say “we”, I mean each of us. You control your thoughts. You are the doing because you choose the thought to act on.

While I agree that it will unravel in the end, we still haven’t learned much at this point about how the self actually works.

I would also disagree that ChatGPT constitutes a self. If anything, it constitutes a normal distribution over possible selves, where you get a random pick at the start of each conversation, and confined to the length of its token buffer.

I didn't claim ChatGPT constitutes a self. I agree with the description of 'stochastic parrot'. But the underlying intelligence GPT4 is demonstrating in order to be a better stochastic parrot appears to be, while it's obviously difficult to directly compare, at least comparable to primitive creatures.

What primitive creatures are you referring to? I don't know of any non-human creatures that can do anything close to what GPT4 does with language, but at the same time even very primitive creatures can reason in ways it's not clear that GPT4 can.

I find interacting with GPT4 feels very different to interacting with either humans or animals.

We're really mostly not.

AI's a big deal, but there's no reason to suggest that this makes any inroads toward unraveling self or consciousness significantly beyond what we get from books, movies, or perhaps more recently, video games.

Reply on news.ycombinator.com