Comment by bko | Hacker News Reader

Comment by bko | Hacker News Reader

I read through the conversation and it strikes me as silly [0]. The goal of training these bots is to try to predict actual conversations and this conversation reads kind of like you would think it would read like. Also, this model is almost certainly probabilistic so there's no real "sentience", there's setting a few meta-parameters, and often times cherry picking the output that makes the most sense.

I wrote about this before [1]. I just don't think any neural network as we have them today could result in "sentience". At the end of the day, you're taking words, turning them into some numerical encoding, doing a series of matrix multiplications and non-linearities on the numbers and getting some other numbers out, which you can then convert back to words. That's it. There are magic numbers out there (weights to the neural net) that when multiplied by vectorized word embeddings and transformed back to words could convince someone that they're speaking to a human. But that's not sentience. What would even be sentient? The numbers in that order? The neural net meta-data? The bits in the computer? They're just numbers. Why don't we apply the same reasoning with words. Someone can write a novel so captivating and prose so well written that we could believe the events in the book really happened, or the characters really exist. But we don't because they're just words on a page. But somehow we think numbers in the write order in the right program can somehow create conciseness.

[0] https://twitter.com/tomgara/status/1535716256585859073

[1] https://mleverything.substack.com/p/artificial-general-intel...

> At the end of the day, you're taking words, turning them into some numerical encoding, doing a series of matrix multiplications and non-linearities on the numbers and getting some other numbers out, which you can then convert back to words. That's it. There are magic numbers out there (weights to the neural net) that when multiplied by vectorized word embeddings and transformed back to words could convince someone that they're speaking to a human. But that's not sentience. What would even be sentient?

This is interesting, and I think you've reached the right conclusion, but the reasoning is flawed. It has to be flawed, because it can be used to prove that we're not sentient either.

Just create an arbitrarily accurate digital model of a human brain and step through time. Is it sentient? If not, make it more accurate until it is. If it is, where is the sentience? It's just numbers and algorithms, right?

You could argue that sentience just doesn't exist, which may be valid, but if it does, I don't think you can get very far with a reduction-to-parts analysis.

> Just create an arbitrarily accurate digital model of a human brain

When you can ""Just"" do that we'll see I guess.

You can go that route, but thought experiments are a very fruitful part of science/philosophy.

You'll miss out on a lot if you stop as soon as our engineering can't catch up to our imagination.

Maybe physical reality isn't the base layer. Maybe it's consciousness. I'm not sure I would go that far, but there are a number of phenomenon that are difficult to explain through materialism (physical reality is all there is). I was particularly moved by Bernardo Kastrup. Here he is on simulation:

> This, however, is premised on the notion that a simulation of a mental phenomenon is equivalent, in essence, to the phenomenon itself. There are many compelling arguments against this notion in philosophy of mind, the most well known of which is perhaps John Searle’s (2004). To gain some intuition about what these arguments generally entail, consider this: Do we have any reason to believe that, by performing a perfectly accurate simulation of kidney function in a computer, the computer will begin urinating on its desk? Clearly not. There is an essential difference between a computer simulation and the phenomenon it simulates; they are not the same thing, no matter how accurate the simulation. Yet, those hoping to “upload consciousness” under the physicalist narrative seem to become so engrossed in abstraction that they lose touch with basic intuitions of plausibility. Their neurosis is, in this sense, comparable with religious dogmatism.

https://journals.sagepub.com/doi/full/10.1177/21582440166745...

A few things wrong with this in my opinion.

1. If simulation of a mental phenomenon is not equivalent, in essence, to the phenomenon itself, then what's a meaningful difference between the two? What is this elusive essence of the phenomenon and why is it a requirement in any meaningful definition of intelligence, consciousness, sentience, etc? If we build a silicon brain some day that passes as human, does it matter that its essence is different from a carbon-based brain? 2. Searle's Chinese Room neglects to acknowledge that the room itself is what would be exhibiting intelligence, not the homonculus within. His argument fails by presuming without evidence that consciousness is not an emergent phenomenon. 3. Of course we don't expect the computer to urinate on the desk, just as we don't expect a silicon mind to produce serotonin. That's irrelevant to the question of whether or not it exhibits the qualities or behaviors that we associate with consciousness.

I guess I just don't understand why there needs to be this added, untestable, metaphysical requirement attached to all of these words. And my question to those who suggest this is, what would it convince you otherwise?

Could you elaborate on what phenomena are difficult to explain through materialism?

This is the big one

https://en.wikipedia.org/wiki/Hard_problem_of_consciousness

Well, as the article itself describes, there are many decent materialistic explanations for this problem - I personally am fond of Daniel Dennett's interpretation of it.

>Just create an arbitrarily accurate digital model of a human brain and step through time

This is a thought experiment, there's no actual evidence that you can create an accurate digital model that is equivalent to an analog and biochemical brain. Just claiming that you can reduce biological organisms to computational processes does not make it so, it's just an idea that hackers find cool.

That either sentience or intelligence is just "numbers and algorithms" is conjecture with not much going for it. We as humans use numbers and algorithms (which are really figures of speech, not things), to explain physical reality, not the other way around.

> there's no actual evidence that you can create an accurate digital model that is equivalent to an analog and biochemical brain.

There are mountains of evidence in favor of the Standard Model, which is computable. Therefore, there actually _is_ evidence that the human brain can be similarly computed. (Whether our technology can ever come near to this kind of power is where I have doubts). What there _isn't_ evidence for is that there is anything non-computable within the human brain.

What about bells theorem, which implies that the universe has inherent non determinism associated with it?

Meaning that computability doesn't matter if there's inherently randomness at the smallest scales

You can have computation that requires randomness either in its output or in its processing.

> We as humans use numbers and algorithms (which are really figures of speech, not things), to explain physical reality, not the other way around.

The numbers and algorithms here aren’t what anybody is saying could be sentient. Those numbers and algorithms are representations used to coerce physical objects into doing computation. Processors don’t care about the algorithms you see on your computer screen. So then the questions are “can computation be sentient”, or “are sentient things sentient by way of computation”, and I think that’s much, much harder to answer.

I do agree, my point is rather that the question "“can computation be sentient” is trivially answered as, no it cannot. Is a computed brain sentient is like asking, 'is a computed sun hot?'. Computation is a subjective interpretation of a physical process. You don't get wet from a machine that computes the weather.

A digital machine and an analog one aren't equivalent merely because their behaviour can both be explained in the same terms. (i.e. computationally). That's not to say by the way you can't make some kind of sentient digital system, just that computation isn't really relevant to answer that question.

Computation does take energy and precisely modeling the sun would take a lot of it. Makes me wonder: would precisely computing the physics of something hot produce more heat than computing something cold?

Higher temperature systems tend to have more microstates than lower temperature ones, so it seems like in principle you need less information to encode the state of a cold system than a hot system. I'm not sure how to translate that to the energy cost of computation of that system though.

> Just create an arbitrarily accurate digital model of a human brain and step through time. Is it sentient? If not, make it more accurate until it is. If it is, where is the sentience? It's just numbers and algorithms, right?

You can trivialise anything using this rhetoric, but that doesn't make it meaningful. Just think hard until you can move objects with your mind. If you still can't, think harder. The majority of things are more complicated than that [1], even if they are 'just numbers and algorithms' (also not proven). 'Doing more of the same' is not the only, best, or universally-applicable route to advancement.

We don't fully understand the brain, let alone consciousness itself. Throwing more and more compute at the problem without discernment is like trying to understand music by setting louder and louder alarms.

[1] https://xkcd.com/793/

> Just think hard until you can move objects with your mind. If you still can't, think harder.

That only works if "move objects with your mind" is possible to do. "It's not possible to make a model of a human" is a possible objection to the argument, but it raises the question: "why not?". There seems to be no blocker beyond cost and lack of research.

That there is no blocker doesn't mean there is isn't one.

I am as optimistic as they come, but let's look at much smaller problem: modeling an arbitrary analog circuit with DSP. The Nyquist theorem implies it's impossible to model such a circuit perfectly. Is there hope for perfectly modeling a physical brain with an electrical computer?

> What would even be sentient? The numbers in that order? The neural net meta-data? The bits in the computer?

What in would even be sentient in the human brain? The molecules in that configuration? The neurons? Maybe large groups of neurons?

Computation isn’t just abstract pieces of software executing off in some aether. It’s done through physical processes, taking place on real, physical pieces of hardware that exist in the same universe that our brains do. Most would say the atoms in your brain aren’t sentient, but you are sentient. So the best answer is that no, the bits and numbers in the computer aren’t sentient, the system as a whole is the thing you would call sentient.

I’m not trying to die on the hill that neural nets are sentient, because I think at this stage they are not. But I don’t think this common framing of it is the nail in the coffin people think it is.

Reply on news.ycombinator.com