Hacker News Reader

Of course Mac running Safari is a glorified boolean circuit. It's programmed in a way to show the pixels on the screen in a way that you understand, and take inputs from your mouse and keyboard in a way you understand

It is true that a Mac running Safari is just a glorified boolean circuit. It is also true that a human being is just a glorified bunch of subatomic particles. It's not a useful way to understand what's going on. There is a higher-level structure built out of the lower-level pieces, and it is the higher-level structure that is important and not the lower-level pieces.

It would be possible to make a web browser out of a different type of logic circuit. It's the higher-level structure of the browser that matters, and not the fact that it is built out of boolean logic.

Similarly, with ChatGPT, it is the higher-level structures (whatever they may be) that matter, and not the low-level details of the neural network. The higher-level structures could be far too complex for us to understand.

I’m sure you don’t think it follows a human is a glorified rock? Both objects are processed by the universe under the same rules, and understanding how consciousness arises directly from physics is likely intractable.

> That would point to higher-level phenomena going on inside ChatGPT and its ilk, than merely statistics and predictions.

No, it wouldn't, because nothing in "higher-level phenomena" precludes it being caused by statistics and predictions.

I don't understand that. In computer science everyone learned that computation is best described and explained at several levels of abstraction. E.g., HW/SW interface; machine code vs C++; RTL vs architecture, the list of levels of abstractions goes on and on. So what is the reason for not appropriately extending this idea to analyzing whatever a neural network is doing?

I look at in the following way: understanding something by abstracting over lower level details doesn't mean the abstraction is how things actually work, the extra layer of abstraction may just be a nice way of thinking about something that makes thinking about it easier. But in the end the true mechanics are the sum of low level details.

In general abstractions are not perfect, hence 'leaky abstractions'.

That's a philosophical argument, whereas people are implicitly denying that ChatGPT can have higher level abstraction at all as if it were valid to claim that Stroustrup's contribution to C++ was ultimately just a way to write a lot of assembly code, so let's forever only think in assembly terms.

(As a separate tangent, I don't accept the philosophy that abstractions are merely human niceties or conveniences. They are information theoretic models of reality and can be tested and validated, after all, even the bottom level of reality is an abstraction. The very argument used to deny the primacy of abstractions itself requires conceptual abstractions, leading to a circular logic. But then, I'm not a philosopher so what do I know.)

(Poster from several comments upstream.) I'm not trying to deny the ChatGPT model has formed higher-level abstractions; in fact, I think it's fairly obvious you have to do this in some way in order to obtain such a level of breadth and accuracy of response.

However those abstractions and their formation still boil down to statistics. This is similar to how e.g. mechanics of macroscopic bodies still boils down and reduces to quantum field theory and gravity, even though that's not the best way to explain or understand what's going on.

You’re definitely on to something.

An example: when we interact with other human beings, we often really only care about the surface, don’t we? Mannerisms, looks, behaviour. Very rarely do we question those with “why?”. But who does? Psychologists.

Same with any technology. Consumers don’t care about the why, they care about the result.

Scientists and engineers care about the “why” and “how”.

Now, is it important to understand “what’s behind the curtain”? Yes. But for who is it important?

It isn’t sentient because it stops when there is no input. If OpenAI were implementing a continuous feedback loop of not just textual info, but one that can sense the world like we do(visual, spatial, audio, tactile) and it keeps improves as it interacts with other people and world. It gets closer.

Humans also has continuous input from our body itself all the organs, cellular etc all giving the brain continuous input

Careful. You just proved that a human whole brain emulation is not sentient.

I don’t see the problem with LLM having a world model and superhuman intelligence without sentience. It seems very unlikely sentience comes from computation and that it’s not a physical property: why would only certain Boolean or mathematical operations in a certain order make your pen, piece of paper or CPU see colors or hear sounds? That the operations you do follow a complex plan is irrelevant. Conversely if you do enough mindfulness you can attain a state of complete empty mind / "no computation" while still being physically there, and it would not be surprising some animals with a very limited intelligence and world model have sentience. The burden of proof is on sentience not being some kind of fundamental property of matter or EM fields for me.

It's pretty much the truth. What the ChatGPT is good at is "keeping in mind" various associations between words that occurred in the session so far. To keep those associations some internal structure bound to get conjured. It doesn't mean the transformer understands anything or can do any kind of reasoning, despite the fact that it can mimic a bit how reasoning output looks like and even get it right sometimes if the context is fairly close to something it seen in the training data.