Hacker News Reader

Comment by nateb2022 | Hacker News Reader

nateb2022

Consider a calculator, a machine capable of valid transformations wrt the semantics of arithmetic.

Does a calculator reason its way to 2 + 2 = 4? No, it merely transforms.

An LLM is simply a much more complex version of this; except that its complexity hides a lot of what is otherwise reducible to a transformation of semantic, conceptual, and inferential relations. There are lots of NLP models prior to the introduction of modern "AI" that would make these representations more clear.

However I don't believe any of the aforementioned philosophers would ever consider reason itself as a transformation, as Kant especially spent a lot of time differentiating apprehension from transition.

View on HN

I’m not so sure. While I completely agree on pure transformation, I continue to believe that the most concerning aspect of neural networks is that they really strengthen the argument that consciousness is the illusion.

The interplay between deductive frameworks and inductive frameworks is still always buoyed by the black swan problem… which is a the heart of Hume’s solipsism. Kant’s pure reason never considered instinctual logical gates that come prepackaged in the mind. That seems frighteningly close to a prediction algorithm, no matter how much we wrap it in a concept of understanding. In the end, we’re just building a model with a minimal error rate.

> I continue to believe that the most concerning aspect of neural networks is that they really strengthen the argument that consciousness is the illusion.

Based on what? Just because you can model the weather using math doesn't imply at all that the weather is the product of a mathematical model itself. I don't see how reasoning is different.

We deliberately construct LLMs so that they approximate certain relations typical of reasoning. Suppose they become extraordinarily successful at doing so. What exactly have we demonstrated? That these effects can be artificially reproduced through such a mechanism. But how would it follow that the thing being reproduced must itself be nothing more than that mechanism?

> Based on what?

Based on the face that we have zero explanation for consciousness. Based on the face that biologically neural networks are exactly a kind of neural network.

My point isn’t that we’re able to explain consciousness as an illusion. It’s that we’re able to explain consciousness-like behavior presumably without consciousness, and the infrastructure look uncomfortably like we do.

What does consciousness have to do with reasoning? You're conflating two different things.

Consciousness is the what-it's-like of sensation you have as an organism from your sensory modalities and having a body. For human, that's colors, sounds, tastes, smells, feels, feeling of balance and what not that happen in perception, hallucination, memory, imagination, inner dialog and dreams along with emotions. Each sentient organism has their own sensations and emotions making up subjective experiences that nobody else has access to except inference from behavior and language (for humans).

That's not reasoning (just the having sensations/emotions). Reasoning is an adjacent activity that can be conscious but also unconscious. And it's hard to say that colors, sounds, emotions are illusions. What does that mean? We do experience them. There is something it's like to see color which is different from what it's like to feel pain or to be in love. And none of those (presumably) are what it's like when a bat experiences echolocation (there are many examples from the animal kingdom if one takes issue with echolocation).

>Consciousness is the what-it's-like of sensation you have as an organism

>Each sentient organism has their own sensations

This does not follow besides feeling like it should.

Western tradition has firmly put pure reasoning in a cage. It serves its purpose only as a source of hypotheses, conjecture and thought experiments. Abrahamic religions, while still not accepted sensu proprio by many, are much less mystical than the Eastern Religions. You can see the difference in the unusual creative process of Ramanujan.

Which tradition are we to follow then whilst training the AI models? Pushed beyond their limits today's models tend to hallucinate and cannot be accepted sensu proprio. Philosophers will call it speculative reasoning. And yet, some speculation can be useful in breaking new grounds. Anyway, I have no answers. Just musings.

Philosophy isn’t “western,” it’s reason. The “non-western” traditions you reference are effectively religious and are not based in rationality.

> I continue to believe that the most concerning aspect of neural networks is that they really strengthen the argument that consciousness is the illusion.

I think it’s not a question of it existing, but to what extent does conscious can control the organism (as opposed to being a passenger). There’s some intriguing research on the topic.

Suppose you put an LLM in a "inference -> observe -> retrain weights" loop.

Supervised training uses this precise loop. The difference between inference and observation is called the loss function.

Models are now starting to show emergent behavior that they were not explicitly trained for. This is leading to the current debate.

You made me start listening to Critique of Pure Reason [1] :-)

1. https://www.youtube.com/watch?v=cyhXtJN2AQQ

> Does a calculator reason its way to 2 + 2 = 4? No, it merely transforms.

Yes, it reasons through machine instructions in the silicone.

Can someone explain how we reason that 2+2 is 4.

Reply on news.ycombinator.com