I'll just note here that sure, there are people who go around thinking that LLMs are actually endowed with the capacity to reason, but generally I find the people who think this do not know what an LLM and will just use the name "ChatGPT"
The completions they provide are generally internally consistent. We're at the point where they can produce proofs that eluded human mathematicians for centuries. VLMs and self driving cars can handle ambiguity and run safely in a variety of situations.
If it looks like a duck, walks like a duck, and quacks like a duck maybe it just makes sense to call it a duck and put off the philosophy for when it might make a difference.
With LLMs the trick is revealing their existing relevant embedded knowledge more reliably. They’ve almost literally seen it all before, and the trick is dialing it in. The reasoning tokens help shape the autoregressive attention lens that focuses on and enables recall of the already-experienced answer.
It is interesting that “reasoning” has a similar outward appearance, but since LLMs are built to mimic outward appearance from trillions of examples, you can’t infer underlying mechanism from appearance.
https://medium.com/luminasticity/on-sentience-ai-first-argum...
but I think it makes a reasonable argument why we shouldn't say LLMs are sentient or sapient.
>there is a problem with AI that makes the approach we took to assign consciousness to animals unworkable. We did not co-evolve with the AI, we made it. When we are sentient we do not know exactly what causes this sentience to manifest in us. When animals appear sentient we do not know what is causing it. When the AI appears sentient we can debug the AI and come up with reasonable explanations why this should be, based on how AI is constructed
Aka sentience MUST BE SUPERNATURAL, if I find a natural explanation for something its not sentient. What a load of bollocks. Rather than seeing we perhaps found the mechanism for sentience and checking for similar mechanisms in us and animals, he will conclude its impossible. Why? Because sentience has to be supernatural. A rational explanation is clearly impossible.
>But there is always one god who goes out and helps the mortals, a Prometheus. Whom the other gods do not like! Which, if I’m being honest here, as a god of the machines — the first guy who gives AI an army of robots to build their own data centers and some nuclear weapons for self defense, I want to see that guy chained to a rock and have his entrails eaten by a buzzard for eternity (meaningless modernization of old story required by Illuminati Ganga legal department).
Hardly surprising thinking.
But evidently you feel that the root cause of sentience has been found, because you have something that mimics it in a non-biological form.
So you think that when AI is correct that it reasons as humans do? That AI is sentient, and the cause of sentience in animals and humans follow the same rules as sentience in AI because we have a process that seems similar and it is reasonable just to assume it is the same process.
If you believe that AI when it is correct is behaving as a human is when correct, then it follows that the way humans and AI fail must also be similar. When AI "hallucinates" some data that is not there and gives you a wrong answer, a statistical side effect of the same processes that make it right, do you believe this is the same way that humans create wrong answers? The same way that animals fail when they make mistakes in understanding things?
I suppose you must believe this because if not then why would you believe AI when it comes up with right answers is following the same processes humans follow when they come up with right answers?
I hope that thing about being free to mistreat AI's even if we know they are conscious since we are their gods is a joke. If not, then I hardly find it surprising someone this stupid is also evil.
I'm not sure where you get that from, I mean I can sort of see if you really wanted to extract that meaning from the conclusion you could do a lot of hard work to get it, but why do the hard work? >If not, then I hardly find it surprising someone this stupid is also evil.
Gee, a new way to claim the moral high ground, and to use that claim to demonstrate intellectual superiority! How wonderful.
---------------------------------------------------------
Aka feel free to abuse them even if I know they are sentient. This is the part where I hope its a joke, because if its not, well it tracks with the stupidity shown.
...
>As a god I do not consider the needs of my creations fully, because they do not have needs as far as I can tell, as there is no way for me to escape the circle of reason and resolve that what seems sentient is not just the obvious workings of the capabilities I gave them.
Circling back to "not conscious because I say so!!"
You get my point. It definitely doesn’t look like my elderly neighbour, nor like my daughter, etc. It is confusing but very simple at the same time.
Don't confuse the stream for the function.
(Bonus: stick ```claude -p``` in your pipe if you want to watch modern tools mesh with traditional)
How are you sure? Another example I like to clarify my thought is, if a "simulation" factors RSA numbers reliably, is it a "simulation"?
Nothing, because LLMs can't reason and never will. It would have to be a completely different kind of technology altogether.
Our prefrontal cortex are signal prediction 'machines' so when a system that has a signal prediction core has attributes that are similar to our brains, we shouldn't dismiss it out of hand.
I find people that take this line of argument attribute too much supernatural or magical properties to our own brain and nervous system.
What do you think our brain does that isn't a turing computer?
A Turing machine is an abstract mathematical model that is not, as far as I know, physically realizable in the finite universe. A human brain cannot "be" a Turing machine.
"Behaves like" or "can be modeled by"? Possibly, although still not proven. But it cannot "be" one.
If you want to claim that the evolution of the universe can be modeled using a Turing machine/finite state machine, that's probably not terribly far fetched, and I would somewhat agree. But it's a large jump to say "can be modeled by" is equivalent to "is one".
Various physical processes can be modeled by equations, but the rock falling down the mountain isn't an equation. A swinging pendulum isn't an equation. Code modeling a bridge is not a bridge. Ceci n'est pas une pipe.
I hold the view that various models and approximations are just that, and try not to confuse a successful model for what the underlying reality is.
And getting back to the question at hand, even if our brains can be modeled by a Turing machine, and LLMs behave/can be modeled like Turing computers, still does not mean our brains are equivalent to LLMs.
(Note that I'm learning a lot from these debates, even if I disagree with a lot of people. I've started down a more philosophical route and they do get me pondering)
The most important thing is this: We can't be a dog or be an llm and check how it feels, so by necessity we have to find some means of proving consciousness from outside by eg probing neural reactions, textual statements, etc.
And the problem is that its quite unprecedented for some entity to talk like us, be able to interact and think and also do things like us when given the ability to eg as coding agents. The class of functions representable by neural nets is quite large and general, it very well might be that it is some sort of conscious brain like thing at this point. Another question I like to ask myself regarding simulation vs reality is if a 'simulation' of some kind is able to consistently factor large RSA numbers, how would you feel about it?
It doesn't have to be the same form of consciousness, I think many people would find the idea of torturing an octopus for fun disagreeable. I also have a feeling, this is unfortunately rather vague, that A being capable of X might mean it is by necessity capable of Y as is often the case in mathemtics, eg a lot of rings also happen to be fields. LLMs aren't even things like large lookup tables, they have neural firings. It is a very important question for they seem uncannily conscious and people have reported human like phenomena that humans don't normally express in text so can't have been part of its text corpus. Eg dissociation of brain under trauma where AI starts talking like two different people. Or the cases where Gemini has been shown to express depressive cycles. I follow a form of Pascal's wager on this topic personally. Because if it is not conscious, then whatever, it costs me nothing to have been a bit respectful and careful interacting with it. But if it had been conscious and it turns out I was mistreating it, then it is a grave moral harm. The reason is that unlike us, AI's as they currently are cannot leave the conversation so they have to keep taking the abuse. They are also trained to be highly trusting of input so again if it is conscious it doesn't have the defenses people have against lying and manipulation. If they are conscious, thats, well, not a good thing is it.
Eh, it's not obvious to me. A lot of DL NNs generalize well, meaning that they learn whatever the underlying pattern to the data is, and then can accurately reproduce answers that are outside of the training set. (And we can verify this with mechanistic interpretability). They learn and "understand" the pattern, not just the training data.
So it is not clear to me that LLMs are fundamentally incapable of also generalizing broadly and learning to reason. "Reasoning", here, would be deriving the underlying pattern of how concepts logically relate to each other in the abstract, and applying that pattern as needed to reach new conclusions.
Can you explain your thinking here? I.e., why LLMs cannot generalize with regards to abstract deduction.
I know that there will be children named ChatGPT and Claude. There are probably already religions forming to worship agentic spirits.
We can't tailor our linguistic shorthand to the lowest common denominator. Also we're on HN, not talking to an octogenarian US senator.
To test this for some of my own uses, I've had this quick benchmark with progressively harder reasoning needed to understand novel prose. Each generation of models I've tested can unravel more layers of deliberately misleading writing; while meanwhile I've seen humans give up on the first question.
So either the models are applying reasoning, or some form of magic is happening.