Hacker News Reader

Comment by estearum | Hacker News Reader

> Any conscious agent, textual or physical, has to inhabit a world whose structure is stable, self-consistent, and rich enough to support persistent internal dynamics. Even a purely symbolic mind would still need a coherent symbolic universe. And this is precisely where LLMs fall short, through no fault of their own. The universe they operate in isn’t a world—it’s a superposition of countless incompatible snippets of text.

The consistency and coherence of LLM outputs, assembled from an imperfectly coherent mess of symbols is an empirical proof that the mess of symbols is in fact quite coherent.

The physical world is largely incoherent to human consciousnesses too, and we emerged just fine.

View on HN

Coherence here isn't about legible text, it's environmental coherence where you can deduce truths about the world through hypotheses and experimentation. Coherence isn't about a consistent story narrative, it's about a persistent world with falsifiable beliefs and consequences.

Right but as empirically demonstrated by LLM outputs, they can in fact make "true" predictions/deductions from their environment of tokens.

They sometimes get it wrong, just like all other conscious entities sometimes get their predictions wrong. There are (often) feedback mechanisms to correct those instances though, in both cases.

Peoples interior model of the world is very tenuously related to reality. We don't have a direct experience of waves, quantum mechanics, the vast majority of the electromagnetic spectrum, etc. The whole thing is a bunch of shortcuts and hacks that allow people to survive, the brain isn't really setup to probe reality and produce true beliefs, and the extent to which our internal models of reality naturally match actual reality is related to how much that mattered to our personal survival before the advent of civilization and writing, etc.

It's really only been a very brief amount of time in human history where we had a deliberate method for trying to probe reality and create true beliefs, and I am fairly sure that if consciousness existed in humanity, it existed before the advent of the scientific method.

I don't think it's brief at all. Animals do this experimentation as well, but clearly in different ways. The scientific method is a formalized version of this idea, but even the first human who cooked meat or used a stick as a weapon had a falsifiable hypothesis, even if it wasn't something they could express or explain. And the consequences of testing the hypothesis were something that affected the way they acted from there on out.

Reply on news.ycombinator.com