Hacker News Reader

> Is it possible even in principle for a conscious entity to have an instinctual understanding and awareness of every low-level detail comprising its design?

Access is possible, complexity limits what you can do with it.

It's like having a JTAG port on a chip with a lot of test access. You can read the state of most of the gates, but now you're overwhelmed with data. Processing that data takes more resources than the system being monitored.

At a larger scale, though, this starts to work. Modern industrial plants and cars have sensors all over the place, more than are really needed to operate the system. They're there to diagnose it.

Kinda like we have sensors all over the place, but the info a lot of them deliver (pain) is pretty vague. And because of how it gets aggregated and delivered to the brain, sometimes you have funny glitches like how I can scratch the inside of my left ear and it makes me cough.

https://www.icd10data.com/ICD10CM/Codes/K00-K95

Evolution really looked at this big long list of gastrointestinal diseases we as humans can suffer from, and thought that traumatizing amounts of random nausea, constipation, diarrhea, and tummy hurt would be enough for us to figure out when we're supposed to be eating more nuts and less berries, or whatever the hell. I hate it.

This is what hit the NSA.

I guess evolution beats at the pace of signal to noise ratio

Or, in a more modern sense, it would be easy to give an LLM access to its own weights. But it would not be useful.

Speaking as a horse, I wonder why you think we have any ideas at all. But anyway, ideas are less complex than the systems that they explain, that's the whole point of ideas. We're always progressing to simpler explanations while at the same time discovering that the world has new complexities. (The simplification is winning.)

I'd go into more detail, but I've seen a white gatepost and have to stop now.

Many of the systems that complex biological systems rely on, evolved long before the brain, some of them go back to before there were nervous systems at all. Truth is that it's all a hodge-podge of layers of complexity built on-top and sideways. In tech terms we would call it enormous technical debt.

We're all vibe coded :(

The test suite did the heavy lifting.

L&LoL

Imagine what it's like to debug this biological mess that God vibecoded out of boredom.

https://news.ycombinator.com/item?id=37029912

I’ve thought about this. Or at least something like that.

When reading dune there’s a class of women who can control their biochemistry in such a way that they can eg prolong lifespan, or anything else within the realm of feasible biochemistry.

Here I was like: how does the nervous system function in that way? How could signals from the mind control the behavior of molecules?

The mind cannot itself feel pain, similarly the mind has sensory limits within the body. As argued in a book called the body a guide for occupants: cancer is a very good example of a disease that our nervous system should be able to detect but for whatever reason we don’t, probably because cancer is something we can only now treat.

Not all of the body can be made legible to the mind.

Although maybe a better question: why don’t we have a dedicated organ that can sample blood with laboratory like precision and make anomalies available to the conscious mind beyond whatever faculties we currently have?

PS

One thing that definitely should be within the realm of conscious control: body fat. There are ways of forcing the body to metabolize more fat for energy and the biggest problem is managing excess heart (easily becomes lethal). But this could be super useful in cold climates. Imagine being able to literally burn body fat to stay warm? The amount of heat that can be released is enormous. Nowadays most of us could probably afford the otherwise superfluous expenditure of body fat (beyond essential functions).

> why don’t we have a dedicated organ that can sample blood with laboratory like precision and make anomalies available to the conscious mind beyond whatever faculties we currently have?

"My triglycerides are at 220 mg/dL" is not a useful signal to a hominid banging rocks together.

Instead, what we got was the budget version, keyed for resolution rather than understanding: thirst (osmolality of plasma fluid), suffocation (blood acidity induced by excess CO2), and when all else fails, at least vague malaise as a catch-all "something's off, let's maybe sit down."

> One thing that definitely should be within the realm of conscious control: body fat. There are ways of forcing the body to metabolize fat for energy and the biggest problem is managing excess heart (easily becomes lethal). But this could be super useful in cold climates.

Non-shivering thermogenesis is primarily mediated by brown fat. It's how babies keep warm. Adults still retain some brown fat, and it appears spending time in the cold can stimulate its production. https://stagetestdomain3.nih.gov/news-events/nih-research-ma...

> Imagine being able to literally burn body fat for heat? Nowadays most of us could probably afford the otherwise superfluous expenditure of body fat.

Brown fat is burned in the body by a protein called thermogenin (UCP1). The same process can be stimulated by the drug 2,4-Dinitrophenol (DNP), albeit to lethal consequences if you're not careful. I suppose the reason we can't do this at will is because our body can do it unconsciously for us better and more safely than our frontal cortex ever could.

> suffocation (blood acidity induced by excess CO2)

Which, being the budget option, has a failure scenario: When low oxygen occurs without high CO2, people feel fine... until it's too late and they fall over, unconscious.

This can occur in enclosed spaces like caves where some other gas-mix has excluded the oxygen, or when someone hyperventilates before holding their breath.

> This chemoreceptor patent-proposal is kicking my ass. Hundley won't let me down until it's done. Hardly worth filing for, in my opinion. Who wants to smell the difference between xenon and radon?

-- Complaining Versalife employee, Deus Ex (2000)

> This can occur in enclosed spaces like caves where some other gas-mix has excluded the oxygen

Maybe this why some people innate fear of caves, primordial adaptation for suffocation avoidance behavior. I don't know, just fun to wonder about things like that

I would wager a much-bigger feedback loop involves the other things that would like to live in those caves, and the odds you might get into conflict with their jaws/claws/stingers.

The brain doesn't directly control a whole lot of basic functions, other then breathing. It is entirely possible to keep a brain-dead person alive using little more than a feeding tube and forcing air into the lungs.

> Although maybe a better question: why don’t we have a dedicated organ that can sample blood with laboratory like precision and make anomalies available to the conscious mind beyond whatever faculties we currently have?

Because random mutations and selective pressure has never lead to such a trait that persists in a population.

We've never seen randon mutation and selective pressure ever really build anything, only trigger and exploit existing adaptive rulesets... pushed to extremes mutations may often yield a benefit in a narrow situation but at some greater cost, like throwing the backseats out of your car helps you accelerate faster, but reduces the overall utility and flexibility of the use of the vehicle. Im sure if we started honestly looking at organic systems as the product of thought, it would yield a greater understanding of them that we would be able to leverage in industry.

There is a Sci-Fi novel about fancy forest frogs, which possess that ability down to the biomolecular level. And a human researcher stranded on their world, being modifyied by them, to help her survive and adapt to the toxic environment.

https://en.wikipedia.org/wiki/The_Color_of_Distance

(QVACK! °oO0°·.)

I think considering yourself as a consciousness inhabiting a body, maybe in a way similar to the aliens in The War of the Worlds, is a false premise. Consciousness is one control system, but there are many, and your mind relies not just on the brain but the entire body.

So when you say being aware of every detail of your design, that hinges on awareness in the realm of critical thinking - but all the while, parts of your body are absolutely aware and in control of every little detail. I'd even say, these "subsystems" send a notification to your consciousness if anything is outside normal operational parameters, which makes total sense: Surfacing only details that require immediate attention is a good strategy to control complex systems, and one that we apply to many superficial constructions as well.

Another fun aspect of this is that when you try to suffocate yourself by holding your breath for too long, you'll pass out. Why does that happen? Because your body switches off your rogue consciousness in an attempt to stay alive - because staying alive overrules your subjective experience.

It's difficult to say anything about experiential/first-person consciousness as we don't know what it is. Some say it doesn't exist and everybody is a p-zombie. That aside...

As you mention, there's a problem where if you want to know yourself fully - including observability of your mind - you need to be aware of the thing that is aware, and aware of the thing that is aware of the thing that is aware, etc, down to infinite regress.

But that doesn't necessarily mean that it's impossible. It would suggest that such a being works by moving along a set of fixed point. Suppose x is the state of the physical mind (whether that's a brain, a set of chips, or something else). Suppose f(x) returns the updated state after this mind takes into account what it knows (i.e. f(first-order aware) = first- and second-order aware). Then if the mind-state always changes so that x is a fixed point of f, then the mind already knows all there is to know about itself.

So the question, on the mind side, is whether there exists a sufficiently generally intelligent type of mind for which moving along fixed points is computationally easy. I don't know. It could be possible, it could be impossible. Finding fixed points is hard in general, though.

There's also the problem of sensors, which may seem simpler but hides a similar difficulty. What happens if whatever thing that measures blood sugar fails? Or if the immune system goes auto-immune due to an immune cell defect that makes it report benign cells as foreign? Then you need an integrity sensor for the sensors - then you need an integrity sensor for the integrity sensor... At least this fixed-point problem for sensors should be easier than the one for general minds.

The consciousness:complexity ratio would probably express itself in that a mind of a certain complexity with the fixed-point-of-f constraint would be less intelligent on average than a mind of the same complexity without such a constraint.

It makes me tangentially think of the speech in Virtue's Last Reward about how it takes a higher intelligence to understand the intricate structures that termites build, ending with a question of what it is that humans are building to be understood by a higher intelligence.

Humans are kept alive by quadrillions of closed control loops. Being aware of them all would be very distracting.

If you try to understand by knowing every little detail it is bound to fail because that attempt only increases the number of things you need to understand.

I think the Buddhist had it right in this instance, if you want to understand yourself completely stop trying to know every little thing and just be aware.

Not strictly possible per https://en.wikipedia.org/wiki/G%C3%B6del%27s_incompleteness_...

> "A horse is an extremely sophisticated biological machine, but it has no idea what makes itself tick."

To my ears, a lot of detail is hidden in "knowing what makes itself tick". The limit of human Umwelt, cognitive closure, is an arena I find fascinating.

Modern people tend to identify understanding with mechanistic explanations. Veins, glycogen, immune states, hoof pumps. These are interesting and satisfying to us. But does that kind of explanation satisfy only because we are apes with ape language and interests, leaving aside even that we live in a society that values those explanations?

If a horse-level intelligence existed, perhaps its most satisfying explanations would not be "I can inspect the venous plexus in my hoof," but something else. Maybe more proprioceptive: pressure, load, gait. Or maybe that's just still my primate prejudice poking through.

What would satisfy a horse scientists might seem opaque or insufficient to us, but maybe our biochemical account would seem equally beside the point to it?

Maybe there is no species-neutral answer to what counts as understanding what one is.

On a deeper level, I suspect that a "complete understanding" in an absolute or divine sense is infinite. Our perceptions are necessarily limited, only a model of reality. In order to understand in a way that helps us survive, we must filter out a lot of information that is not absolutely irrelevant, but is irrelevant to us. Even Funes remembered only that which is relevant to a human, that which he could perceive in the first place. He could not remember changes in magnetic fields, background radiation, subtle shifts in gravity.

Indeed, specification of what "understanding" even is is one of the hard problems that would have to be solved before answering such a question! Nobody disagrees, for instance, that an LLM has no understanding, awareness, or even access to the underlying numerical weights of its neural network, but still there is much debate whether it understands anything at all, even subjects that it is clearly superhuman at performing in. Perhaps "understanding" will only have a meaningful definition to us if it is tied with conscious awareness, but that's hardly better.

Another very relevant Borges story is "On Exactitude in Science." If we could precisely model every physical law and even the precise distribution of matter in the universe, but this model conveyed to us no intuition or usefulness on a human level, can it even be called understanding? Instead we seek teleological anthropomorphic explanations in everything, and it's baked into how we discuss things. "Big cats developed powerful muscles to run fast and catch prey," "our immune system exists to keep us from getting sick," "rooks are worth more than knights," "the universe favors states with the highest possible entropy," "de Rham cohomology detects topological holes."

> naturally its complexity would have to scale with the complexity it is attempting to self-model...

And yet, in order to understand how a recursive function work, one only have to understand the base case, and the relation between the function's input and output, and not everything that it does in its recursive steps.

So the point is, if an organism/consciousness is a recursive algorithm, it is feasible that it could be understood by itself.