Hacker News Reader

Comment by hooloovoo_zoo | Hacker News Reader

Any artificial neural net has a finite set of inputs and a finite set of outputs. That makes it equivalent to some lookup table. You must therefore claim a lookup table has the ability to reason.

View on HN

Not even going to bother with the weird leap of logic you've made here but Memory augmented LLMs are computationally universal - https://arxiv.org/abs/2301.04589

Then I won’t bother with your citation that says if you add infinite memory you can achieve infinite states :)

infinite memory ? guess you can't read then. Well you do you.

And what, humans are made out of an infinite number of neurons?

One neuron has an infinite number of possible states (as far as we know).

GPT-4 thinks you should provide a citation:

Me: Is it true that, in humans, one neuron has an infinite number of possible states (as far as we know)?

GPT-4: It is not accurate to say that a single neuron in humans has an infinite number of possible states. Neurons are complex cells that can exhibit a wide range of behaviors and responses, but the number of states is not infinite.

Neurons communicate with each other through electrical and chemical signals, and their primary functional unit is the synapse, where communication between neurons occurs. The strength and patterns of these connections can vary, resulting in a large number of potential states for a single neuron. However, this number is still finite, albeit extremely large.

Moreover, the activity of a neuron can be described in terms of its membrane potential, which is a continuous variable ranging from hyperpolarized to depolarized states. While there are many possible values for the membrane potential, it is still a bounded range, not infinite.

So, while the number of possible states for a single neuron is vast and complex, it is not infinite. The complexity and diversity of neuronal states contribute to the rich repertoire of cognitive and behavioral functions that humans possess.

That implies a lookup table requires me to provide a citation :) In fact, assuming space is not discrete, not only can a neuron be in any of infinite states, per quantum mechanics it can be in multiple at the same time.

The lookup table says I should tell you that quantum mechanics applies to subatomic particles, not very large objects like neurons.

(Would it bother you to be corrected by a lookup table?)

Schrödinger's cat will be so relieved!

How does that hold in the case that connections among neurons exchange state information via subatomic particles?

quantum mechanics ? you think that is applicable for neurons ?

Neurons apparently contain microscopic tubes that make use of some quantum effects:

https://phys.org/news/2014-01-discovery-quantum-vibrations-m...

The user was implying neurons could be in multiple states at the same time because...quantum mechanics. Wrong conclusion even from this link.

Since we're providing citations :) https://iopscience.iop.org/article/10.1088/2399-6528/ac94be

That doesn't mean that neurons could be in multiple states at the same time which is what you were implying. That particular assertion is still horribly wrong.

'...the assumed brain function producing entanglement, must use a kind of quantum distillation process on mixed states.'

again, that doesn't mean neurons can exist in multiple states. There is also no also indication higher level computational functions in the brain responsible for reasoning or whatever have you require any quantum computations.

But can you distinguish those states reliably?

We don't know that in any way that is relevant. A biological neuron is complex and made of lots of elements where very fine interactions can plausibly have effect, so in a sense, there is a combinatorially infinite space of neuron-states, but there's no evidence its upper-level computational function that leads to learning and producing adaptive outputs in response to inputs is not discrete. Indeed it seems that most of the complexity [that isn't just "piping" for life functions – don't forget these things aren't semiconductors in metallic sheets fabricated by TSMC, they have to migrate to their sites and branch out on their own, they consume, they excrete, they function for many decades!] is redundant and serves to compensate for various biological shortcomings like low transmission speed, limited bandwidth, noise and stochasticity, developmental abnormalities, cell death etc.

Analog computers also don't have infinite states, because of the noise floor.

A synapse can encode at most some low tens of states[1], synapses are discrete, spiking is discrete, receptor density is discrete, each neurotransmitter release is made up of discrete number of vesicles with a discrete number of molecules, even epigenetic expression is discrete. That it's not all perfectly algorithmically executed is not key to the function being performed – and anyway, at the limits of high-performance compute we also start to deal with esoteric processes, and ways of suppressing them. We didn't design the brain nor do we have perfect ways of observing it in vivo, so we can't always neatly distinguish the wheat and the chaff; but this cannot be seriously taken as a cause to think it's all wheat. Non-human animals show that it's not magic either; even apes, with brains architecturally almost (not wholly) identical to ours but smaller, cannot learn any of the hard symbol-manipulating stuff that we or later GPTs can and that we colloquially refer to as intelligent behavior and reward economically. This, I think, is a very strong prior for that sort of intelligence not being dependent on some low-level expressivity of biological neural computational substrate.

Penrose-Hamerroff's "quantum effects in microtubules" thesis is crank science not supported by any direct evidence.

1: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5247597/

One of the reasons I think LLMs are somehow analogous to brains is that they scale based on size. This seems true for animal brains as well.

But you seem to be suggesting that all animal brains are equally powerful, because they have infinite neuron states, quantum superpositioned neurons, etc.

As far as I can tell, you seem to be suggesting that my old cat’s brain has the same complexity as my brain - right?

So, I’m interested in understanding why my cat is demonstrably stupider than I am, despite having infinite complexity, and why whatevrr you come up with doesn’t also apply to LLM size.

No, I’m saying having a finite number of neurons doesn’t imply a particular number of brain states. Scaling laws actually don’t apply very strictly to brain size. For example, whales have very large brains but aren’t generally thought to be more capable of reasoning than humans. Women generally have smaller brains than men but aren’t thought to be less capable of reasoning.

> No, I’m saying having a finite number of neurons doesn’t imply a particular number of brain states.

But of course it does imply that. It's vast but necessarily finite number of "brain states" however that's defined, which is itself a subset of the vast but also finite number of states of the finite number of component particles.

That clearly isn’t true, since a neuron that could be in infinite states would thereby be capable of storing infinite information, and (as the neuron itself is finite in size) have infinite information density, but there is an established physical upper limit on information density.

https://en.wikipedia.org/wiki/Bekenstein_bound

Mm not sure that bound applies because neurons are neither of fixed size nor fixed energy. But on the other hand one starts to stretch the definition of neuron.

> Mm not sure that bound applies because neurons are neither of fixed size nor fixed energy

Neurons are not of unbounded size nor energy.

What’s the bound?

Can you not work this out yourself? Use some common sense. Do you disagree, are you saying that animal body cells can be infinitely large?

No bound, got it.

Which animal body cells can be infinitely large then? This is foolery. "Neurons are of unbounded size" is a ridiculous statement, we any fool can see it's false. IDK what you're aiming at here, try making a case for something sensible?

We hit comment cooldown but it's not foolery exactly, more like reflecting the futility of trying to make these kinds of arguments. The problem with this whole chain is we haven't actually defined what a neuron is. Take the purported bound above. Suppose I move an associated potassium ion farther away. Have I not changed the state? If yes, what's the limit to how far away I can move that ion and have it still count as part of the neuron? That's just one of the limits of physical laws, you can make very precise statements, but only under very narrow constraints.

Here are some bounds, from most theoretical to most practical:

The size of the earth's biosphere

The size of the largest animal.

The size of a human skull cavity.

The size of a human skull cavity, divided by the number of neurons therein ( a few Billion).

Is this not obvious? There's nothing futile about stating these obvious facts.

Do not fool around with "if I move a potassium ion farther away, isn't that a different bound" ? correct, but moving the bound slightly, is irrelevant to the obvious statement that there are still bounds, and they are practical.

Stating that the neuron is not well-defined is similar irrelevance, if you want to precisely measure the bound, you will need to know precisely what is bounded, but I did not, merely stated the obvious truth that every such definition will come with a bound. One can say "definitely smaller than this" in a wide range of ways, without defining the exact edge.

What's the alternate argument: "I won't define it, therefor it might be larger than the solar system". This is not a serious thing to say.

And to think that such bounded systems "have infinite states would thereby be capable of storing infinite information" is pseudo-mystical foolery. Large, or even "vast", is not infinite. And as sibling comment has mentioned, there is a "noise floor" below which analog resolution means nothing.

Reply on news.ycombinator.com