Hacker News Reader

Comment by ImHereToVote | Hacker News Reader

So what is it about a biological neuron that sidesteps the halting problem? Is it because there is a weight and bias set to it. Someone should tell OpenAI to add weights and biases to their LLMs. That will make them make have an emotional like intuition, will it not?

EDIT: Maybe if we train the ANNs to focus their training on attention based techniques. Then they will simply tire on halts and continue on other problems until the model has sufficiently grokked lesser problems to focus on the previous halts.

View on HN

Why do you think "biological neuron"s sidestep the halting problem?

People, who happen to run on biological neurons, have a sense of boredom that tries other approaches, and is also willing to eventually "give up", which aren't well captured in the standard algorithmic approaches.

The halting problem simply doesn't apply to biological neurons, they don't side step them, unless your context is humans writing down an Algorithm.

The rules of a human writing down an algorithm is the same thing as a Turing machine running an algorithm.

The halting problem applies for any system of computation that is at least as powerful as a TM, including any type of arithmetic or non-arithmetic calculation that is well-defined, AKA deterministic.

Cortical neuron firing is non-deterministic and more closely is modeled as probabilistic but still stochastic.

https://www.biorxiv.org/content/10.1101/2022.12.03.518978v1

Machine learning is constrained by the halting problem.

HALT is the conical example for what is decidable, but other problems exist and sometimes PAC learnability hits practical limits far before the finite time limits of RE.

As an example not invoking HALT:

https://arxiv.org/abs/2208.10255

There are absolutely constraints on BNNs, but as BNNs aren't deterministic Turing machines, it doesn't apply.

The real question is why do people resort to elementary oversimplified models of biological brains?

If you are in the field of studying the brain you will look for deterministic models that fit your needs to make computation more likely to be tractable.

But the false equivalency of ANNs to BNNs is problematic as a distraction from finding tractable solutions for computation.

Isn't that exactly how attention based neural networks work?

Attention is probably most easily conceptualized as run time reweighting.

A powerful tool but it doesn't really change the underlying model it is just modifying the weights at runtime.

Reply on news.ycombinator.com