Hacker News Reader

Comment by ben_w | Hacker News Reader

Perceptrons are, by design, analogous to organic neurones — only analogies, not fantastic models. Likewise the artificial networks are analogies to organic networks.

It's therefore not surprising that the artificial behaves analogously to the organic, but it would be a mistake to assume they reproduce us accurately: GPT-3 is about a thousand times less complex than a human, while trained on more text than any of us could experience in ten millennia and only text. It has no emotions, unless the capacity to simulate affect happened to lead to this as an emergent property by accident (which I can't rule out but don't actually expect to be true).

View on HN

I find it pretty remarkable how in many visual recognition neural networks (say for the MNIST digits) you see neurons close to the input layer that respond similarly to neurons in the V1 area of the visual cortex.

http://www.lifesci.sussex.ac.uk/home/George_Mather/Linked%20...

It's wavelets all the way down.

There's basically only one way to solve that problem. Are we surprised every time 2+2 is 4?

Did we have any reason to expect that in advance? This was the fist time we tried, in this metaphor, to calculate 2+2 in the abstract.

Sure, they are modeled. One is math, one is atoms.

But look at a single 'real' neuron, it is just calcium ions, electrical potentials, there is no 'emotion'.

Once you can completely model a 'real' neuron (I know there is still some scale to achieving this). Then link together these exactly modeled 'real' neurons. What is to say it is not experiencing 'emotions'. Even though it is silicon.

Humans give themselves too much credit for being special. "I feeeeeel, the computer can't". That is just not understanding yourself.

You clearly lack understanding here too. Simulating "real" neurons at the scale required to simulate a brain is probably np hard. Even if we wanted to try, we don't have any maps of neuron connectivity with nearly the resolution required to do so.

I think the bigger problem is we have no idea what the necessary or sufficient requirements are, neither for qualia nor for intelligence. (Not sure why you think it's np rather than just lots?)

With intelligence we can at least tell when we've achieved it, to whatever standard we like.

Emotions could probably be a thing where we can map some internal state to some emotional affect display, eventually; but what about any question of emotional qualia?

AFAICT, we don't even have a good grasp of the question yet. We each have access only the one inside our own head, and minimal capacity to even share what these are like with each other. When did you learn about aphantasia, for example? Is there a word for equivalents for that for other senses besides vision? I can "visualise" sounds and totally override the direction of down coming from my inner ears, but I can't "visualise" smells, and I don't have a non-visually oriented word for the idea, as both "visualise" and "imagine" clearly are and "idea" itself more subtly also is.

"Qualia" are really just a reframing of (an aspect of) consciousness, which has been speculated to be purely epiphenomenal. Maybe we're just along for the ride, and our actions merely happen to mirror our decisions - or the other way around, same difference.

Qualia is how to discuss the problem of consciousness without a pointless discussion about if this is the opposite of unconscious, the opposite of subconscious, something that needs a soul, or any of the other things that go wrong if you don't taboo the "c" word.

We don't know the answers (nor, I assert, the correct questions), though "what even is this?" and "what has it?" were already relevant to animal rights questions (qualia might have started with humans, but there's no reason to assume that) well before current AI, and even if we find a convenient proof that current AI definitely can't and that's fine… some of us want to advance the AI to at least as far as brain uploading where the qualia is the goal, though virtual hells like in the novel Surface Detail seem to me to be an extremely plausible dystopian outcome if uploads ever actually happen.

Look up the paper on the neuron ability to do XOR logic. It is far less than NP hard.

Do you mean biological neurones or perceptrons? There have been publications about both regarding XOR. If the latter, be aware that this was only about single layers and that perceptrons have an unnecessary restriction on the way they can combine inputs.

No. Sometime last year there was paper measuring inputs and outputs on a real biological neuron and discovered it could do XOR logic.

Can do more logic also, that it could also do XOR was some new breakthrough.

And, not saying Neurons are just logic gates, it's just that they are something well below NP hard.

There are analog fluctuations in potential all along the neuron, and they peak and fire at different thresholds. Really, seems like 'weights' if we want to map terminology.

It does not seem like it will be that long before the human neuron can be 'modeled' accurately enough that quibbling over a few percentage points between the 'model' and 'reality' will not leave much room in-between to find a 'soul' or 'spark', or anything anybody wants to say makes humans special.

> we don't have any maps of neuron connectivity

We do for a at least one classic, small-scale example: C. Elegans. Despite mapping the roughly 300 neurons, the simulation attempts I'm aware of weren't very fruitful.

> with nearly the resolution required to do so.

I agree this may be part of why. Accurate simulation may require replicating subtle behavior outside the neuron body. Further maps or simulation attempts may have since been made, and possibly with better results. Given I don't remember headlines about this, it's likely that any improvements weren't groundbreaking.

I don't know enough about the roles of glia and inter-neuron (not interneuron) behavior to discuss this further beyond wild speculation. Nor does anyone, as far as I know. Gaining that knowledge would probably be necessary to build connectomes with sufficient accuracy for simulation.

That's exactly my point.

> But look at a single 'real' neuron, it is just calcium ions, electrical potentials, there is no 'emotion'.

Indeed; this is why I am willing to entertain the possibility that emotions may be present as an emergent property of simulating us. I don't expect it, but I can't rule it out.

> GPT-3 is about a thousand times less complex than a human

Also note that it takes the 'brute force' approach to architecture by using a transformer model, basically learning a connection graph from scratch. If you want that to scale to human complexity in function, you're probably going to have to overshoot in size by an order of magnitude.

GPT-3 is about a thousand times less complex than a human

How did you figure that? How many synapses are dedicated to text processing in a human brain? How much information do those synapses encode compared to the information encoded in GPT-3? How about GPT-4?

I don't think GP was restricting 'human' to 'text processing elements of a human'.

At any rate, it's certainly true that GPT-3 is missing a great deal. So is GPT-4.

Correct; mine was an overall statement rather than purely about human language processing.

Estimates for how much information the brain encodes are several orders of magnitude higher than the biggest llm, to the point where trying to replicate it is pushing the boundaries of computability. The brain is also significantly more adaptable and generalized thanks to neuroplasticity.

Reply on news.ycombinator.com