Hacker News Reader

Comment by daveguy | Hacker News Reader

All this anthropomorphizing of activation networks strikes me as very odd. None of these neurons "want" to do anything. They respond to specific input. Maybe humans are the same, but in the case of artificial neural networks we at least know it's a simple mathematical function. Also, an artificial neuron is nothing like a biological neuron. At the most basic -- artificial neurons don't "fire" except in direct response to inputs. Biological neurons fire because of their internal state, state which is modified by biological signaling chemicals. It's like comparing apples to gorillas.

View on HN

>None of these neurons "want" to do anything. They respond to specific input.

Yes well, your neurons don't "want" to do anything either.

>Maybe humans are the same, but in the case of artificial neural networks we at least know it's a simple mathematical function

So what, magic ? a soul ? If the brain is computing then the substrate is entirely irrelevant. Silicon, biology, pulleys and gears. all can be arranged to make the same or similar computations. If you genuinely believe the latter, it's fine. The point is that "simple" mathematical function is kind of irrelevant. Either the brain computes and any substrate is fine or it doesn't.

>Also, an artificial neuron is nothing like a biological neuron.

They're not the same but "nothing like" is pushing it a lot. They're inspired by biological neurons and the only reason modern NNs aren't closer to their biological counterparts is because they genuinely suffer for it, not because we can't.

>Biological neurons fire because of their internal state, state which is modified by biological signaling chemicals

Brains aren't breaking break causality. The fire because of input.

Comments like this are incredibly grating. You condescend to the interlocutor for making a mistake which only exists in your own mistaken world model. Your confidence that neurons and ANN weights and «pulleys and gears» are all equivalent because there is, in theory, an intention to instantiate some computation, and to think otherwise is tantamount to belief in magic and broken causality, is just confused and born out of perusing popular-scientific materials instead of relying on scientific literature or hands-on experience.

> The fire because of input.

No they do not fire because of input, they modulate their firing probability based on input, and there are different modalities of input with different effects. Neurons are self-contained biological units (descended, let me remind you, from standalone unicellular organisms, just like the rest of our cells), which actually have an independently developing internal state and even metabolic needs; they are not merely a system of logic gates even if you can approximate their role with a system of equations or an ANN. This is very different, mechanistically and teleologically. Hell, even spiking ANNs would be substantially different from currently dominant models.

> So what, magic ? a soul ? If the brain is computing then the substrate is entirely irrelevant

Stop dumbing down complex arguments to some low-status culture war opinion you find it easy to dunk on.

>Your confidence that neurons and ANN weights and «pulleys and gears» are all equivalent because there is, in theory, an intention to instantiate some computation, and to think otherwise is tantamount to belief in magic and broken causality, is just confused and born out of perusing popular-scientific materials instead of relying on scientific literature or hands-on experience.

Computation is substrate independent. I'm not saying neurons and ANN weights and «pulleys and gears» are the same. I'm saying it does not matter because what you perform computation with does not change the results of the computation. If the brain computes, then it doesn't matter what is doing the computation.

>No they do not fire because of input, they modulate their firing probability based on input, and there are different modalities of input with different effects. Neurons are self-contained biological units (descended, let me remind you, from standalone unicellular organisms, just like the rest of our cells), which actually have an independently developing internal state and even metabolic needs; they are not merely a system of logic gates even if you can approximate their role with a system of equations or an ANN. This is very different, mechanistically and teleologically. Hell, even spiking ANNs would be substantially different from currently dominant models.

Yes, a neuron is firing because of input. To suggest otherwise is to suggest something beyond cause and effect directing the workings of the brain. If that is genuinely not the case then feel free to explain why, rather than an ad hominin attack on someone you don't even know.

> So what, magic ? a soul ? If the brain is computing then the substrate is entirely irrelevant

>Stop dumbing down complex arguments to some low-status culture war opinion you find it easy to dunk on.

I personally don't care if that's what anyone believes. The intention is not to attack anyone.

If you believe in a soul or the non religious equivalent, that's fine. We just have different axioms.

If you don't believe in a soul(or the equivalent) but somehow think substrate matters then you need to explain why because it makes no sense.

I am not well versed in any of this, but from reading the counterarguments, I think two good points are being made:

* Analogies aside, neurons are quite different than NN nodes, because each neuron has an incredibly complex internal cellular state, whereas an NN node just has an integer for state.

* A brain is not a "function" in the way that a trained LLM model is. Human life is not a series of input prompts and output prompts. Rather, we experience a fluid stream of stimuli, which our brain multiplexes and reacts to in a variety of ways (speaking, moving, storing memories, moving our pupils, releasing hormones, etc). That is NOT TO SAY a brain violates causality; it's saying that the brain is mechanically doing so much more than an LLM, even if the LLM is better at raw computation.

None of this IMO precludes AGI from happening in the medium term future, but I do think we should be careful when making comparisons between AGI and the human brains.

Rather than comparing "apples to gorillas", I'd say it's like comparing a calculator to a tree. Yes, the calculator is SIGNIFICANTLY better at multiplication, but that doesn't make it "smarter" than a tree, whatever that means.

I do not even think any of this has much of impact on AGI timelines. Human brain cells are not a superior substrate for computing "intelligence". They just are what they are; individual cells can somewhat meaningfully "want" stuff and be quasi-agents unto themselves, they do much more than integrate and fire. Weights in an ANN are purely terms in an equation without any inner process or content.

> Computation is substrate independent. I'm not saying neurons and ANN weights and «pulleys and gears» are the same. I'm saying it does not matter because what you perform computation with does not change the results of the computation. If the brain computes, then it doesn't matter what is doing the computation.

It's a tautology. If the substrate did change the computation, then it wouldn't be the computation.

Claims where it isn't possible for you to be incorrect may be less impressive than they seem.

There are views where there is an implicit substrate that exists in another layer of reality, ie. dualism. That layer is generally not counted among the substrate. So a computation can be substrate dependent by introducing a non-material cause. (Disclaimer: I don't personally know anyone who believes anything like this, so this may be a bad paraphrasing.)

If the computation doesn't compute completely and deterministically on all substrates, then it isn't substrate independent though is it?

Human cognition would be a good example of substrate dependent computation I'd think....it even varies per instance of substrate.

It's not a tautology. That you can go out tomorrow and buy a deluge of computers with different hardware and run the same software without change is exactly a demonstration of substrate independence.

https://www.edge.org/response-detail/27126

And if one of those computers failed, it wouldn't classify as a (proper) computer.

You can move your pointer anywhere you'd like, it is ultimately tautological. Infinite regress is a bitch lol

Say, have you taken into consideration the role consciousness and culture are playing here? Like this "reality" you are describing, do you know what the actual, biological/scientific source of it is? :) But now I'm kind of cheating, aren't I...I think we're not supposed to say that part out loud! ;)

> Stop dumbing down complex arguments

It's not dumbing down. It's extracting the crux of the matter that the complexity of arguments is trying to hide, perhaps unintentionally. Either the brain implements a function that can be approximated by a neural network thanks to universal approximation theorem, or the function cannot be approximated (you need arguments for why it is the case), or magic.

This is technically true but kind of misses the point in my opinion. A neural network can approximate any function in theory but that doesn't mean it has to do so in a reasonable amount of time and with a reasonable amount of resources. For example, take the function that gives you the prime factors of an integer. It is theoretically possible for a neural network to approximate this for an arbitrarily large fixed window but is provably infeasible to compute on current hardware. In theory, a quantum computer could compute this much faster.

This is not to say that the human brain leverages quantum effects. It's just a well known example where the hardware and a specific algorithm can be shown to matter.

I also think it's strange to describe the brain as implementing a function. Functions don't exist. We made them up to help us think about building useful circuits (among other things). In this scenario, we would be implementing functions to help us simulate what is going on in brains.

> is just confused and born out of perusing popular-scientific materials instead of relying on scientific literature or hands-on experience.

I suspect there's some fundamental metaphysical framework protection in play here, the sort of language being used is pretty common, I believe it to be a learned cultural behavior (from consuming similar arguments).

I think you're being a bit pedantic here.

Neurons also don't "respond" to specific input either. They can't speak or provide an answer to your input.

These are all just abstract metaphors and analogies. Literally everything in computer science at some point or another is an abstract metaphor or analogy.

When you look up the definition and etymology of "input", it says to "put on" or "impose" or "feed data into the machine". We're not literally feeding the machine data, it doesn't eat the data and subsist on it.

You could go on and on and nitpick every single one of these, and I don't think the use of "want" (i.e. anthropomorphizing the networks to have intent) is all that bad.

Wait what are you referring to specifically? Any anthropomorphism in the article is _clearly_, clearly the author's admitted simplification due to the incredible density of the subject matter.

Given that, I honestly can't find anything too upsetting.

In any case, anthropomorphism is something I don't mind, mostly. Is it misleading? For the layman. But the domain is one of modeling intelligence itself and there are many instances where an existing definition simply makes sense. This happens in lots of fields and causes similar amounts of frustration in those fields. So it goes.

> Humans also use neural nets to reason about concepts. We have a lot of neurons, but so does GPT-4.

I feel this is an abuse of the language. Biological neurons and ANN neurons aren’t the same or even all that similar. Brains don’t do backprop for example. Only forward passes. There’s a zoo of neurotransmitters which change the behavior of individual neurons or regions in the brain. Unused neurons in the brain can be repurposed for other things (for example if your arm is amputated).

>Biological neurons and ANN neurons aren’t the same or even all that similar.

They're not the same but they're definitely similar.

>Brains don’t do backprop for example.

We've developed numerous different learning algorithms that are biologically plausible, but they all kinda work like backpropagation but worse, so we stuck with backpropagation. We've made more complicated neurons that better resemble biological neurons, but it is faster and works better if you just add extra simple neurons, so we do that instead. Spiking neural networks have connection patterns more similar to what you see in the brain, but they learn slower and are tougher to work with than regular layered neural networks, so we use layered neural networks instead.

The only reason modern NNs aren't closer to their biological counterparts is because they genuinely suffer for it.

The secret of bird flight was wings. Not feathers. Not flapping.

> The only reason modern NNs aren't closer to their biological counterparts is because they genuinely suffer for it.

And yet, there's no ANN that's as good at interacting with the real world as the simplest worms we've studied, despite having many times more neurons than those worms have cells.

We are clearly still missing some key pieces of the puzzle for intelligence, so claiming that the difference between ANNs and biological neurons is irrelevant is quite premature. We are far away from having an airfoil moment in AI research.

Simple organisms are not necessarily easier to imitate than more complex ones. Our vehicles can drive faster than the fastest animals, but no car can absorb nutrients across a cell membrane. The simpler a system is, the more an imitation has to stick to the specifics to the system's particular implementation. Larger, more complex systems can usually perform more abstract tasks, which allows more diverse approaches.

I assert that most robotics AI, including self driving cars, are at least as good at interacting with the real world as the simplest worms we've studied. Do different things, but at least as good.

My point isn’t about whether ANNs work, it’s that they’re so fundamentally different (both locally at the level of the neuron and globally at the level of the brain) that calling them both “neurons” is pretty imprecise.

Silicon simulations of brains may “suffer” from being faithful but this also discounts the advantages that brains have. As I mentioned for example, brains can repurpose neurons for other tasks. Brains can also generalize from a single example, unlike neural networks which require thousands if not millions of examples.

Brains also generally do not suffer from catastrophic forgetting in the same way that our simulations tend to. If I ask you to study a textbook on cats you won’t suddenly forget the difference between cats and dogs.

>Brains can also generalize from a single example, unlike neural networks which require thousands if not millions of examples.

There is not a single brain on earth that is the blank slate a typical ANN is. "Brains generalize from one example" is pretty dubious. Millions of years of evolution matter.

>As I mentioned for example, brains can repurpose neurons for other tasks.

Isn't this just a matter of the practical distinction between training and inference and not some fundamental structural limitation ?

>Brains also generally do not suffer from catastrophic forgetting in the same way that our simulations tend to.

This suggests CF may well be a simple matter of scale - https://palm-e.github.io/

Since individual anns are much closer to synapses, we don't have anything near the scale of the brain yet.

> There is not a single brain on earth that is the blank slate a typical ANN is.

Of course structure matters, but biological neurons have far more degrees of freedom than those in ANNs. The fact that we even need to keep differentiating between the two is an indication that classifying both as “neurons” is not accurate.

> Isn't this just a matter of the practical distinction between training and inference and not some fundamental structural limitation?

It’s a difference in capabilities of the things themselves. A biological neuron organically seeks out new connections. Sure we could program that into an ANN somehow but the fact that nodes in an ANN don’t have this capability out of the box is a fundamental difference.

> CF may well be a simple matter of scale

For a moment, a big enough network might be able to mirror an entire brain with the lottery ticket hypothesis. But if it takes two or ten or a thousand ANN neurons to simulate the degrees of freedom of a biological neuron, are they really the same?

>There is not a single brain on earth that is the blank slate a typical ANN is.

You are the one saying that biological neurons and ANN are similar...

> Brains can also generalize from a single example, unlike neural networks which require thousands if not millions of examples.

Since the other comment already went for the evolution and structure angle, I'll go for the other part. What single example? What test have you seen done on the brain capacity of few weeks old fetuses? Our brains start learning patterns in the world before we are even born. How much input does a baby receive every single second from it's eyes and ears and every other sense?

Even when you are "analyzing" a new object for the first time, you receive a continuous stream of sensory input of it. Our brain even requires that to work, if you put a single frame different in a fast enough display, most times you won't even notice the extra frame and your brain will just ignore it.

I feel anthropomorphizing is perfectly reasonable especially in this context. How would you like it described?

There’s so much baggage attached with it. The only thing a neural network “wants” is to minimize its loss function. That’s all.

Can we anthropomorphize the loss function then?

That's like anthropomorphizing utility functions in place of the humans that (are conjectured to) have them, isn't it?

You must not believe in human free will then. Or maybe you don’t believe the brain is the key aspect of human intelligence. Or believe the brain is more than just a organic NN

I’m saying that neural networks are so different from brains that anthropomorphizing them runs a real risk of glossing over important differences.

Terms like “free will” and “intelligence” are too fuzzy to talk about precisely unless we’re on the exact same page regarding what we mean. And applying our imprecise definitions to machines is not doing us any favors.

The neural network doesn't want to minimize its loss function any more than you want to maximize your inclusive genetic fitness.

The neural network training process wants to minimize the neural network's loss function. The neural network, if it "wants" anything, will have such wants as were embedded in its weights through the process of minimize its loss function, which will mostly be "wants" whose satisfactions correlate with reduced loss function. Of course this is using the term "want" in a behaviourally-descriptive sense, not a subjective-experience sense.

For example, AlphaGo's training routine wants to minimize AlphaGo's loss function. AlphaGo wants to beat you at Go.

See also: Reward is not the optimization target https://www.greaterwrong.com/posts/pdaGN6pQyQarFHXF4/reward-...

All humans “want” to do is obey the laws of physics.

Anthropomorphizing is valid in relation with laguage, human language. Other than that anthropomorphizing would be somewhat valid with other human outputs but ideally it is understood it’s just one of the lenses in the toolkit, that it has pros and cons just like any idea or any tool.

It's one thing to deliberately anthropomorphize in order to construct an analogy.

It's another thing to anthropomorphize by accident or by illusion, as per pareidolia: https://en.wikipedia.org/wiki/Pareidolia . Just as in pareidolia, where the human brain is primed to "see" a human face in a certain pattern of light and shapes, it seems that human brains are primed to "see" a human intelligence in the output of an LLM, because our brains are pattern-matching on "things that look like human speech". But that's a reason to not anthropomorphize LLMs, precisely because people are inclined to do so without thinking.

I didn’t hear much anthropomorphism outside of the subtitle.

Anyways, this is Scott’s writing style. I recall an earlier ACX post on alignment that was real heavy on ascribing desires and goals to AI models.

Do you mean that artificial neurons are inherently passive while biological neurons are inherently active i.e. they would act in spite of external input?

Just wondering if I understood you, I don't know anything on the subject.

That was already addressed in the comment you're responding to. They wrote that in the case of artificial neural networks "at least we know [there is no god inside the machine]". With regards to humans, we don't know.

That wasn't clear to me because having internal state doesn't mean that any process is running. That's the difference between memory and CPU.

There are a great many possibilities between:

- a fully deterministic machine (even if the interface and the way OpenAI let people access ChatGPT make it seem like it's non-deterministic, there are fully deterministic models out there, who not only only respond to inputs but also always respond the exact same thing given the same inputs [query, seed, ...]),

and:

- god exists

There could be chaos at work when human thinks. There may be interferences at play, say because whatever element that traveled trillion of kilometers just traversed our brain.

While a fully deterministic machine that always respond to the same input in the same way is just that: a deterministic machine.

P.S: I don't know about other LLMs like Falcon 180b but image-generation models like StableDiffusion are fully deterministic. I think a model is using a broken design and shall quickly hit limitations if it cannot be queried in a deterministic way (and its usecases are certainly limited if repeatability is not achievable). If you want different answers, use a different seed or a different query. But the same query+seed should always give the exact same output.

Also not all notions of God are dualistic — where you get to (notionally) talk to the guy using “you” and “I”. India’s Advaita Vedanta, Ibn Arabi’s version of Sufism and even some sects of Hasidism all hold that everything is in God. I haven’t found the Christianity that does this, but if it was discovered under Islam is probably thinkable here.

Are these different ideas of God entirely? Yes: in India there are however many gods and Brahman says these are lesser precisely because they don’t include the whole universe.

For the record LLM are theoretically fully deterministic, but non-deterministic in practice. First, some randomness is deliberately set via 'temperature', and second some randomness comes from things like the order of floating point operations when you divide the model on your GPU(s) without being super careful about it.

LLMs output probability distribution of the next token. And that automatically makes them non-deterministic. You can make their output deterministic by greedy sampling, fixing the seed of a pseudorandom generator, or by computing exponentially growing probability distribution of all possible continuations, but it doesn't change the fact that LLMs produce probability distributions you need to sample somehow to get a definite result.

I'd say 'in spite'-ness is pretty spot on for life in general

Can you be more specific about what particular anthropomorphizing you object to? The only place the author uses the word want is in describing the wants of humans.

Wait till you find out how we talk about evolutionary adaptation... a couple analogies and elided concepts here and there and you'd swear Lamarck had smothered Darwin in his cradle

Yes, Lamarck is alive and well. I frequently hear colleagues in biology with years of experience making lazy statements such as, "Humans evolved to walk because..." as if evolution is a conscious act. In many cases, these people are using the term evolution as shorthand for, "Random mutations gave rise to new gene variants (alleles), and over numerous generations, natural selection leads to an increase in the prevalence of beneficial variants, specifically those that provide a reproductive advantage. The genetic variants that combined to enable bipedal locomotion provided such an advantage to the human population". Evolution is a passive, natural process driven by genetic variation, environmental changes, and the differential reproductive success of individuals with advantageous traits. This in turn results in population changes. Evolution doesn't involve a conscious choice or direction. Evolution is the result of the cumulative effects of these factors over long periods of time.

While I generally agree with you, is it really "lazy" to say "humans evolved to walk to"? Usually in the context, it's not being used to claim there is some sort of intent or purpose, but rather to shortcut the (rather verbose) description you gave.

Also I can think of some counterpoints to yours: the people who bred teosinte into corn (or any wild grain into a domesticated one) appear to be making conscious choices or direction- that is, they used their intelligence and reasoning from observed examples of pairings to conclude that they could make improved specimens based on selective breeding (without knowing about random mutations of natural selection!).

And if we start to modify human germline then would also be an example of evolution with conscious choice or direction (assuming the modifications became fixed in the population).

Even in math, the most exact of all fields of study, people in practice are being nonrigorous and ambiguous and take shortcuts all the time. That’s just how communication works, because it would be incredibly grating and inefficient to try to be 100% unambiguous when everybody already knows what you mean.

Social, memetic evolution is at least partially Lamarckian.

All this anthropomorphizing of humans strikes me as very odd. None of these humans "want" to do anything. They respond to specific input. Maybe artificial neural networks are the same, but in the case of humans we at least know it's a simple reaction to neurotransmitter signals.

You're probably just being tongue in cheek here, but I still wanted to add that we humans do actually "want". Desires, intentions, these are real things. You can try to represent them as loss functions or rewards in some mathematical model but that isn't the original thing. Consider this excerpt from https://scottaaronson.blog/?p=7094#comment-1947377

> Most animals are goal-directed, intentional, sensory-motor agents who grow interior representations of their environments during their lifetime which enables them to successfully navigate their environments. They are responsive to reasons their environments affords for action, because they can reason from their desires and beliefs towards actions.

In addition, animals like people, have complex representational abilities where we can reify the sensory-motor “concepts” which we develop as “abstract concepts” and give them symbolic representations which can then be communicated. We communicate because we have the capacity to form such representations, translate them symbolically, and use those symbols “on the right occasions” when we have the relevant mental states.

(Discrete mathematicians seem to have imparted a magical property to these symbols that *in them* is everything… no, when I use words its to represent my interior states… the words are symptoms, their patterns are coincidental and useful, but not where anything important lies).

In other words, we say “I like ice-cream” because: we are able to like things (desire, preference), we have tasted ice-cream, we have reflected on our preferences (via a capacity for self-modelling and self-directed emotional awareness), and so on. And when we say, “I like ice-cream” it’s *because* all of those things come together in radically complex ways to actually put us in a position to speak truthfully about ourselves. We really do like ice-cream.

> but I still wanted to add that we humans do actually "want".

I would also like to add that the subject of conversation is artificial and natural neurons, which humans, though contain some, are not.

If a NN is trained to do something, it can be equally considered as "wanting" to do that thing within the autonomy it is afforded, as much as any human.

> If a NN is trained to do something, it can be equally considered as "wanting" to do that thing within the autonomy it is afforded, as much as any human.

Human wants are driven by instinct though - i.e. our preferences; if you like women, you like women, if you don’t, you don’t.

Our “output” is in service to those wants.

Current AI doesn’t have instinct / preprogrammed goals - except for goal-driven AIs but the hyped up LLMs aren’t such AIs. Their output isn’t motivated by any goal - a LLM can’t deliberately lie to you to get you to do something; it lies because it doesn’t differentiate between what’s true and what’s false.

> Human wants are driven by instinct though

What is instinct physically?

> a LLM can’t deliberately lie to you to get you to do something; it lies because it doesn’t differentiate between what’s true and what’s false.

A LLM also can't do multi-step reasoning, yet here we are.

> What is instinct physically?

Does it matter? As long as it conceptually exist that’s all that matters.

LLMs don’t seek any goal. It’s advanced autocomplete.

You can’t give it a bunch of facts and a goal then expect it to figure out how to achieve said goal. It can give you an answer if it already has it (or something similar) in its training set - in the latter case, it’s like a student who didn’t study for the exam and tries to guess the right answer with “heuristics”.

> A LLM also can't do multi-step reasoning, yet here we are.

Where’s “here”?

> Does it matter? As long as it conceptually exist that’s all that matters.

It does matter. Where is it? By evading this you have basically described "instinct" as something computers simply cannot have, axiomatically. That's boring.

> LLMs don’t seek any goal. It’s advanced autocomplete.

These two sentences are contradictory.

> You can’t give it a bunch of facts and a goal then expect it to figure out how to achieve said goal.

Have you used a language model lately? It sounds like you're saying things you think a LLM shouldn't be able to do as if they can't do them. Giving it a bunch of facts and a goal and expecting it to figure out how to achieve the goal is something you can do. It's not perfect, but they can be surprisingly good.

> Where’s “here”?

At a point in time where LLMs can do multi-step reasoning.

Just because a model converges towards some behavior doesn’t mean it “wants” like a human does. You can call it that in your framework but it is overly reductive to apply it to humans and animals imo

If my grandmother had wheels, she'd be a bike.

Fish don't like ice cream and we don't feel the need to spawn. It's because of how we are built.

I’m sorry what? i don’t understand how your first sentence applies to what I’m saying, and many humans feel a need to spawn.

I'm using the word literally, as in swim upstream and lay eggs.

This is the definition of anthropomorphism: “Anthropomorphism is the attribution of human traits, emotions, or intentions to non-human entities.” By definition the point you’re making doesn’t make sense.

> None of these humans "want" to do anything. They respond to specific input

Weird, because I'm pretty sure I had the choice whether to respond to this comment.

It wasn't the light waves hitting my retina from an HN post, leading to nerves firing and neurotransmitters all coming together to post this.

I posted it because I have free will. I almost didn't.

Unless you truly feel that there is no free will, and that reality is just a bizarre movie we have to experience .... well, then ... we'll disagree.

At some point after the light waves hit your retina, did one or more of the atoms in your body not obey the exact same physical laws as the atoms outside your body?

Reply on news.ycombinator.com