Hacker News Reader

Comment by Waterluvian | Hacker News Reader

Humans just do spicy autocomplete too.

Really? I can guess at what spicy autocomplete might actually mean but I doubt a LLM ... OK ChatGPT did a pretty good job of it (I've just asked it), whilst sidestepping the definition of spicy in this context. It is after all a very good next word guesser, given a context, and its not ... me! I am capable of hallucinating but it was 30 odd years ago since I hunted out certain fungi on Dartmoor, or smoked hemp.

To be fair, we humans do often interrupt each other to second guess a sentence completion. Done correctly it is a brief satisfying collaboration. Done wrong ... I've been married for 18 years and know when to bite my tongue, but I still get it wrong from time to time - sometimes deliberately. Despite that, me and the wiff can autocomplete each other's sentences with uncanny accuracy and end up with perfect harmony or a cough slight disagreement as a result.

We are getting some phenomenal slide rules these days but the darleks are not going to be flying up the stairwell just yet, nor will SkyNet be taking over tomorrow.

That said, you just know that some noddy is trying to sell a nuclear "deterrent" LLM AI thingie somewhere. Thankfully, production military equipment takes quite a while to get to deployment. There is a good chance that we will get to grips with all this stuff before SkyNet is let loose for real 8)

No, a human isn't born with a set of knowledge like a freshly trained LLM, keeping the model fixed and responding to input. The analog to the model changes based on the human's experience. Just making bigger and bigger LLMs won't give you this.

Humans are born with millions of years of evolutionary training embedded in their dna and brain. We are not born with nothing.

Yeah but chimpanzees and cats and mice have all the same million-year-old stuff that we do.

The million-year-old stuff is not what makes humans interesting.

I have been thinking about this lately. Things that are recently developed like language skills are easiest to replicate by machines. But things that too took a time to develop like walking and grabbing are still something that machines struggle with.

It's what makes humans possible. "Necessary but not sufficient" is the phrase that pays.

Ah, but I believe you forget the implicit biases of genetic programming. Instincts in my experience are the skeleton, and in a sense the default basis functions for the structure of how we live, see, do, and learn.

No, I don't forget that. There's obviously a starting point, behaviors and abilities that newborns already have. The point is that the model is not static.

So a human is different because it keeps training its neural network?

Whoa, imagine you get a good base LLM model and save all conversations with it. Run a batch process every night to fine tune a LORA on convo dataset. If I ever came across such a chat bot I'd probably freak out as to why it remembers things outside of the context window, without summarisation

That's a pretty neat idea, I would be surprised if no one is already working on that.

I did it years ago on a lark with a seq2seq model in a matrix chat room.

How did it perform? Was it well received by members?

Poorly! It was a small seq2seq and was gibberish to start with. Although it did tell my friend that it loved him which was nice.

one reason why a human is different: just based on word count alone, most LLM's are trained on 3-5 orders of magnitude more input.

could be a difference that makes no difference, or ...

Bit of an unfair comparison when humans also have a bunch of senses that LLMs don't have. They might be trained on orders of magnitude more words, but more data? Doubtful.

That's the key. I'm reminded of the Helen Keller story. She was completely blind and deaf. Her teacher spent a very long time signing into her hand. It took a very long time before she realized that the sign for "water" designated the thing she could feel flowing onto her hand; before that breakthrough the signs were meaningless to her. An LLM only knows the structure of language. It doesn't know that there is an external physical world that the language refers to. It only can predict what words follow which other words, and which output is preferred. Without any senses (and the huge bandwidth of information provided by them) an LLM is very crippled.

Crippled, yes, but I would disagree that it is fundamentally limited, or that an input stream of human language is inadequate to bootstrap "meaning", or in some way philosophically inferior to native biological senses.

It's very interesting you bring up Helen Keller because she's generally regarded as possessing the same level of sentience - and indeed intelligence - as anyone else, despite the extreme narrowness of her sensory input. It took her much longer to get going, but it's not as if she only understood concepts that directly related to touch. The experience with "water" taught her the concept of a symbol, and from there she could bootstrap everything else. LLMs already work with symbols - that is their sense.

In fact we're all a bit like Helen Keller, in the sense that if sensory input is the basis for our entire world model, then it is a very small foundation supporting an incredibly vast and intricate edifice. There is a considerable abstraction gap between concepts like "capitalism" and any direct sensory input. We all of us, all the time, manipulate concepts without thinking through what they "mean" all the way to something we can see and touch.

No they don't. You're "just" doing what everyone else in the past has done with the brain/human intelligence and using the latest technology as a metaphor without realizing it.

We want to think we’re exceptional but all we can do is say “human consciousness is special” without having any way of measuring it or disproving the assertion that we’re just really fancy pattern matchers.

Take any metaphor you want, it’s the same outcome: we may all be philosophical zombies.

We may "just" be neural networks that run on meat instead of silicon, but that does not mean that we're LLMs.

Why doesn’t it?

It's a formal logical error. One does not follow from the other without affirming the consequent.

Because not all neural networks are LLMs.

A GAN is a neural network, does that make it an LLM?

We have inputs other than words, for a start

I’m conscious, maybe you’re not? Not that I really believe that. I think you probably see colors and hear sounds, even in your dreams! But engineering types tend to be persuaded by a particular view of the world, failing to understand that it is a view and not nature itself.

Maybe you do. Luckily, not everyone is quite so simple.

monkeys make monkeys accidentally.

monkeys make meseeks on purpose.

there is a difference, but will it be fun?

I am 100% invested in how much ridiculous fun this era is going to be. Right up until the moment when it becomes a horror.

Reply on news.ycombinator.com