Hacker News Reader

Comment by simonkafan | Hacker News Reader

> that it’s possible for a model to someday reach human levels of generalization in NLP

Fully disagree. There is no evidence that we are now closer to human level text understanding than before GPT-3. Yes, GPT-3 produces grammatically correct sentences but it still can't form a coherent idea or meaning and express it in sentences afterwards - that's what humans would do. GPT-3 is just better at obfuscating that the model has no clue what it's talking about.

Nevertheless, compared with Eliza or other bots from 1960-2000 we made remarkable progress.

View on HN

> Yes, GPT-3 produces grammatically correct sentences but it still can't form a coherent idea or meaning and express it in sentences afterwards - that's what humans would do.

There's considerable debate over whether humans can have a coherent idea before it is reduced into symbolic language, and it's not clear how you would distinguish this sequence of events, anyway.

It's pretty clear what GPT-3 does doesn't match the common rationalization of human subjective experience of cognition, but it's not at all clear, AFAICT, that what the human brain does matches that rationalization, either.

Which is not to say I think GPT-3 has anything like the kind, much less the level, of understanding humans have, I just think some of the common arguments arrayed in casually dismissing it are based on suppositions about human cognition that aren't sufficiently examined.

> There's considerable debate over whether humans can have a coherent idea before it is reduced into symbolic language, and it's not clear how you would distinguish this sequence of events, anyway.

This sounds like the thing that is so silly a person has to be very educated to believe it.

You know how I know that humans have coherent ideas before rendering it into symbolic language... because they do. The GPT-3 paper, itself, is a bunch of ideas that were formed and then rendered into symbolic language. Literally every new book/work/presentation that a person decided to write because they said to themselves "I have a great idea, I should share it with the world" comes from this.

GPT-3 doesn't even know when it thinks it has a new idea. Contrast this with humans, which have to go out of their way to communicate and promote their idea because they understand it's novel.

I think the idea here is that symbolic language is the tool with which we forge our ideas.

To continue with the GPT example, the ideas are not rendered into symbolic language only at the point of writing - the ideas are formed in the mind using symbols and then expressed afterwards

I see it like this: With no way to represent my thoughts and the context around them succinctly, I would not be able to string various complex ideas together coherently

> GPT-3 is just better at obfuscating that the model has no clue what it's talking about.

There's an interesting angle to this as well, which is that it makes the models "unfalsifiable" in a way. You can never prove whether the data is a straight compression lookup or whether the network has generated an insight, because the model can't tell you (to anthropomorphize).

This, more than anything else, would be the value of having explainable models. I don't blame the ML community for this gap, but it puts them in the unenviable position of not being scientific in the Popperian sense. There's a great element of "trust us, the intelligence is in there" or "the intelligence will get there", but when everything's a mashup of more hardware and data without a known structure, we ultimate have to take that on faith. We can do empirical measurements after the fact, but the guiding projections for how an experiment should behave is lacking. (I don't think anyone in any community has a satisfactory answer to this btw.)

I wonder if 10 years ago one would believe that we’d have a model capable of generating an article that fools many given a complex prompt, yet is essentially incapable of any reasoning.

The Chinese room argument shows lots of people were thinking about it being a possibility but to have it happen so soon is something else.

I assume I wasn't the only early reader of blogs who thought they point at this being possible, but I admit I did not expect to see it so soon.

It is capable of reasoning. Good lord what does it have to do to prove it???!

There's also the fact that it's completely missing structures analogous to human memory/consciousness. I'm not talking about philosophical notions of consciousness and qualia here but the difference, in neuroscience, between a subliminal stimuli and a superliminal stimuli. Stimuli that aren't abstracted and moved to working memory leave no trace in the brain just a couple of seconds after they're removed, analogously to the 2048 character memory of GPT3. That's something that's still conspicuously missing from GPT3 and AlphaStar if you've watched enough of it's matches.

Reply on news.ycombinator.com