Hacker News Reader

Comment by mach1ne | Hacker News Reader

>"stochastic parrot" is a term coined by Emily M. Bender in the 2021 artificial intelligence research paper "On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?"

This might be the first time the term was seen in an ’official’ context, but is it really the origin? It feels like the term has been hovering around for longer, and even Google Trends shows significant search trends way before 2021

View on HN

I see only a few peaks starting in 2008 that are in the double digit numbers. Could these just be queries containing the words "stochastic" and "parrot"?

For instance, there's this ecology paper from 2014: Influence of stochastic processes and catastrophic events on the reproductive dynamics of the endangered Maroon‐fronted Parrot Rhynchopsitta terrisi

Not sure what happens under the hood, but it wouldn't surprise me if people searching for this paper would show up under "stochastic parrot" in google trends even if that's not what they literally searched for.

I feel this way too. Maybe there's some very similar term that we're both thinking of though that is just at the tip of the tongue because I can't find what it'd be

Most people have never actually dealt with something like modern LLMs, so we haven't really developed the proper language to describe them and how they behave. It's either too simplistic and reductive (stochastic parrot, xerox machine) or presupposes sentience and intent ("fabricates","hallucinates", etc.)

Also "a blurry jpeg of the internet". LOL, all we need is "a series of tubes, not a truck" and we're set.

I think we are focusing on the model too much and miss the real hero - language. The corpus of text these models are trained on is a marvel of human creativity. This cultural artefact is the diff between primitive and modern humans. And it is the diff between a random initialisation and a trained GPT-4. Maybe the brain or the model don't matter, but what you train them on.

Even more, language is special. Ideas are self replicators, they have a lifecycle, they have evolutionary pressure to improve. Ideas travel a lot. No single human can recreate this knowledge, it is the result of massive search. I'd say more than 99% of human intelligence is based on applying ideas invented by someone else. So let's be more lenient on the parroting accusations. AIs can be smart if they get feedback, like AlphaZero, but without feedback they of course have to parrot.

The Chinese room, maybe? https://en.wikipedia.org/wiki/Chinese_room

Their paper was the first time I heard the term.

If you find something, post it. Otherwise this sounds like sour grapes that women coined the term.

I don't think this coinage is something to be proud of, given that the stochastic parrot analysis is now rejected by most top AI researchers, like Geoffrey Hinton or Andrew Ng. Even LLM skeptic Yann LeCun says LLMs have some level of understanding:

https://twitter.com/ylecun/status/1667947166764023808

As far as I know, none of these 3 work specifically in NLP, most of their work is in image processing and to the best of my knowledge none of them have any background in linguistics.

Well, they are absolutely top AI researchers, so their opinion should count for a lot. If you specifically ask for people working on LLMs: Paul Christiano invented RLHF when he worked at OpenAI, and I'm pretty sure he also rejects the stochastic parrot analysis.

There is nothing in OP's comment to indicate misogyny.

Reply on news.ycombinator.com