Comment by kkoncevicius | Hacker News Reader

Comment by kkoncevicius | Hacker News Reader

Parent Full thread

Yeah very similar for me, something with the way the author expresses himself. For example in the very beginning:

> Some are angry that GPT’s intelligence is being overstated and hyped up, when in reality it’s merely a “stochastic parrot,” a glorified autocomplete that still makes laughable commonsense errors and that lacks any model of reality outside streams of text. Others are angry instead that GPT’s growing intelligence isn’t being sufficiently respected and feared.

> Mostly my reaction has been: how can anyone stop being fascinated for long enough to be angry?

I am in the "stochastic parrot" group, but in no way am I angry. And I know people who think it should be respected and adopted faster, etc, etc, and those people aren't angry. Something just out of touch between how I feel about things and how the author describes them.

And these:

> Again and again the past few months, people have gotten in touch to tell me that they think OpenAI (and Microsoft, and Google) are risking the future of humanity by rushing ahead with a dangerous technology.

I seriously question how many people "gotten in touch" with the author to share such a silly message. But also this contributes to the writing style: "everyone is angry, I am reasonable; everyone asks me about the dangers of AI, and I explain it to them".

Context: the author is peripherally around the rationalist / lesswrong subculture (or at least read by them). A ~200-upvoted post in that community from yesterday basically starts with "we are in AGI endgame mode, with no solution to the alignment problem, everything is fucked, we are all going to die", reflecting on chatGPT, which is created by OpenAI, the organization the author is working for. That community has 1.6M visitors per month, and their members are about as much tech-savvy as HN.

That's the target audience for this piece, and I think he got a lot.

Does the should-not-exist camp say how they would eliminate rationality?

Sufficiently powerful governments can sometimes eliminate physical objects in territory they control when the means to produce them are difficult to acquire, such as fighter jets.

Simpler objects are harder to eliminate, such as rifles.

And as objects merge with information, they're almost impossible to eliminate, as with 3D-printed rifles.

Pure information is more difficult still, but powerful governments can and do censor history, criticism and ridicule they don't like, when it is distributed over a network they control, like telecom wires or government schools.

But ML is more slippery than all of these because ML isn't just information, it's math. Math can be independently derived.

Once enough people understand geometry, it's impossible to eliminate the knowledge that the sum of the angles of every triangle equals 180 degrees.

If the government of one region purges the people and burns the books, the knowledge still exists in another region. If a world government burns all the books everywhere, people will derive it again, independently.

If you're not angry, I suppose you're not in the "GPT shouldn't exist" camp, so it's not that it's a strawman, it's just that you're not in the group of people the author is arguing with.

I have indeed met people in the "stochastic parrots" camp who are very angry, and think GPT should be stopping, which strikes me as elitism as the author describes.

Interesting that the author, who supports the "stochastic parrot" interpretation, still works on GPT detection. If it's just a stochastic parrot, surely it will be trivial to detect, and its outputs useless anyway.

> One of the most immediate is how to make AI output detectable as such, in order to discourage its use for academic cheating as well as mass-generated propaganda and spam. As I’ve mentioned before on this blog, I’ve been working on that problem since this summer

I don't really know what a "stochastic parrot" is supposed to mean. It is an LLM that can do 1000 NLP tasks, or a glorified Markov Chain word salad factory?

It's also a bad term: demeaning to parrots - they can reproduce and don't need our help to survive, and demeaning to LLMs that can actually do more than parroting - they generate novel text that makes sense. Gebru really did a number on NLP, inventing such an insulting name, no wonder she was kicked out, I wouldn't want her in my team, she would just piss on any idea.

Reply on news.ycombinator.com