Hacker News Reader

GPT3 and its cousins do things that no previous language model could do; it is qualitatively different from Eliza in its capabilities. As for your argument about random projections in the evaluation of GLoVE, comparisons with random projections are now routine. See for example https://aclanthology.org/N19-1419/

Why do you say it is qualitatively different from Eliza in its capabilities?

It does something totally different. However that totally different still depends on people being desperate to see intelligence inside it. It’s like how people see a face in a cut stem or on Mars.

Is it because it does something totally different that you came to me?

You're trying to prove some kind of point where you respond as ELIZA would have to show how "even back then we could pass for conversation". The truth is that GPT-3 is actually, totally qualitatively different and if you played with it enough you'd realize.

The difference is quantitative, rather than qualitative, as compared to primitive Markov models that have been used in the past. It's just a numerical model with a very large number of parameters that extends a text token sequence.

The parameter size is so large that it has in essence memorized its training data, so if the right answer was already present in the training data you'll get it, same if the answer is closely related to the training data in a way that lets the model predict it. If the wrong answer was present in the training data you may well get that.

And ice is the same thing as water, just at a quantitively different temperature.

There's a famous saying "quantity has a quality all its own".

If GPT-3 is known to have some sort of intelligence, I think it logically follows that one can differentiate that from the reflected intelligence of all the humans that produced all the data it ingested.

How do you see intelligence in GPT-3 and not see it in the data fed to it?

Does it bother you that I'm trying to prove some kind of point?

What is your criterion for "truly" detecting intelligence? Do you have a test in mind that would succeed for humans and fail for GPT3?

It's a legitimate practice in science to speculate. Having heard the Harvard guy explain more fully the Oumuamua thing, it's struck me as perfectly fine activity for some scientist to look into. His hypothesis is almost certainly going to be untrue, but it's fine to investigate a bit of a moonshot idea. You don't want half the field doing this, but you absolutely need different little pockets of speculative work going on in order to keep scientific inquiry open, dynamic, and diverse.

> the public would quickly learn that it can answer any question at all... if you don't mind if the answer is right.

There appear to be an awful lot of conversations in which people care about other things much, much more than what is objectively correct.

And any technology that can greatly amplify engagement in that kind of conversation probably is dangerous.

> there are four projections of the 50-dimensional vector space that would project out just as well from a random matrix because, well, projecting 20 generic points in a 50-dimensional space to 2-dimensions you can make the points fall exactly where you want in 2 dimensions.

Well, no. Linear projections follow a bunch of rules that enforce conservation of various linear structures. You can't manipulate things arbitrarily.

For example, if three points are colinear in the original space, they will be colinear in every projection. Perhaps more relevant to the GloVe examples, if a-b = c-d in the original space then the same equality holds in every projection. Since the projections are Lipschitz continuous, we can also say that is a-b is close to c-d in the original space then they are also close in every projection.

Anyway, the point that these figures illustrate is easily confirmed by downloading the embeddings yourself, so insinuating that the authors are getting away with something that they should be held to account for is silly.

Confused. If professor Loeb tries to at least open discourse to the idea that ET space junk might be flying around like our space junk in a desire to reduce the giggle factor around that hypothesis, what sort of "consequences" do you think he should face for that?