Hacker News Reader

I don't understand why the goal is to "guarantee safety". It seems to be generally accepted that human error causes approximately 90% of motor vehicle accidents [0]. Surely then the goal of any autonomous or semi-autonomous transport system should be to merely reduce that percentage? If all motor vehicles were "self-driving" and the total number of annual deaths was reduced by one, wouldn't that be a good thing? I would rather reduce my chance of dying early than eliminate the possibility of being killed by a bot.

0. http://cyberlaw.stanford.edu/blog/2013/12/human-error-cause-...

Because humans are illogical creatures and will ignore objective facts if they believe control (or perceived control) is being taken away from them. The challenges of rolling out true autonomous vehicles are psychological as much as they are technical.

Until GPT-3 can write something meaningful

What is "meaningful"? Honest question. Isn't meaning assigned by a reader? If I'm reading poetry generated by GPT-3 and I like it just as much as poetry written by a human poet, does it make it meaningful? What if I finetune GPT-3 (or the bigger next gen version) on every scientific paper ever written, and as a result it generates a novel idea that turns out to be valid and useful, should I care if it happened by accident, or without "understanding"? Can the knowledge encoded in the model parameters be interpreted as some form of understanding? If no, why not? What's missing exactly? What is different from how human scientists operate?

In communication, meaning is a collaboration between writer and reader. The writer does their best to convey something; the reader does their best to understand.

There's also the kind of meaning that scientists and researchers talk about when they extract knowledge from data. That's pretty different from communication; it's more a process of internal generation of notions and explanations that could later be conveyed in communication.

And then there's the meaning that is even more internal. E.g., reading tarot cards or tea leaves, people generate meaning out of nothing. And then there's Pareidolia: https://en.wikipedia.org/wiki/Pareidolia

If you're saying machine-generated text is meaning in the sense of that last category then sure, it's something we loosely call meaning. But it's fundamentally the same as other sorts of cleromancy [1], just more elaborate.

[1] https://en.wikipedia.org/wiki/Cleromancy

a process of internal generation of notions

How is this "knowledge extraction from data" process different?

Immanuel Kant would call the former "rendering a synthetic judgement" and David Hume termed the latter revealing the a priori knowledge. Plato would call both simply giving substance to the forms.

It seems none of this is something new. It's just that you no longer need a good education to learn about old ideas; you can come up with them on your own.

Different from what? Regular communication? Because the first involves trying to sync up two minds to have the same idea. The latter involves one mind, trying to generate an idea that ends up being useful. Useful in the George Box sense: "All models are wrong, some models are useful."

I meant how is knowledge extraction done by a language model different from knowledge extraction done by a human scientist?

Sorry, I don't understand the question.

GPT-3 does some form knowledge extraction, right?

You said above: "There's also the kind of meaning that scientists and researchers talk about when they extract knowledge from data."

So if we agree that in both cases some form of knowledge extraction is happening, I wonder how these two forms compare.

A search for "GPT-3 knowledge extraction" doesn't yield much, so I don't know what you're talking about. But as far as I'm concerned, knowledge exists in the heads of people, so I don't think we agree on that.

knowledge exists in the heads of people

How is this knowledge encoded in the heads of people? Perhaps through the choice of synapse strength and connectivity between neurons?

What is being encoded in GPT-3 weights as it reads through millions of pages of text? How is it different from tuning biological synapses?

Still don't know what I'm talking about? It's OK, neither do I :)

The thing is, if we could write a specific, closed-end, prescriptive definition of "meaningful" or "understanding" or whatever, then we'd be able to program it. And we can't, so we have to settle for something else, usually how a thing fails to be what we (indeed subjectively) consider meaningful. Still, it's not arbitrary.

The way that something like GPT-3 tends to fail basically is that you 2-3 paragraphs where paragraph 3 will containment statements that subtly contradict the semantics of paragraph 1. Things like seeing things inside closed drawers and other cues that the thing has no fixed world-model. That's what gives an impression of "meaningless" or "no understanding". (admittedly, I've only played with GPT-2 but this is a description of how texts written by these models at first seem plausible and then implausible as you read more).

It's not some philosophical objection akin to "nothing but a human brain can think".

the thing has no fixed world-model

What is "world-model"? What makes you think GPT-3 does not have some kind of a world model? It's clearly not a very good one, but at the same time it does not mean it can't get better. A 3 year old also does not have a very good world model, what's the difference between his world model and one of GPT-3? Again, clearly there's a big difference, I'm just not sure we know enough about what's going on inside 175B model to make any dismissive statements about it. I'm also not sure what will happen if you train a much bigger model on much bigger data. Things might start to emerge at some scale.

Well, GPT-3 isn't any kind of general intelligence - it's explicitly architected as a language model - something that learns to pay attention to prior context to predict what word comes next. The only kind of world model it has is a statistical model of what word is most likely to come next based on the corpus it has been trained on.

You could argue that general intelligence is also based on prediction, and that a human's world model therefore isn't so different in nature, but there are some very significant differences ...

1) GPT-3's model is based only on a corpus of text (facts, lies, errors, etc) it was fed... there is no grounding in reality.

2) GPT-3 is only a passive model - it's not an agent that can act or in any way attempt to validate or augment it's world model.

3) GPT-3 is architecturally a language model .. it can get better with better or more data, but it's never going to be more than a language model.

The difference between a 3-year old's brain and GPT-3 is that the 3-year old's brain is not a one-trick pony ... it's a complex cognitive architecture, honed by millions of years evolution, capable of performing a whole range of tasks, not just language modelling.

The 3-year old's brain also has the massive advantage of being embedded in a 3-year old autonomous agent able to explore and interact with the world it's world model is representing... It you tell GPT-3 pigs can fly then as far as it is concerned pigs can fly, whereas the 3-year old can go seek out pigs and see that, in fact, they can't.

> What if I finetune GPT-3 ...

> if you train a much bigger model on much bigger data...

Yep, no one is denying that the potential exists for something to be created that can do something useful.

However, the point being made is that GPT-3 doesn't do anything particularly useful.

The generated text from is not consistent, and apparently fine tuning can result in poor performance (ie. generates random crap). It scored better at a bunch of metrics, which is great, but of questionable practical value.

"world model" or not, it's currently interesting, amazing... probably not useful unless you're writing a spam bot.

What is unclear, is if bigger models will actually solve that or not.

We know much more about the world-model of GPT-3 (which is none, but it has a language model) than we do about the three year old (which we can't build or reproduce in silico) but we know plenty well enough to say that no, things are not going to magically "emerge" at some scale. You need something fundamentally different from endless cloze deletion exercises to learn what a drawer is.

Well you can randomly string words together and occasionally get lucky and make a meaningful argument, but that doesn't mean you've created a good method for constructing new ideas.

It seems to me the simplest way to decide whether or not something is meaningful (in this context) is whether or not the author (which is the algorithm GPT-3) can respond to criticisms against its own argument in a coherent way. In which case it has to get lucky twice, so it's that much less likely to happen randomly. If the author cannot respond to comments in a comprehensible manner, it's hard to defend the author.

can respond to criticisms against its own argument in a coherent way

I'm not sure about GPT-3, but let's imagine GPT-4 next year will be able to do this. It just does not strike me as a particularly high bar to clear. Let's go further, and assume GPT-5 in 2022 will pass the Turing Test (you personally will not be able to tell). What would you say then?

Please don't wildly speculate about technologies you clearly don't understand. "Does not strike me as a particularly high bar" means you're unfamiliar with how these systems work at a deep level (by which I mean, you personally cannot sit down at a terminal and build one). So rather than -ahem- making things up and asking "what then?" -- please ask for textbook recommendations on these topics if you'd like to know more.

I guess we'll just have to wait and see what happens next year :)

p.s. I built my first language model (LSTM based) back in 2014. Then I built a VAE based one. Then a GAN based. None of them were especially good, so I switched to music generation (this actually works pretty well). My most recent project is using sparse transformers for raw audio generation. Building novel DL models is literally in my job description.

Please don't wildly speculate about strangers you meet on HN.

Why just wait when you can bet?

In order to bet, we would have to agree on evaluation criteria. A task like "respond to criticisms against its own argument in a coherent way" is difficult to evaluate. Turing test is also pretty vague, and some people already declared it passed many years ago: https://www.bbc.com/news/technology-27762088

Yes, it would be difficult but not impossible to agree on clear criteria. We would also need an impartial third party to make a determination.

The Turing test hasn't been passed if I'm the judge. Supposing I were unable to tell the difference between any AI system and a human interlocutor defending the same argument, at any time before 2023, I'd admit my prediction was wholly incorrect.

I doubt very much I'll find a taker for this bet, however. The AI field has always been bigger on optimism than results, as we both know.

Yes, I agree TT has not been passed. But probably starting later this year we will be seeing more and more claims it has. At first it will be clear it's not. Then not so clear, and then the goalposts will be moved again, so that when GPT-5 is announced and it is clearly capable of keeping a conversation, the reaction on HN will be the same as the current reaction to GPT-3: "meh".

That seems like a pretty testable prediction. If I can't tell the difference between GPT-5 or whatever it is and an adult human native English speaker by the end of 2022, you'll win the bet.

How many questions will you need to determine if it's a bot? How many sessions, and what percentage of correct guesses will determine the outcome of the experiment?

If it wasn't a bot but a human, we'd typically have an unbounded conversation, except by the bounds of politeness, until I was satisfied one way or the other, so that seems a reasonable protocol here, perhaps with some reasonable upper bound on the time spent (an hour?) to avoid putting a potentially unbounded commitment on human participants. (Actually, "We've been chatting since four, I have somewhere else to be" is a pretty good signal of humanity... and should help you see why you're not going to win this bet.) I wouldn't expect it to take more than a few minutes, and I would expect to be wrong in (much) less than 1% of sessions.

The more time you spent chatting the better your chances are to guess correctly. I admit that in 2 years the models might not be good enough to fool you for hours. If you put enough thought into it, especially knowing what the bot was trained on, you could devise a set of tricky questions which would expose it.

However, I believe the models will be good enough to fool you during, say, a 20 question/response dialog. They will definitely be able to fool vast majority of unsuspecting humans. And they will definitely be able to keep track of conversation (remember what you said previously, and use it to construct responses to follow up questions).

How will the bot answer "when and where were you born, and how do you know?" How will it answer "what color, besides red, best communicates the flavor of a strawberry, and why?". How will it answer "What historical figure does my communication style make you think of most, and why?", or "Which of your family members comes to your mind first?" or "What do you think the context was in which the following poem was written?". I don't need to know what it was trained on to win this bet, and 20 open-ended questions is more than enough.

Between the vast majority of unsuspecting humans and me there is a considerable gap. Mind the gap!

You are kidding, right? All these questions you provided are extremely simple to answer, compared to many other things clever human interrogators might say during TT. I'm starting to doubt your NLP expertise.

The TT ready model I'm envisioning will be trained on many billions of chat sessions. It will contain dozens of preconstructed graphs and will dynamically construct dozens more (personality graph, common sense knowledge graph, domain specific knowledge graphs, causality graph, dialog state graph, emotional state graph, etc), it will have a bunch of emotion detectors, humor detectors, inconsistency detectors, lie detectors, praise detectors, etc. It will have the ability to query external sources (e.g. google search --> web page parsing --> updating relevant graph). All these modules will filter, cooperate, and vote, providing input to higher level decision making blocks. These blocks will use those inputs to condition and constrain response generation process. This is finally where a language model comes in, and this until recently has been the hardest part - generating a coherent, grammatically correct, interesting text, directly addressing a specific prompt. This part has been solved. Until GPT-2 last year we simply could not generate high quality text. Now we can, and GPT-3 is even better at that. Sure, there are plenty of non-trivial problems left to solve, but I don't view them on the same level of difficulty - some of them have already been solved in the process of IBM Watson development, so I'm optimistic. The hardest remaining challenge is probably constructing common sense graphs. [1] looks promising.

p.s. your questions are so naive I'm not sure if you're trolling me. A human might answer them like this (and a bot built 50 years ago could easily imitate that):

"when and where were you born, and how do you know?"

- [personality - redneck] I was born on a farm in Oklahoma. How do I know what?

"what color, besides red, best communicates the flavor of a strawberry, and why?"

- Red is the right color for strawberries.

"What historical figure does my communication style make you think of most, and why?"

- You talk like one of them big city hipsters.

"Which of your family members comes to your mind first?"

- My little bro Jimmy, we just went fishing together on Tuesday.

"What do you think the context was in which the following poem was written?"

- [depends on the poem] I don't get this poem. What is it about?

[1] https://arxiv.org/abs/1906.05317

You're betting that in the next roughly two-and-a-half years, the common sense problem will be solved well enough to fool me, despite not having been solved in the entire history of AI up until now. I'll take that bet. How confident are you?

To fool you for 20 questions, yes. I'm ~70% confident, so I'll bet you $100 :)

To clarify, the common sense problem is a hard one. It is similar to level 5 autonomy driving. That will take a while to solve. But what we are talking about here is kinda like Waymo cars which can drive themselves in ideal weather at slow speeds in Arizona. So in 2.5 years I think the best chatbots will be as far from having common sense as the current Waymo self driving cars are from level 5 autonomy. Which is to say they will be pretty good.

You're on for $100. I'm > 99% confident that I won't be fooled by any AI before 2023, and would have bet any amount that I could afford to set aside.

Ideal weather at slow speeds, with a professional human driver throwing road-condition curveballs at you and challenging your responses? I like my chances.

Sounds good! My email is in my profile, if I'm wrong I'll pay up! :)

Keep in mind that you would have to differentiate bot's responses from those of an old truck driver, a snobby philosophy student, a conspiracy theorist, a stoner, a bimbo following latest Kardashians news, etc. Lots of different personalities of real humans who can throw curveballs at you during a chat session. I hope you don't expect to only chat with bay area devs? :)

I'd be very disappointed if I only get bay area devs (since that's a miniscule fraction of the people I've had the pleasure of having interesting discussions with during my life). And indeed, "give me a few sentences about your background, schooling, and interests" is an excellent opening question ;)

>But it leads to things like Teslas driving into trucks parked sideways across the road because the engineers simply never predicted it would ever happen. And now it's happened multiple times!

This is a completely wrong reading of the situation. Tesla engineers clearly understood it is impossible to determine whether a stationary object is actually on the street or just an overhead sign with the equipment that is available on current Tesla cars and therefore they simply ignore all stationary objects. Because Autopilot is not a self driving technology this is not considered a problem. If a Tesla with enabled Autopilot ever runs into a stationary object it is clearly the driver's fault. It's extremely predictable and since it's easy to blame the driver there is no need to fix the problem by adding the required sensors to Tesla cars.

There's a ton of problems that a quality language model like GPT-3 can solve, beyond just spamming text generation - stuff like translation quality, automatic post-editing of text, classification, etc.

"Sure it's cool, but what problem is it solving?"

Isn't it good enough, or very nearly, to generate fake news? And if you can generate something that fools a large percentage of humans, even for a second or two, you can sell ads. It would be solving a problem for anyone who can profit from it. I thought the people who developed it stated that it was too dangerous to release widely? Dangerous = useful to bad people, no?

That was just PR fluff designed to play to the ideological biases of their Valley employee base. They released GPT-2 anyway some months later because other people were going to replicate it anyway, and guess what, the river of fake news we're flooded with daily is still not being generated by AI. It's being generated by journalists with an agenda, same as ever.

Don't get me wrong. You can absolutely generate news articles with these text generation models. But if you look at examples of the "fake news" generated by the monster GPT-3 model, it's not really any different to junk churned out by supposedly respectable news organisations i.e. grammatically correct but filled with logical contradictions, false statements that can be checked in 30 seconds with a search engine and so on. Most people already learned which news sources are trustworthy and which are pushing an agenda. If those outlets replace their journalists with GPT-3 it won't make any difference. The readers who don't trust them will still not trust them, and the readers who just want partisan cheerleading will continue to be satisfied.

The proportion of people who will spend 30 seconds with a search engine to check on something is practically infinitesimal. Even people who do it frequently don't do it most of the time. And it doesn't matter anyway, because it's too late - you only have to fool people for a second, or a fraction of a second.

Great, so focus on solving that problem generally instead of worrying about AI.

What I see is actually quite different. If the people who could detect fake news was practically infinitesimal then you'd see the vast majority of people having super high trust in the media. In fact most people don't trust the media, lots of polls showing that. Sure they may only fact check something occasionally (often by reading about a topic they happen to understand), but people aren't stupid. After they notice or hear about a few mistakes and observe they're always in the same ideological direction, they get the picture.

In ODS.ai Russian slack-chat a Replika developer has showed the increase in metrics that was provided by GPT-3 in comparison to GPT-2. Here you can see it on Telegram (also in Russian, but the pic is in English) without the access to ODS-slack. https://t.me/dlinnlp/931