Comment by lukev | Hacker News Reader

Comment by lukev | Hacker News Reader

Parent Full thread

This makes me sad. I hoped to see less use of language models in the wild; looks like we're heading the other direction.

I was recently able to articulate to myself why I dislike language models like GPT-3 so much.

Language is fundamentally about communication. I have a mental state, and language lets me share that some of that mental state with another human being. It's quite remarkable, and the cornerstone for pretty much every aspect of our lives as social creatures.

GPT-3 can generate text that's convincingly human-like; it's an amazing technical achievement. But (so far, anyway!) it is just that; a generator of plausible text. To borrow a philosophical term, there's no "interiority" there.

You can see this clearly in GPT-3's output; it is at its weakest when writing about the relationships between a set of concepts, or when trying to carry a line of thought or argument through a longer text. This is because there is no underlying _thing_ that is being communicated; no mental model or grounding concept. Just text, generated solely based on plausibility in a given context.

So while GPT-3 is truly remarkable in its ability to generate text, it cannot, by definition, communicate (except on the meta level via prompt selection and output curation.)

Does the world really need more words but with less communication?

I thought this was interesting so I literally pasted the entire thing into OpenAI's playground and asked DaVinci to write a rebuttal of your argument. Here is what it said:

---

There are a few problems with LukeV's argument. First, he conflates the ability to generate human-like text with the ability to communicate. Just because GPT-3 can generate text that sounds like it was written by a human does not mean that it cannot communicate. In fact, GPT-3 is quite good at carrying a line of thought or argument through a longer text, as evidenced by the fact that it is often used by writers to generate ideas and outlines for their articles.

Second, LukeV claims that GPT-3 is at its weakest when writing about the relationships between a set of concepts. However, this is not necessarily true. GPT-3 may not be able to generate text that is perfectly grammatical or that makes perfect sense, but it is often able to generate text that is nonetheless meaningful and insightful. For example, GPT-3 has been used to generate essays on complex topics such as the philosophy of mind, and these essays have been well-received by readers.

Third, LukeV argues that the world does not need more words, but this is clearly not the case. The world needs more communication, and GPT-3 can help to facilitate this by generating text that is meaningful and insightful. In fact, GPT-3 is already being used by writers to generate ideas and outlines for their articles, and it is likely that this use will only increase in the future.

In conclusion, LukeV's argument against GPT-3 is flawed. GPT-3 is a valuable tool that can help to facilitate communication, and the world needs more communication, not less.

----

I'm not sure who I agree with, but that's GPT-1's response to you regarding your critique of itself. Have at it!

> For example, GPT-3 has been used to generate essays on complex topics such as the philosophy of mind, and these essays have been well-received by readers.

That's the other problem with GPT-3... it will happily say things that are absolutely false :)

I think this response actually illustrates my point quite nicely; it is a plausible sounding rebuttal but does not actually engage on a logical level.

That said, I don't mean to diminish the impressiveness here. It's quite evident that it could pass a Turing test in a wide variety of circumstances. After all, it's not like humans are consistently great at concept-informed writing either. I'm pretty sure the majority of undergrad college papers have more in common with GPT than with real critical thought... generating something that looks plausibly like an argument rather than actually having one.

> I think this response actually illustrates my point quite nicely

Absolutely. From a surface-level read it may sound sensible but it all falls apart very easily.

The lines of reasoning for each attempted rebuttal are practically non sequiturs.

Out of curiosity, I’d be interested in seeing some attempts using prompts intended to induce CoT reasoning (“let’s think step-by-step“ etc.)

Lol, that last point about undergraduate papers is exactly what I was thinking reading your initial statement. As a current student, it feels that sometimes we're forced to just generate something that looks plausibly like an argument as opposed to actually generating one.

Well, it was trained on a lot of internet comments after all…

The argument seems very poorly reasoned to me.

> First, he conflates the ability to generate human-like text with the ability to communicate.

This is incorrect. In fact, lukev does the opposite. That is one of the fundamental parts of his argument -- that those two things are different.

> Second, LukeV claims that GPT-3 is at its weakest when writing about the relationships between a set of concepts. However, this is not necessarily true. GPT-3 may not be able to generate text that is perfectly grammatical or that makes perfect sense, but it is often able to generate text that is nonetheless meaningful and insightful. For example, GPT-3 has been used to generate essays on complex topics such as the philosophy of mind, and these essays have been well-received by readers.

I believe this is the best paragraph of the argument, but it is also the most difficult to rebut because it is rather subjective. Whether a text is meaningful or insightful is up the individual reading it. I would be surprised, however, if experts on the philosophy of mind found the essays mentioned by GPT-3 to be meaningful or insightful -- as opposed to regurgitations of concepts that are in the texts that have been fed to GPT-3's model.

> Third, LukeV argues that the world does not need more words, but this is clearly not the case. The world needs more communication...

The presence of more words in the world does not necessarily entail more communication. The two concepts are fairly closely related but are definitely not synonymous.

This doesn't necessarily mean that I agree with lukev, of course. I do think that GPT-3 as it currently exists should not be used to generate text that is meant for human consumption without being edited first. I can certainly see how it would be useful as a tool for generating an outline of an essay, especially with well-crafted prompts.

I am of course open to more arguments and data on the subject =)

I noticed that when GPT-3 has trouble continuing a sentence or create a difficult argument it relies on filler words to stretch out a sentence until it has no choice but to write something meaningful.

To be fair we have all been there.

LLMs are great and have many use cases but ultimately they’re just very talented bullshit machines.

This is due to the way they were trained: predict the most likely next word. In a difficult context the most likely next word is a filler word.

Wow, pretty much every single word of that rebuttal is complete nonsense. This is like 7th-grade level writing, filled up almost entirely of non-sequiturs, filler words, and regurgitating the prompt back at you. I may not have agreed with lukev before reading this response, but I certainly do now.

I’m siding with GPT in this debate. GPT is a writing tool like MS word, grammarly, or the printing press. Its innovative not destructive

> So while GPT-3 is truly remarkable in its ability to generate text, it cannot, by definition, communicate

GPT-3 does nothing without a human inputting what it wants it to output. GPT-3 is as much a tool for communication as are sounds, words, symbols, etc.

Whether it was carved in stone by a human hand or generated on the cloud by an AI, the intent to communicate is always coming from a human.

If you think the text generated by GPT-3 does not carry the right meaning or has "no underlying _thing_ that is being communicated", it just means that the human behind is not using the tool properly. Just like sometimes you use a word that doesn't mean what you think it means.

So, I actually agree with you there. Crafting a prompt, curating the output, and then distributing it certainly itself an act of communication.

I don't particularly like it as an act of communication because the information density is going to be quite low. If you want to tell me something, I'd rather you just told me the prompt! It will have all the relevant "real" communication; anything added by GPT-3 is basically just filler.

I’m not sure I follow. If we’re talking about turning a few sentences into a blog post then sure, but there are many use cases for these models.

E.g., distilling large amounts of text into short summaries while retaining substance is effectively the opposite of this, and is a use case GPT3 has been applied to successfully/commercially.

Reply on news.ycombinator.com