Hacker News Reader

Comment by Chris2048 | Hacker News Reader

Chris2048

I think the real problem with GPT3 etc is it's not really original.

It's a corpus of text so large and varied, that simple requests can bring up a few good examples, which can be mashed together to seem new. It's essentially a smart search & plagiarism process. I suspect if you could view the main contributing sources it would be less impressive.

for example, I was really impressed that GPT3 could write code - until I realised that it contained most of stack-overflow (including code examples) - no actual inference was happening, it was just search.

View on HN

That's not true, or is at least making GPT3 out to sound more trivial than it is. I've spent hours "interviewing" people about dragon sightings in Washington D.C., or Elon Musk about a fictional startup, or similar. I've had it generate startup ideas that are definitely original and elaborate on them, including company names that are concatenations of words that have never been used as a proper noun. I had it generate an interview with the founder of a fictional 1980s video game company, and it went into detail about the series of fictional games. It was all plausible, but the names were unique strings that I could not find on Google.

I'm not impressed by fictional nouns, be they people names or fictional corporations. GPTs special sauce was supposed to be text by, at the least, the paragraph, so it's the "generate/elaborate on startup ideas" and "went into detail about the series of fictional games" I'd be interested in.

Without seeing the source material, how original do you know these generations to be?

I've had to explain this to non-techy family. GPT3 is still a monkey at a keyboard. Only, the keyboard is huge, and each button is a whole phrase. Maybe the keyboard reconfigures itself after each keypress, to make the phrases scan a bit better. But it's still a monkey at a keyboard.

On the one hand, there always seemed something kind of flawed about the work OpenAI is doing versus DeepMind.

On the other hand, people talking about OpenAI all the time helps it thrive - all publicity is good publicity. So the best thing you can do personally is, if you don't like what they do, just don't talk about it anymore.

This is a valid literary technique, if you're not aware.

https://en.wikipedia.org/wiki/Cut-up_technique

It's a know technique, I wouldn't say that makes it "valid". It also seems to be a inspirational technique i.e. adding an artificial limit to spur creativity.

The text is ultimately managed by a real human being otherwise capable of writing original text. That human being hasn't been exposed to English entirely through the corpus of test given, as would be the case of GPT, which has no previous knowledge of language.

TLDR; the technique gives the text to an already create human being, rather than expecting creativity to arise from the technique itself.

> I think the real problem with GPT3 etc is it's not really original.

Everything's a remix.

Reply on news.ycombinator.com