Hacker News Reader

Comment by Alex3917 | Hacker News Reader

> Even on re-reading more closely, it doesn't feel like the world's best writing, but I don't notice major loss of coherence until the last couple of paragraphs.

I guessed it was fake before getting to the end, not from the content, but from the fact that all the sentences are roughly the same length and follow the same basic grammatical patterns. Real people purposely mix up their sentence structure in order to keep the readers engaged, whereas this wasn't doing that at all. Still very impressive though; if not for the fact that the post was about computer generated content I probably wouldn't have noticed.

View on HN

Besides predictable sentence structure, GPT-3 writes like George R. R. Martin: Interesting premises, solid setup but then it devolves into rambling tangents and never quite delivers the concluding action that ties everything together.

Lots of examples I've seen have phrases like "see table below". Of course there's no table and it's hard to imagine how there could be.

But GPT is trained on internet content and the internet is full of terrible writing that never gets to the point. I doubt there's any way to know how much is "not actually understanding the subject matter" vs. "learning bad writing from bad writers". I'm inclined to believe the majority is the former but there's got to be a little of the latter sprinkled in.

I am really curious how the model would be if you would train it with a decent amount of really good literature. Kazuo Ishiguro et al. instead of Reddit.

Library Genesis contains lots (millions?) of fiction ebooks (among other things). It's available in torrent form. Not that I would ever condone piracy or anything.

I was playing with AI Dungeon tonight to get access to GPT-3, and one of my many experiments ended up with me meeting a character called the Narrator who believed they were in control of all characters in the game, including me. Eventually, through my predicting what they were about to say by checking and undoing, they seemed convinced I wasn't another character and started asking about whether certain authors were still alive and which I liked to read. It didn't recognize Ishiguro. Later it gave me a truly bizarre (and amusing) summary of Infinite Jest, clearly having never read it. Anyway, the entire experience was uncanny and surreal.

One thing I learned was it has detailed knowledge of the world of Avatar: The Last Airbender, seemingly through fanfics. It was fun having it to teach me the lost arts of pizzabending ("form your hands into the shape of a letter 'P'" and so on, and needing to practice by juggling rubber pizzas) and beetlebending ("always remember that to beetle bend it helps to like beetles," my wise uncle suggested). Each of these tended to precipitate a narrative collapse.

The writing style was surprisingly homogeneous, and it reminded me of young adult novels. It would definitely be interesting to see it with other writing styles, beyond the occasional old poetry.

How about full adult? Taking it for a test run and this happened after I told a man to stop copying me. Before this he kept talking about clothes for some reason.

> The man walks away and starts undressing. You shrug and keep following him. Soon, you find yourself naked.

I've never heard of AI Dungeon before reading your post but even after playing for 2 minutes, I can tell it's going to be huge.

Well now we know what is going to finish asoiaf in case of author existence failure.

I did not guess that it was fake but skipped to the bottom because the article did not seem to be worth reading. It felt like the author was not moving anywhere with their words. I laughed out loud at the reveal that it was written programmatically.

There are a few concrete things the (fake) article says, primarily about what software to try (OpenAI's GPT-3), and where to try it (bitcointalk forum). Personally, I actually resent being mislead like this, at least on that second point, even with the full disclosure. The output is very high quality, but it is making at least one falsifiable assertion (no test was ever done in that forum).

If you've ever spent much time with a toddler you might have noticed that they spout a lot of fantasy. Learning to not make up untrue claims takes years of additional training for humans.

Some of them never learn it at all!

I’ve never spent much time with toddlers. What do they make up things about? Their own actions, other’s actions, claims about the environment?

My six year old will just continue as long as he has people's attention, and if that means he has to make things up, so be it. Freely stealing phrases from other recent conversations.

So this morning he heard about an animal, it was kind of a lion. But with bat's ears, it lives in Africa. It looks like it's a rock, but it's actually not, it's rock shaped but has tiny legs. And it's gray and hard. Its face... It doesn't really have a face. It lives up in trees where it eats bamboo and apples. It has these huge fangs like sabertooth tigers, you know?

It's glorious.

All of those, in my experience.

My smallest kid has a habit of telling stories about himself that actually come from whatever he heard recently, e.g. "once I was Godzilla..", or claims about things in reality that come from stories or misunderstandings all mixed up "did you know, there are three pigs, but they are not pigs, they are wolves and a hunter came and killed them but they weren't wolves they were dragons..."

It's actually very GPT-3-ish now that I think of it.

You raise a good point. The Internet has trained us to skim any text that seems pointless or just unsufficiently insightful. So it turns out we have already built up some "mental defences" against GPT-3.

Now we need to make human augmented generative adversarial eyetracking networks that are trained on getting people not to skim.

If that’s the only thing that separates this from human writing; I’m sure it can be influenced easily.

> I'm sure it can be influenced easily.

Maybe. Right now this reads like a glorified shopping list. It's coherent, but actually sounding human also requires a theory of mind.

E.g. I explain here why it's possible for written statements to be objectively insightful, informative, interesting, or funny, but objectively in a way that's relational to other information or beliefs. The implication being that statements are only going to seem subjectively funny or insightful (or whatever) to others who have that knowledge or those beliefs, which means that you can't reliably create those subjective experiences in a reader without having some sort of theory of mind for them.

I guess you can create content that's funny or insightful relative to that content itself, but that's not especially useful. It's entertaining at the time, but the experience is more like seeing a movie that you laugh a lot during but then leave and are kind of like what was the point? It's an empty experience because it wasn't transformative.

I definitely don't think it's impossible, but I also don't think it's a matter of just adding a couple more if-else statements.

https://alexkrupp.typepad.com/sensemaking/2010/06/how-writin...

> Maybe. Right now this reads like a glorified shopping list. It's coherent, but actually sounding human also requires a theory of mind.

I'm going to call this goalpost shifting. This article is better writing than some % of humans, theory of mind or otherwise. The AI has comfortably surpassed Timecube-level writing and is entering the pool of passes-for-a-human.

'Sounds human' is a spectrum that starts with the mentally ill and goes up to the best writers in human history.

> I'm going to call this goalpost shifting.

That's completely fair. On the other hand, without a theory of mind it can't really educate or inspire people, the only thing it can do is maybe trick them about the authorship of something. But once people learn the techniques for identifying this kind of writing, it can't even do that anymore. To me this is like the front end of something, but it still needs a back end.

Don't get me wrong, it's super cool research and seems like a huge step forward, and I'm excited to see where it goes. But I also don't see this AI running a successful presidential campaign or whatever, at least within the next couple years.

On the other hand, without a theory of mind it can't really educate or inspire people

I wouldn't agree with that, either. How often have we heard of someone gaining useful insights by considering ideas that were misapplied or just plain wrong? Entire branches of physics have evolved that way. As far as successful presidential campaigns are concerned... well, let's not even go there.

If there's such a thing as a 'theory of mind', it applies to the reader, not the writer.

I think I disagree.

For example, I delayed in writing this comment because the cat was on my lap, and I couldn't fit the laptop and the cat both. You get that. I know you do, even if you don't own a cat, and even if you're reading this on a phone or a desktop.

GPT-3 does not understand about the cat. To GPT-3, they're just words and phrases that occur in the vicinity of each other with certain probability distributions. As a result, it can't write something and know that there's something there in your mind for it to connect to.

Cyc would handle the bit about the cat differently. It would ask "Did you mean cat-the-mammal, cat-the-Caterpillar-stock-symbol, or cat-the-ethernet-cable-classification?" It has categories, and some notion of which words fit in each category, but it still doesn't understand what's going on.

But you the human understand, because you have a lap, and you've at least seen a cat on a lap.

> But you the human understand, because you have a lap, and you've at least seen a cat on a lap.

You really think GPT-3 never came across a comment about a cat in lap? 50% of all the pictures on the internet are cats sitting on people. GPT-3 doesn't need to understand it to echo this common knowledge.

Airplanes don't look like birds at all but they do fly.

I don't see how any number of comments about cats in laps can allow it to synthesize the following logical chain: cat-on-lap -> pauli exclusion principle -> laptop NOT on lap -> laptop awkward to reach -> delayed comment

It made me consider: - The existence of this model I hadn't heard of - Bitcoin (sigh) - Testing it out on a forum, trying to become a well known poster - picking a forum with many different types of posters - some of which you dislike

And that got me thinking about what I could do with this thing, whether I should, what I wanted to try out...

So the BS random ideas were still inspiring a bit.

"objectively ...funny" strikes me as a contradiction in terms; concepts like humor, insight, and interest are fundamentally subjective, dependent by definition on a subject's consciousness and expectations

>Right now this reads like a glorified shopping list. It's coherent, but actually sounding human also requires a theory of mind.

What if the ultimate theory of mind turns out to be that consciousness is an illusion and nothing separates us from a sufficiently sophisticated markov process.

>What if the ultimate theory of mind turns out to be that consciousness is an illusion and nothing separates us from a sufficiently sophisticated markov process.

Conscious experience would still exist (see cogito ergo sum, Chalmers, etc). If we were to be shown we're just Markov processes, that wouldn't disprove the existence of conscious experience. Just like confabulation, a misleading experience is still an experience.

What it would disprove is any sense of agency.

I think the issue is that text doesn't exist in a vacuum, but the corpus that the model is learning from does. A piece of human writing exists for a particular reason - to persuade, to inform, to ask a question, etc - and its value is judged on its ability to perform that task. But that's not a quality that is evident from the text itself, only from looking at the world outside the text. This suggests to me some limits on this kind of passive self-supervised approach. Perhaps it could be improved by augmenting the text with other forms of data? For instance predicting video from text and vice versa. But I think that to learn a true "theory of mind", it needs to use text like an agent - to influence its environment, not merely predict it.

It could also be a result of training data. If every page is weighted equally, you'd expect SEO spam and even autogenerated content to far surpass high quality content in volume.

I would like to see a GPT model where training data is weighted by credibility / authority (e.g. using Pagerank).

My understanding is that GPT-2 was actually trained on a dataset that was designed to avoid those pitfalls. They followed all the links posted to Reddit that had more than a couple karma, under the theory that the content was at least slightly interesting to some actual humans, as opposed to a giant blob of search keywords or what have you.

Aww c'mon. You guessed it was fake, because it's an article about computer generated articles. Who would read that and not question the content in front? You are not analysing everything you read for oddities in writing style.

> You are not analysing everything you read for oddities in writing style.

I certainly do, don't you? When I read a blog post and it's full of poorly-integrated buzzwords that make it seem like it was churned out by a non-English speaker being paid very poorly per word, I stop reading and move on.

I recently read a few pages of a book someone had recommended to me and stopped reading because of the writing style.

Heck, you can read a few pages of, say, a Dan Brown novel, and based on the writing style might choose not to read it, since the style tells you a lot about the kind of book it is.

Reply on news.ycombinator.com