Hacker News Reader

Comment by jmbwell | Hacker News Reader

And when this serpent starts eating its tail and the AI starts returning highly-ranked results consisting of AI-generated content, … well, I hope I can move to a farm by then.

View on HN

Hmm, maybe the singularity won't be a time when the AI take over, but instead when a single element's hallucination causes a cascade failure that brings it all down.

There was a twitter exchange where Sam Altman said something like, all of human progress so far has roughly been "horizontal" on a graph and it's about to go vertical.

He just never specified which direction :)

This is exactly what I think will happen. The internet is quickly going to fill up with AI garbage, and it's going to be more and more difficult for AI to source new human content to feed into it's own model.

It kind of already is. Self-published ebooks on amazon's kindle store are currently flooded with garbage written using ChatGPT.

It’s been like that since humans have been around. That’s the way knowledge spreads.

We humans have been repeating and imitating ourselves forever. That’s even the way babies learn to talk.

And we are not all clones, we don’t all think, say or do exactly the same things. But most just repeat and consume the same content and ideas (movies, music, books, languages, media).

(Why are you being downvoted?)

I think the important part is to have a minimum of filtering. Humans consume knowledge brought by others humans, but most of the time we cherry pick the true and useful knowledge and reject what turns out to be false (ideally).

I think what AI brings to the table is automation and speed, but the quality is not better. So if AI starts consuming its own content, will that decrease the quality of knowledge* in general?

(*Here I mean the first knowledge you quickly get from a search or asking some AI, not what you could get after hours/days of research.)

HN crowd is so very skeptical about the usefulness of a.i., especially related to content generation. I share the skepticism (the positive and hence out of control feedback loop of ai consuming its own content recursively). But on the other hand, when smart people enhance the primitives of the a.i. algorithms, a.i. might become better at rating the quality of its own output. Not by means of the question, 'does this look like anything found on the internet' , rather by the question, 'does this make sense, is this even possible, do the numbers add up' ? Also, it is not very hard to "freeze" a pre-ai knowledge base or NN, and use that as a reference for sensibility.

>But on the other hand, when smart people enhance the primitives of the a.i. algorithms, a.i. might become better at rating the quality of its own output

All incentives of massive industries like SPAM, "content creation", "news" publishing, and advertising, are against it becoming better at rating the quality of its output - or rather, just have it become better at being undetactable but still a cheap fast mass produced wall of text...

Again typical (not necessarily unwarranted though) HN skepticism... Think about other uses of AI. For instance, Medical advice, SW dev tools, engineering aid,... Pattern recognition for climate study, better interactive assistants,... These use cases and the businesses around it are not necessarily "just generate some content as click-bait" driven.

>It’s been like that since humans have been around. That’s the way knowledge spreads. We humans have been repeating and imitating ourselves forever.

Humans repeating and imitating others has historically been a "volatile memory" keep-alive mechanism. Basically saving culture and knowledge in people's minds and helping it transfer, as literacy was low and (manual) writing and book copying scarce, expensive, and time-consuming.

Even when, with the advent of typography writing was easier to reproduce, but still somewhat costly (cost of materials, typesetting, distribution, etc.) and gatekeepers (publishing houses, bookstores, etc.) ensured that most stuff is somewhat original, not just copies or random permutations of the same content. Indexing was also costly manual labor (creating dictionaries, curated bibliographies, books with oversight on a subject matter and references to what the main science/wisdom/etc about that is, library collections, etc.)

Now, however, SPAM and AI-SPAM is on mostly permanent storage - and permanent storage, indexing, and duplication/reproduction costs close to zero and happens automatically at huge scale.

So, no, it's not the same thing. The same way breaking a quick little wind is not the same as having full-blown Taco-Bell-inspired diarhea.

That’s even the way babies learn to talk

No, it's not. Babies learn to talk by babbling, and then by taking cues from their parents which babbles elicit a response and which don't. If you want to translate that to AI, it would mean that the AI would spew random garbage and then learn to filter its garbage from the responses to its outputs. That's pretty far away from what ChatGPT and other language models are doing right now, because they stop learning before they start producing any output.

You haven't heard about RLHF, have you?

That's like teaching a baby to speak by recording it for a month while not reacting to it, and then forming a committee to analyze the recordings and conduct a high-intensity training session with the baby.

Actually if ChatGPT gives you a bad/wrong answer, and you reply telling it is wrong and why it is wrong, it will answer with something that you think is correct.

So you are already doing a kind of real time RLHF in the chat. That is how DAN was prompted to exist.

Well, correct me if I'm wrong, but if I do what you describe, all the learning will be gone when I close the tab, or even when I chat to it some more, so that my correction falls out of its context window.

That just means that it doesn't have a long-term memory. This is not an intrinsic limitation - you can give those things access to some larger storage of information and tell them how they can query it, and they will (try to) do that. It's exactly how Bing AI does relevant searches, for example.

Better move fast, AI will accelerate the production of this type of content exponentially. The ratio of human to AI content could start tilting to the latter much earlier than we imagine.

Reply on news.ycombinator.com