Hacker News Reader

Comment by tlrobinson | Hacker News Reader

One thing that worries me about generative AI is the degradation of “truth” over time. AI will be the cheapest way to generated content, by far. It will sometimes get facts subtly wrong, and eventually that AI generated content will be used to train future models. Rinse and repeat.

View on HN

We are getting closer and closer to a simulacrum and hyperreality.

We used to create things that were trying to simulate (reproduce) reality, but now we are using those "simulations" we'd created as if they were the real thing. With time we will be getting farther away from the "truth" (as you put it), and yes - I share your worry about that.

https://en.wikipedia.org/wiki/Simulacrum

EDIT: A good example I heard that explains what a simulacrum is was this: Ask a random person to draw a photo of a princes and see how many will draw a disney princess (which already was based on real princesses) vs how many will draw one looking like Catherine of Aragon or another real princess.

Similar to how we have low-background (pre-nuclear) steel, might we have pre-transformer content?

Yes indeed. I've been looking for an auto summarizer that reliably doesn't change the content. So far everything I've tried will make up or edit a key fact once in a while.

So you’ve described humans.

Currently computers can reliably do maths. Later AI will unreliably do maths. Exactly like humans.

So it will get stupider... maybe the singularity isn't bad like too smart but bad like dealing with too many stupid people.

Both stupider and less deterministic, but also and smarter and more flexible. Like humans.

The nice thing about math is that often it's much harder to find a proof than to verify that proof. So math AI is allowed to make lots of dumb mistakes, we just want it to make the occasional real finding too.

Unless we also ask AI to do the proof verification...

Why would you do that? Proof verification is pretty much a solved problem.

Maybe making (certain kinds of) math mistakes is a sign of intelligence.

Fair point, though I feel there's a difference as AI can generate content much more quickly.

Anywhere that truth matters will be unaffected. If such deviations from truth can withhold, then the truth never mattered. False assumptions will never hold where they can't, because reality is quite pervasive. Ask anyone who's had to productionize an ML model in a setting that requires a foot in reality. Even a single-digit drop in accuracy can have resounding effects.

The interesting thing is that is some ways this is a return to pre-modern era of lossy information transmission between the generations. Every story is re-molded by the re-teller. Languages change and thus the contextual interpretations. Even something a seemingly static as a book gets slowly modified as scribes rewrite scrolls over centuries.

Certainly possible, though we also have many hundreds of millions of people walking the globe taking pictures of things with their phones (not all of which are public to be used for training, but still).

It’s not too different from how human memory and written tradition or word of mouth works.

I've started seeing more of this crap show up on the front page of Google.

Kind of like how chicken taste like everything.

Jpeg bitrot 2.0

art is truth

Reply on news.ycombinator.com