Hacker News Reader

Comment by zulban | Hacker News Reader

In theory you're right, in practice you're not.

We don't need the metaphysical solution to the problem of detecting AI videos for the rest of time. Certainly, it's fairly easy to make something that mostly works most of the time. Enough to be very, very useful.

View on HN

This attitude of "individual cases don't matter as long as the average case is somewhat covered" is exactly why the world's going to shit.

The parent post's worry is warranted, IMO.

You're wrong from your very premise. The world isn't going to shit. It's better than it's been at pretty much any time in human history, in almost every facet.

(Individual experiences may vary)

That's always going to be true no matter what trajectory the world is heading in.

The point is that, on average, individual experiences are improving, decade by decade. Humanity is doing a remarkable job.

This short video is the best response to what you wrote above:

https://youtu.be/q_Rqvd1ayUE

You could say it's trending up, but there is no way you can deny that we are in a regressive period.

I mean if you focus on the negative and ignore the positive (which is what many people + the media do), you could say it's overall a regressive period. But you could do that (and people have) literally every year for the past 80 years.

Yet, in the grand scheme of things, the world has gotten better in that time. Which is how you know these people are wrong, at least most of the time.

The challenge with positive news is that you have to go seek it out. It will rarely come to you. And then you have to greet it without cynicism, which many are incapable of.

In theory you're right, in practice you're not.

I think it's the opposite

You're arguing the world is at its peak, they're arguing it's directed in a shit direction. You're not disproving them at this point.

There have always been bumps on the path upward. It's never been smooth. Doesn't change the fact that we're not going to shit, and that we haven't been going to shit for hundreds of years. Quite the opposite.

Indeed. See also Pinker's book Better Angels.

On average Steven Pinker is at best a fake hyperoptimist-by-aggregate who puts billionaires on pedestals and rewrites history to entrench shitty systems. Sometimes he says smart stuff but he ignores or actively disregards massive problems with a painfully self-serving neutrality.

Fair enough. The online world is going to shit.

Yeah I get the pretty much, the car was near the mountain top in the 80s and 90s and "pretty much" flew off a cliff more recently. Sure, we're still alive but everyone is going to die in about 5 seconds.

Drugs are out of control. Homeless are everywhere. No one has interests in anything. No one is having kids. All jobs are going to be gone soon. Colleges can't teach (it's all AI cheating now). People are Gang Robbing stores. Cartels are killing hundreds daily. Fraud is out of control. We have 2 maybe 3 world wars going on simultaneously now. Prices are skyrocketing.

Yeah I get why you say "pretty much". lol PS good luck buying a house

>(it's all AI cheating now)

My daughter's English professor is now requiring people to hand write their essays during class. So at least there is that.

People feel strongly about AI generated content; this is a case where false positives can destroy credibility and disrupt careers.

"Works most of the time" isn't good enough here.

This isn't even at the level of the spam filter on your email account. Are there some false positives and negatives? Yes. Are there some people sending emails who are negatively affected by falsely ending up in the junk mail folder? Yes. Are we going to turn off spam filtering because of this? No. Why should we accept video spam any more than text spam?

The problem is that it's not SOME false positives, AI detectors so far have been all so comically bad that they might be classified as pseudoscience. Or an artificial false positive generators even.

We'll I'd think that YouTube would have incentive to get it right. Either there are too many false positives and the content creators go away and YouTube collapses. Or there are too many false negatives and the viewers go away, and YouTube collapses. I mean there is a chance that garbage people will ruin video sharing platforms for everyone.

Having the incentive to do something and having the ability to do it are not the same thing.

It's not like human-generated content is made of carbon and AI-generated content is made of silicon and the science of chemistry can unambiguously tell them apart. If you asked a million humans and a million LLMs to write a sentence on a specific subject, it's not implausible that one of the LLMs and one of the humans would output the exact same sentence. Maybe more than one.

A thing that can take only the output and accurately tell you if it was AI-generated or not is therefore impossible, because if it said no it would be wrong when the LLM generates that sentence, but if it said yes it would be wrong when a human generates the exact same sentence.

All it can do is try to calculate a probability. But then what do you want to do with that? Suppose the probability it estimates for some content is 45%, and that probability estimate is an accurate measure of the true probability, i.e. can't be improved when the only information you have is the content itself. Do you want to ban the 55% of that content which is human-generated, or allow the 45% which is AI-generated?

Right now the problem is the flood of low-quality AI spam that might (or might not) be low hanging fruit. We can worry about high quality AI artifacts later if that becomes a problem. (and yes, there is no guarantee that YouTube won't fail due to these spammers)

But is an algorithmic AI detector really a thing?

I get the idea: get 10k each samples of human data and AI data, train a simple classifier until it gets 99.9999% accuracy or <10k false negatives per day at your scale, ship it as a screening tool.

Is such tool feasible at all with current state of AI technology, or is it just a reasonable take from the past that may not be so reasonable anymore?

> I get the idea: get 10k each samples of human data and AI data, train a simple classifier until it gets 99.9999% accuracy or <10k false negatives per day at your scale

The issue is, that's not a thing. AI-generated content and human-generated content have significant overlap. No amount of training data can allow you to distinguish them with that level of accuracy because many outputs exist that could have been generated by either one. Additional training data allows you to say that the probability is 55.0374% plus or minus 0.0001, rather than only being able to say that it's 55% plus or minus 5%. It can tell you with greater precision exactly how ambiguous it is. What it can't do is remove the ambiguity.

We will find out shortly? YouTube is the one saying they are going to implement this:

"If a creator doesn’t specify whether or not they used AI, but our systems detect significant photorealistic AI use, we will now automatically apply a label."

Excellent analogy. I'm going to use that sometime.

Email spam filtering can clearly cause reputational harm too.

They don’t seem to care about false positives anywhere else on the platform. Being at the mercy of automated Google systems comes with the territory.

There will also be tiktok challenges to do a video that YouTube flags as AI without actually using AI.

Even worse if it's some attribute considered by the algorithm but not disclosed. "Likely AI" is enough to be damaging without even being tagged "Disclosed as AI"

This isn't a choice between "perfectly fine how things are now" and "destroying credibility". If it were, you're right - "good enough to be useful" wouldn't be a high enough bar.

Things are not perfectly fine how things are now. AI slop is destroying the internet. Tons of grifters are earning tons of money off YouTube by brainwashing millions of people with AI slop, including my mom. YouTube needs to do something and this seems feasible and far better than doing nothing.

I also think the false positive rate is going to be far lower than you think - especially if YouTube sets a caution threshold.

I'm open to other solutions but if you propose we just keep what we have now, then you are proposing an absolute disaster.

And this philosophy will only lead to Kafkaesque nightmare scenarios for 1-2% of the population, so we're still coming out ahead.

As Gregor Samsa awoke one morning from uneasy dreams, he found that one of his YouTube videos had gotten a little less engagement.

People make a living off this platform though, this could be really bad for someone that lives off of YouTube to have their videos labeled as AI generated. This would still be OK if there was a person at YouTube you could contact to manually review and reverse the decision, but that doesn’t really exist so there’s no one you can really appeal to in a timely manner.

Lots of people making a living off the platform clearly use LLMs to write their scripts. Its kind of weird hearing a person talk to me about something, and then notice characteristic chatgpt patter in their speech.

I'm sure many content creators' videos will be labelled as AI generated. For good reason.

Wouldn't the human creators be the biggest advocates of labeling, so that their content can be more easily found among the AI dross? And that's not considering the fate of the platform as a whole if it descends into low-effort AI spam swamping out everything else. I guess it will be interesting if it is all bots consuming bot-generated content in a parallel economy.

> Wouldn't the human creators be the biggest advocates of labeling, so that their content can be more easily found among the AI dross?

Only if it actually works

Generally things aren't successful unless they work

The entire discussion was centered around whether or not using AI to detect AI content would work, or if it would create false positives that harm human content creators.

It could work "well enough" for YouTube to consider it a success while still harming a fairly large number of content creators.

This definitely needs substantiation. I've NEVER seen such usable tools EVER. AI flagging in general has always been very sketchy IME.

> something that mostly works most of the time

Problem is that at YouTube's scale the remaining "some of the time" ends up being a collossal figure. On top of that, YouTube's effective monopoly position magnifies the damage done by false positives.

Please tell me this is a joke, or that you're not building anything important at work. It's a very well known problem that YouTube's algorithmic moderation hurts a lot of honest creators, and their ability to make a living, when there is a false positive or is abused.

Reply on news.ycombinator.com