Hacker News Reader

Comment by simonw | Hacker News Reader

There's a key sentence hidden right at the end of this article:

> They were terrified of Google downranking any AI-generated content, and because of the apparently clear results from their AI checker, they couldn’t be convinced otherwise.

It sounds to me like the client might even have believed the evidence that he provided them showing that he didn't use AI to write the work...

... but they're in the SEO game, and they were paranoid about a (so far hypothetical? I don't think Google have announced this yet) AI detection mechanism that Google might use to downrank them.

At which point whether or not he used AI isn't actually the issue - if his writing style is indistinguishable from AI, then in the client's mind the risk from another hypothetical AI detector within Google means that they shouldn't use the work.

View on HN

This pattern has precedent.

I scan releases with a bunch of Anti-Virus software. Not to find any virus, of course, but if one of the scanning softwares "detects" a virus, some customer is also going to "detect" the virus, and we have a problem.

Yeah. And if you want them to fix their heuristics they tell you to pound sand unless you are willing to pay them a consulting fee to "work with you". Sounds like a protection racket to me, but for some reason it seems to be legal. Will be "great" if we get the same now with "AI detector" companies.

Have you tried Virustotal? Automates pretty much this task.

I’ma Googler, but had to try five search queries to find the name. Obviously didn’t try this in ages, but it used to be straightforward (and list which AV software was used in the scan).

Is it becoming less of a problem nowadays?

Windows has some good enough built in stuff, right? I can’t imagine using a third party scanner.

I think this is an example of a cargo cult in the original sense: people who didn't understand how and why cargo planes came and went recreating what they saw the people with the cargo planes do in order to make them appear again after those people had left.

A lot of institutional "wisdom" in IT is that you need a virus scanner if you run Windows and because Defender ships with Windows it is therefore part of Windows and not a virus scanner. This isn't really based in anything tangible or empiric, just a learned knowledge that Windows needs anti-virus software and that there is good and bad anti-virus software and that's all you need to understand. Of course AV vendors are very guilty of feeding into this because their income depends on this lack of knowledge and being perceived as doing something vitally important.

Realistically the biggest attack vectors these days are (aside from server software vulnerabilities and browser vulnerabilities) someone launching a file they shouldn't have while ignoring every warning message along the way or entering their credentials into a phishing site or telling people things that they shouldn't know. None of these can really be helped with commercial AV software which is why a lot of AV software has moved on to MITM'ing SSL connections to intercept dodgy e-mails or bad links but creating their own potential security issues along the way.

Eh, some kind of tool that detects files rapidly getting encrypted on PCs is useful, as encryption doesn't even need a system level takeover to be a pain in the ass.

People still have anti-virus software though. How good window's defender is is not relevant. What matters is the prevalence of various anti-virus software for your target userbase. Pretty sure various common software install tools give options for installing things like McAfee or Avast. Or they come preinstalled on store-bought systems.

I don’t disagree, I was just curious as to whether or not the situation is getting better, and figured it would be interesting to ask someone with firsthand experience.

Microsoft Defender "detects" a "virus" on any .exe or .dll of unknown provenance.

The solution is code signing. Pay whatever you need to pay for a certificate and sign all your releases.

That's the "smartscreen" function, and it also depends on where the file is marked as having come from (and yes, it largely seems to be a racket to get you to buy cide signing). If defender actively thinks there is a virus through a signature match or heuristics then the behaviour is different, and more difficult to bypas (and yes, this can also have false positives, irritatingly).

That sentence caught me, too. The fear of Google downranking AI-generated content with its left hand, while creating and promoting its own Bard bot-generated content with its right hand. Can an ai-generated snake choke on its own tail? Or is it tails...

Doesn't this mean Google has an AI-detector that is somehow accurately finding these? Otherwise the majority of its index would be falsely flagged.

What do they use?

> Doesn't this mean Google has an AI-detector that is somehow accurately finding these?

No, it means that someone was afraid they would use an AI detector similar to the one that was being used and downrank based on it; nothing in the article suggests Google is in fact using any AI-detector in its ranking algorithm, and I suspect that Google would be quite happy if people used AI (especially Google’s AI) to produce content that otherwise ranked highly on Google algorithms.

Microsoft is probably the best placed company to build a good detection method, having both a giant corpus of text predating LLMs from their Bing crawler, and a giant corpus of AI-generated text thanks to their relationship to OpenAI.

But Google is probably a close second, with an even bigger archive of pre-LLM text (including the world's largest collection of digitized books), and supposedly a good LLM and lots of smart ML engineers.

But no matter what method google finds, whether it's some complex AI trained on massive amounts of training data or a simple n-gram analysis that ends up being telling, they aren't going to tell us. That would just accelerate the arms-race between LLM spam and LLM detection. If they have something that works (big if) the details will stay hidden, the best we are going to get is some kind of press release aimed at dissuading people from using AI on their websites.

It is impossible to create detector for LLM generated writing. The LLM detector technology must beat the LLM at its own game and if it can we can easily create a better LLM. Anyone who claims they can do it is sorely mistaken or flat out lying.

By the same logic I guess you could create one for previous generations of LLMs. "Looks like you need to update buddy...busted!" And across multiple industries a decent percentage of users will be using outdated versions...

Your LLM detection method doesn't have to have a better understanding of human language than the LLM you're detecting, it just has to be able to identify patterns of speech typical of a particular LLM. Of course you have to repeat that work for each network and each significant update and finetune, but right now most people just use one of two ChatGPT versions.

And of course you don't tell people how your LLM detector works, don't provide a public API, and preferably leave it ambiguous whether it even exists. That solves most issues with LLMs using your detection method to get better in future versions.

That's counter to the business model of these LLM detection websites, but Google doesn't have a problem with keeping any potentially existing models inhouse and locked down.

Really all of it is bullshit if we don't know the true/false positive/negative rates.

If you detect LLMs 100% of the time correctly, it is irrelevant if you also incorrectly detect human written text as LLM text 80% of the time. "Detect them all and let god sort them out" will eventually fail as a business plan.

> but they're in the SEO game, and they were paranoid about a (so far hypothetical? I don't think Google have announced this yet) AI detection mechanism that Google might use to downrank them.

Google is probably just saying that scare off the SEO spammers from trying to use GPT4 to automate their jobs. I doubt they have a reliable detector (at least not one that detects based solely on the text content in the page).

Reply on news.ycombinator.com