God I'm choking on the irony of an article about the dangers of using AI to train AI based on a study that used AI to detect AI
They also extracted the workers’ keystrokes in a bid to work out whether they’d copied and pasted their answers, an indicator that they’d generated their responses elsewhere.
So while I don't yet know if the article is bunk -- I do know that your hot take is bunk.
I could have said "they didn't just use the detector all by itself", I suppose.
Can someone in this space invest in doing the hard work to have experts manually curate data?
You know back before Wikipedia, publishers used to pay people to write and edit encyclopedias?
It doesn’t scale. Sure. That’s what the AI you’re building is for though - it will scale.
Throwing compute at ‘the entirety of the internet’ feels like such a lazy way to get what we’re after here.
If GPT4 really is 8 230M models, the next bit for us will be a few ~1-5M models that swap in for whatever you want to create, or talk about, or what have you
Imagine a model trained just on English football for the purpose of having a good time in the pub that is used when the topic changes to it. I bet you could pass on the dailymails sports page if you add some "u"s into your words.
Or a model finetuned specifically on the library you're trying to debug, maybe even specifically in combination with other tools you're trying to put together.
Whatever the merits of this or that “ChatGPT detector”, the concept isn’t unprecedented or ridiculous.
i think the core problem is with the generalist classifiers (gptzero, openai detector, etc). ex. openai's classifier has an accuracy of around 25% on it's own text. however, when you train a bespoke classifier (like the authors did), you can get really good results.
Adversarial training isn't infinitely scalable either, has its limitations also.
Also - the moment that companies start training models to resist detectors, they expose themselves to regulation. Won't stop dark AI models running on some website somewhere, but it can be very effectively applied to companies running at Google or OpenAI scale.
I don't know what feels worse for me - that whenever I read a mannered, well-structured and somewhat verbose comment, I now suspect it wasn't authored by a human - or that, as I quickly realized, my own writing style feels eerily similar to ChatGPT output.
That said, compared to typical comments on-line (even on this site), using paragraphs, proper capitalization, correct punctuation, and avoiding typos already gets you more than half of the way to writing like ChatGPT...
ChatGPT has been RLHFed into a pretty distinctive style, but there's no reason to think a better LLM wouldn't have a more natural style. If AGI is possible, then HN will end up with AI users who contribute on an equal basis to the modal HN user, and then shortly after that, more equal. Should all AI be banned? Should you have to present a birth certificate to create an account?
I actually honestly believe that the era of "open registration" forums and discussion places is going to come to a close, largely due to GNN.
It's not going to become a problem until the hardware and walltime costs of training models and running them comes down. You'll know it's a problem when every 10th post on 4chan is a model pretending to be a human that is of a gentle but unyielding political persuasion of some sort.
I don't know what the end pattern will be, but it'll likely be a combination of things
- large platforms, like reddit or facebook, where individual communities "vibe check" posts out.
or
- some sort of barrier to entry, such as a small amount of money (the so called "idiot tax": if you're an idiot, you get banned, and you have to pay again)
- some sort of (manual!) positive reputation system for discussion boards, sort of like how peering works
- some sort of federation technology where you apply and subscribe to federation networks
I don't think we'll really be able to predict what the future looks like right now (it's not even widely recognized as a problem). And since this is HN, I'll add: I don't think there's any serious money to be made running reputation or IDV, unless you've already started. And if it becomes a serious enough problem, players like ID.me/equifax/bureau will be the situation for "serious" networks (linkedin, facebook, chat, etc).