OpenAI won't watermark ChatGPT text because its users could get caught
theverge.com
theverge.com
Like, yay we got Sydney back! Everybody gets their own death threats. It's an open-weights wonderland. Why people think it's a good idea to reinstantiate the BPD AI I am afraid I will never understand.
It's either PR or religious ideology (or both if you drink your own Kool-Aid)
"Due to our concerns about malicious applications of the technology, we are not releasing the trained model."
OpenAI said specifically they didn't release it because they thought it was too dangerous to release. There was nothing about precedent setting in there. The idea that this "was and remains" a good thing is hard to understand, given there are dozens of companies that have made much more powerful models available to anyone who signs up and nothing bad has happened. But if you're willing to ascribe intentions to AI companies they never demonstrated having, of course you can argue anything you want. Doesn't mean it's real.
Of course there's an argument to be made that this was inevitable, but that doesn't mean that the early OpenAI crew were necessarily wrong when they said "this model may make things significantly worse".
Or was it the fact that OpenAI has demonstrated how good LLMs can get and that bad actors can train their own, uncensored, unaligned LLMS to do "dangerous" things?
I guess it’s less of a worry now? Why?
Like giving everyone nuclear weapons. Or machine guns. Or bazookas. Or slaughterbots. Or labs to create any sort of virus.
The solution is very, very simple: just regulate it. Force all companies training LLMs to add some method of watermarking with a mean error rate below a set value. OpenAI's concern is that users will switch away to other vendors if they add watermarks? Well, if nobody can provide that service, OpenAI still has the lead. A portion of the market may indeed cease to exist, but in the same vein, if we had always prioritized markets over ethics, nobody would be opposed to having a blooming hitman industry.
Open weights models do exist, but they require much greater investment, which many of the abusers aren't willing to make. Large models already require enough GPU power to be barely competitive with underpaid human labor, and smaller ones seem to already fall into semi-predictable patterns that may not even need watermarking. Ready-made inference APIs can too include some light watermarking, while with general purpose notebooks/VMs, the question may still be open.
Still, it's all about effort to effect ratio. Sometimes inconvenience is enough to approximate practical impossibility for 80% of the users.
How do you watermark plain text?
my example was just a reference to Jarvis in the avengers.
Open weight models do not require great investment. In fact I can run them on my 400 EUR computer.
Also why you want to regulate text output from machines in the name of "public good"? That's insanity.
They also said many other things that never happened. And they never showed it. I bet $100 they do not have a semi-resilient method with 99.9% accuracy, especially with all the evolving issues around "human vs computer" made content.
I bet you also the `semi-` in the beginning leaves a lot of room for interpretation and they are not releasing this for more reasons than "our model is too good".
If it's not possible to watermark, then just ban LLMs.
Tech people have this weird self-serving assumption that the tech must be developed and must used, and if it causes harms that can't be mitigated then we must accept the harm and live with it. It's really an anti-humanist, tech-first POV.
As for the rationality of watermarking itself, firstly I'd like to reiterate, no spam wave of this magnitude and undetectability has ever happened in the history of the web. A word processor cannot write a petabyte of propaganda on its own. A Markov chain can't generate anything convincing enough to fool a human. Transformer-based LLMs are the first of their kind and should be treated as such. There is no quick analogy or a rule of thumb to point to.
If statistical watermarking is proven to have sufficient recall and error, there'll be nothing to lose in implementing it. A demand already exists for detecting AI slop; half-working BERT classifiers and prejudiced human sniff tests already provide for it, with little incentive to reduce false positives. With watermarks, there'll be a less painful, more certain way to catch the worst offenders. Do you really think the same operations that produce papers with titles like "Sorry, as an AI model..." or papers with pieces of ChatGPT UI text will care to roundtrip translate or rewrite entire paragraphs?
We already had this exact dilemma back when email spammers tried Bayesian poisoning [0]. Turns out, it actually creates an identifiable pattern, if not for the system, then for the user on the other side. People will train themselves to look for oddly phrased sentences or the outright nonsense roundtripping produces, abrupt shifts in writing style, and other heuristics, and once the large enough corpus is there, we can talk about training a new classifier, this time on a much more stable pattern with less type-I errors.
I think the biggest concern are state actors who have no problem spending money on some gpus. I don't think it is feasible to watermark open weights LLM outputs.
I can run a specialized AI trained to a certain domain on my laptop that reaches GPT levels and sometimes even beyond that. The work needed here is exactly what malicious actors would also invest and they would have competitive advantage.
I believe we should watch the developments, but I don't believe regulation is warranted yet.
On a society level? I'm stil ok with this. Watermarks are trivially removed by the motivated and unpleasant.
Why do people lift weights at the gym if machines can trivially do that?
I'm not necessarily arguing that there's no more need for any ahead of time "study", but I think that with the equivalent of a personal tutor for everyone, we can achieve a lot more by learning while doing, instead of study being a dedicated activity.
I might be a bit sentimental here, but some of these interactions with an LLM bring back memories of me being a kid working on some small project at home on a Saturday, and being able to come to my dad for advice and even hands-on assistance. And on a less sentimental note, I believe that effective use of an LLM while working on a project fits really well with Papert's Constructionism approach to learning.
And it's not about tech's anti-social influence in this case, but on the intellectual state of humanity as a whole. We've made a whole generation addicted to their phone, and now you want to remove any bit of knowledge?
And because they are entirely uncontrollable they have to add a lot of checks in the prompts. but we all know that hasn't worked. A series of manually built if statements is akin to an expert system. The first thing I learned about those decades ago was that they were mostly a failed experiment
I heard some chatter from OpenAI folks some 1-2 years ago that the way they'd watermark text is that rather than just using random numbers in the top-k/p sampling process, you have a pseudorandom sequence of numbers that either follows a pattern or you save them directly. This way you could fairly trivially build a tool that determines with very high accuracy whether or not a sequence of words has been generated by your model.
I think such a watermark can't be trivially removed unless you rewrite the text, or at least large portions of it.
The functionality that was introduced that would save info about you as a person to "improve" responses cross-chats basically made a whole profile out of you as a person.
Oh, and what a surprise - OpenAI directors since this year are ex-CIA or have very close connections to the agency.
https://www.wsj.com/tech/ai/openai-tool-chatgpt-cheating-wri...
Because I see a lot of the patterns you already see on those. Specifically, paragraphs that start with conjunctive adverbs and phrases. Things like; however, furthermore, moreover, in summary, in conclusion, etc.
Besides, smart students that are not utterly lazy will be able to work around it anyway. They'll let chatGPT (or whatever LLM) turn out an entire paper for the contents, then rewrite it themselves.
So a tool like this will only catch the most obvious cases. Meaning that in the end it only battles a symptom by effectively sweeping it under the carpet.
The AI cat is already out of the bag. I'm for regulation when it comes to AI direst threats, but students using ChatGPT to cheat is a problem that can be solved with live, supervised exams, the kind I had at school in the nineties...we couldn't even use a pocket calculator.
If anything, I'm torn that today's AI systems aren't good enough to do the really serious words. Mark my words, we will have self-aware murder robots before we have AI systems able to write quantum-simulation software.
Do you work at Crowdstrike?
But seriously, I can only imagine how bad their code would have been without ChatGPT.
> In addition, providers will have to design systems in a way that synthetic audio, video, text and images content is marked in a machine-readable format, and detectable as artificially generated or manipulated.
https://ec.europa.eu/commission/presscorner/detail/en/ip_24_...
If we want to watermark GPT, fine — then let’s watermark absolutely everything not directly and personally created by the claimed creator. But we’re getting into interesting legal territory here — work for hire agreements would be in jeopardy because authorship of something under work for hire is owned by then company, not the contractor. There’s also a First Amendment issue — requiring companies and individuals to watermark creative works or disclose uncredited authors amounts to compelled speech that doesn’t serve a public interest high enough to provide an exception to First Amendment protections. The unintended (or perhaps subversively intended) consequences of requiring watermarks can be astounding. Journalists could potentially be compelled to reveal sources for instance because the content they’ve created was partially provided by someone else. It’s a stretch, but then again, the gymnastics courts and prosecutors routinely employ make such scenarios plausible (albeit unlikely.)
It's a problem with the people in education.
We've had hundreds of conversations with different schools and teachers
Imagine if digital cameras were watermarking their photos because art classes refused to consider photography as a form of art.
Remember that next time OpenAI and Sam Altman throw sand in your eyes with “our goal is to empower and serve humanity” or whatever they try to sell you. If they believed what they preach, the choice would’ve been obvious.
If people are just salty about OpenAI and want to buy into the Sam Bad culture war then so be it, but trying to make sense of this contradictory backwards-engineered justification is tiring.
Bullshit rationalisations are bullshit no matter who they come from. Being open-source doesn’t excuse you from it.
> If people are just salty about OpenAI and want to buy into the Sam Bad culture war then so be it
Your mistake is assuming to know why someone you’ve never met has an opinion about something and immediately thinking the worst of it, instead of understanding individuals can think and form opinions for themselves.
I was criticising Sam Altman for his Worldcoin crypto scam way before OpenAI was a worldwide phenomenon.
https://www.technologyreview.com/2022/04/06/1048981/worldcoi...
https://www.buzzfeednews.com/article/richardnieva/worldcoin-...