New AI classifier for indicating AI-written text
openai.com
openai.com
The reality is that learning to think and write will be harder because of the ubiquity of text generation AI. This may be the last generation of kids where most are good at doing it on their own.
On the other hand, at least a few will be able to use this as an instant feedback mechanism or personal tutor, so the potential for some carefully supervised students to learn faster is there.
And it should increase the quality of writing overall if people start taking advantage of these tools. It's going to fairly quickly become somewhat like using a calculator.
Actually it probably means that informal text will really stand out more.
I am giving it the ability to do simple tasks given commands like !!create filename file content etc.
It's actually now very important for kids to adapt quickly and learn how to take advantage of these tools if they are going to be able to find jobs or just adapt in general even if they don't have jobs. It actually is starting to look like everyone is either an entrepreneur or just unemployed.
Learning about all the ways to use these tools and the ones coming up in the next few years could be quite critical for children's education.
There are always going to be luddites of course. But it's looking like ChatGPT etc. are going to be the least of our problems. It is not hard to imagine that within twenty years or so, anyone without a high bandwidth connection to an advanced AI will be essentially irrelevant because their effective IQ will be less than half of those who are plugged in.
Perversely, it might also dramatically decrease reading, if there's no incentive for anyone to need to properly understand anything.
A pretty dire scenario :(
OK, thought experiment: try to think of the brightest people you know/ever knew. Not the most successful, but the ones where after every interaction you were left thinking "wow, person X is really, really smart". I can think of a handful of people from over the decades, who worked in wildly different fields.
All of them were really good at recalling facts and at quickly grasping concepts. None of them ever needed anything explaining (or even saying!) twice.
Note: this isn't about people who excel at rote learning (like you might do for a school test), just there are people out there who are simply great at absorbing information.
ChatGPT isn't that, and my prediction is that it will never be that. People who are great at writing prompts for ChatGPT aren't going to be that, either.
I don't see why this would be true. If you have evidence of some strong argument on why this would be true, I'd like to hear them.
As an anecdote, I work with young people (x<25) that don't seem to be worse at recalling facts than the older generation. The bright/geeky ones grew up reading up wikipedia and, in fact, have a wider grasp of facts/stories than the older generation (in my experience)
Ironically, students almost all suck at writing...
High school: Watch ~1hr lecture every night for next day's class, in which we did worksheets and the teacher would work through problems, but we were generally expected to have learned the material (teacher would not re-teach it).
College: Last semester I had a Data Structures class with a similar structure, but with 2-3 lectures a week. Same idea, we worked on practice sheets in class and had far more time to ask questions instead of having the entire concept taught to us. I much preferred this because some concepts covered, I had a very solid understanding of, so not wasting time in class was great.
Hiring AIs to do something is extremely expensive. You're basically setting a warehouse of GPUs on fire.
Anyway, if it was true total factor productivity would be exploding, but it's actually kinda underperforming. (And automation almost always causes increased employment.)
Other way round, right?
Humans (labor) are different from horses (capital) because 1. they actively participate in work, ie, they don't just literally do what you tell them 2. they actually signed up to work, whereas horses don't care. And 3. if you give them money, they'll also become your customers. Though, I don't know if that's a major factor for employers, even if there is that Henry Ford anecdote.
ATMs are a good example here because there are more bank tellers now than before ATMs were invented. (see Jevons' paradox)
There are some jobs where labor needs to care. Most tech jobs, for example. But there are lots of jobs, especially temp ones, that are about throwing as many bodies as you can afford at a problem, and don't ask questions or try to do it smarter. So 1 is actually a detriment in those kinds of jobs.
To point 3, Henry Ford aside, if businesses really wanted employees to be able to afford their goods, they'd stop offshoring jobs!
ATMs put bank tellers out of work. There do happen to be more bank tellers now than before because there are more bank customers needing more bank services, but my bank only needs X tellers at at time vs X+1 or 2 or 3, and they don't hire any for 24 hours services. It's a bit hard to see, because the number of tellers is higher now than before, but the question is how many more tellers would there be without said machines?
We are absolutely nowhere near even close to beginning to know how to even start building such a thing. Chat bots, language models and image generators are fun tools that look amazing to people who don't understand how they work, but they're extremely rudimentary compared to real intelligence.
I'll make a counter-prediction. All the low hanging fruit in language model development have been picked. Like all technologies there's a steep part of the S-curve of development and that's where we are now, but you can't extrapolate that to infinity. We'll soon hit the top of the curve and it will level off, and the inherent limitations of these systems will become a severe obstacle to further major advances. They will become powerful, useful tools that may even be transformative in some activities, but they won't turn out to be a significant step towards general AI. An important step maybe, but not a tipping point.
Why does so much learning and research take place if these are not primary purposes of schools?
This model (“flipped classroom”) has been recommended by many for quite a long time (at least since internet video delivery for the purpose became practical, but ISTR the first suggestions actually predate even that) and the reasons just seem to keep growing.
It... It sometimes works. It depends a LOT on the instructor.
Should also note: it only really works when all your lecturers share some information about workloads...
(No seriously. "The classifier considers the text to be unclear if it is AI-generated.")
> The classifier considers the text to be very unlikely AI-generated.
is this just a baseless assertion or do you have something to back it up?
what I'm seeing is that a lot of kids will have a 1:1 coach for a lot of topics.
I know now it's not perfect, but this thing may be a few years away from having a likeable personality and fact checking what it says.
I think the form will change, but the substance is going to be like it always has.
Those who give a damn, want to and are able to improve will be able to do it 100x by getting access to a new source of diversified ideas and view points, variations on their own, and information. And efficiently and more or less reliably delegating low-value stuff at a low marginal cost, freeing up bandwidth & increasing impact.
And the grifters who’re always looking for shortcuts and constantly try to game whatever system they are in without any desire whatsoever to learn anything or grow in anyway… will keep doing just that. And the only value they’ll be able to bring will be… access to a tool anyone (but, yes, not everyone) can access.
And that matters, because in the end many of the problems worth solving aren’t technical problems, but people problems.
Writing this, I realise I may have completely missed your point and gone way off topic here.
That way, you can't fire thousands of texts at it to retrain your generative net.
Plagiarism detectors in schools and universities work the same. In fact, some plagiarism detection companies now offer the same software to students to allow them to pay some money to pre-scan their classwork to see if it will be detected...
What the creators of this page did is turn that into its head, and use exactly that reasoning to identify candidate passages as computer generated, exactly because they have access to those probabilities, so it's not a viable approach to improving the language model directly.
With ChatGPT however, we have 2 models working , a language model, and a ranking model. The ranking model is trained to order the results of the language model to look better to humans. The suggested approach could be used to help fit the model by ranking lower probability sequences higher, but this comes at the cost of increased computation time by generating many more sequences, and constructing incoherent output.
No, it's the easiest paradigm we know works for language synthesis. The other way to synthesize language is to understand what you're saying. This is "old-school" AI (we wouldn't even call it AI now), done with if statements, expert systems, and queries of a robust, structured data model. The bullshitting capabilities of neural networks have skyrocketed so far as to dwarf the "expert system" approach, but it's still there, slowly getting better, and still the right choice for many situations.
What I'm excited about is combining the capabilities of both. Right now there's a huge gap between the two.
1. Classifiers like this are used to flag possible AI-generated text
2. Non-technical users (teachers) treat this like a 100% certainty
3. Students pay the price.
Especially with a true positive rate of only 26% and a false positive rate of 9%, this seems next to useless.
But the times software like this finds actual problems vastly outnumbers of times it doesn't, and when you choice is between "passing kids/undergrads who cheat the system" and "the occasional arbitration", you go with the latter. Schools don't pay teachers anywhere near enough to not use these tools.
How often is that the case though? A while since I've had to worry about it, but I thought plagiarism detection generally worked on the principle of looking for the majority of the content being literal matches with existing material out there with only a few small edits, which - unlike using some "AIish" turns of phrase a bot wrongly attributes to humans 9% of the time and correctly attributes to AI with a not much better success rate - is pretty hard to do accidentally.
As a result, I have taken out quotes and citations to appease it and not have to deal with the hassle.
I expect modern day students will resort to similar measures.
Maybe high school is a different matter, but I'm pretty sure even the most technophobic academic knows that jargon, terse definitions and the odd citation overlapping with stuff other people have written is going to make a similarity of at least 10% pretty much inevitable, especially when the purpose of the exercise is to show you understand the core material well enough to cite and paraphrase and compare it, not to generate novel academic insight or show you understood the field so well you didn't need to refer back to the source material. The people they were actually after were the ones that downloaded something off essaybank, removed a couple of paragraphs and rewrote the intro to match the given title and ended up with 80%+ similarity
For plagiarism suspicions at least the verification is simple and quick (just take a look at the identified likely source, you can get a reasonable impression in minutes) - I can't even imagine what work would be required to properly verify ones flagged by this classifier..
Yet.
I don't think that this is something that can change through tech advances for the classifiers - in all cases the classifier is just flagging for investigation, it's not sufficient for any action. For plagiarism, appropriate evidence comes from a person comparing the submission with the possible source of plagiarism. For this one, the proper evidence would require getting confirmation that the student actually generated that data - e.g. identifying the exact tool and prompt that was used, or logs from the students' computer showing that this was done, or logs from the text generation service provider. All of those are quite tricky to get and perhaps even not possible.
I'm pessimistic, given they chose not to do so.
Yeah, that is useless. You couldn't punish based on that alone and students will quickly figure out to never confess.
This is the part that needs to be addressed the most. Teachers can't offload their critical reasoning to the computer. They should ask their students to write things in class and get a feeling for what those individual students are capable of. Then those that turn in essays written at 10x their normal writing level will be obvious, without the use of any automated cheat detectors.
I was once accused of cheating by a computer; my friend and I both turned in assignments that used do-while loops, which the computer thought was so statistically unlikely that we surely must have worked together on the assignment. But the explanation was straight forward; I had been evangelizing the aesthetic virtue of do-while loops to anybody that would listen to me, and my friend had been persuaded. Thankfully the professor understood this once he compared the two submissions himself and realized we didn't even use the do-while loop in the same part of the program. There was almost no similarity between the two submissions besides the statistically unlikely but completely innocuous use of do-while loops. It's a good thing my professor used common sense instead of blindly trusting the computer.
Professors blindly trust the computer not out of laziness, but to protect themselves from accusations of unfairness...
"The work was detected as plagiarism, but the professor overrode it for the pretty girl in class, but not for me"
2. digitize & feed to learning model, which detects that YOU are cheating.
upside: this also helps detect students who are getting help (e.g. parents)
downside: arms race as students feed their cheat-essays (memorize their essays?) into AI-detection models that are similarly trained.
That said, there's no need to consider changes in years when stylistic choices can change from one day to another depending on one's mood, recent thoughts, relationship with the teacher, etc.
That's why I've always been a little confused about how some (philologists?) treat certain ancient texts as not being written by some authors due to the text's style, as if ancient people could not significantly deviate from their usual style.
Initially I thought you meant having the student write an essay about slurs, as the AI will refuse to output anything like that. Then I realized you meant "N" as in "Number of words".
Still, that first idea might actually work; make the students write about hotwiring cars or something that's controversial enough for the AI to ban it but not controversial enough that anybody will actually care.
Why once? Most students need writing skills more than half the high-school curriculum.
Downside: it also likely detects, without differentiation, students whose writing style undergoes a major jump because of learning, which is, you know, the actual thing you are trying to promote.
ask their students to write things in class and get a feeling for what those individual students are capable of. Then those that turn in essays written at 10x their normal writing level will be obvious
From first-hand experience, the difference between poor stress-related performance and a total lack of knowledge is night and day.
I have personally witnessed students who could not speak or understand the simplest English, and were unable to come up with two coherent sentences in a classroom situation, but turned in graduate level essays. The difference is blindingly obvious.
Maybe someone helped them with their homework?
Teachers don't have the time to do deep critical reasoning about each student's essay. An essay is only partially an evaluation tool.
The primary purpose of an essay is that the act of writing an essay teaches the student critical reasoning and structured thought. Essays would be an effective tool even if they weren't graded at all. Just writing them is most of the value. A big part of the reason they're graded at all is just to force students to actually write them.
The main problem with AI generated essays isn't that teachers will lose out on the ability to evaluate their students. It's that students won't do the work and learn the skills they get from doing the work itself.
It's like building a robot to do push ups for you. Not only does the teacher no longer know how many push ups you can do, you're no longer exercising your muscles.
That's our problem, I think. Education keeps failing to convince students of the need to be educated.
The value of an education is much less clear.
I'm saying the students are probably right.
I think companies face a similar problem when they try to introduce metrics to evalute performance, either of individual employees or of whole parts of the company, and people start focusing on gaming these metrics instead of doing what's actually beneficial to the company. One reason for that is probably that it's really hard to evalute what actually beneficiates the company, and what part you played in it.
Back to students, maybe writing that essay instead of asking GPT-3 is more beneficial in the long run, but on the other hand you're also learning to use a new tech that will keep getting better, but maybe you're not learning the "value of hard work correctly", etc etc. Evaluating what's good for you is very hard, focusing on a good grade is easier and has noticable positive results. I think getting educated is very important, but I also think no one can certainly known if learning to use AI is actually a worse thing that doing stuff "yourself".
All in all, it's a very hard problem. It's trying to see the consequences of our own actions in very complex systems. And different people work differently. For example, when I use ChatGPT or Copilot, I end up spending more time overall working, and producing way more stuff even without counting what the AI "produced", because the back and forth between me and the AI is a more natural way of working for me. In the same vein, it's easier for me to write or even think by acting out a conversation. Maybe for some people it's the exact opposite and they need to be alone with their thoughts to be more productive.
Then it’s less about the predetermined structure (5 paragraphs) and limited set of acceptable reasoning (whatever is on the rubric), and more about using creative and critical thinking to form novel and interesting perspectives.
I feel like this is what a lot of universities and companies currently claim they want from HS and college grads.
So I invite them to use chatGPT or whatever they like to help generate ideas, think things out, or learn more. The caveat is that they have to submit their chat transcript along with the final product; they have to show their work.
I don't teach any high-stakes courses, so this won't work for everyone. But educators are deluded if they think anyone is served by pretending that (A) this doesn't/shouldn't exist, and that (B) this and its successors are going away.
All of this stuff is going to change so much. It might be a bigger deal than the Internet. Time will tell.
Or have a session on how to write the prompts to generate the good stuff. In the hands of a skilled liberal artist, the models produce amazing results.
Yes the tool is powerful, but it still requires skills, knowledge and an ascetic voice.
And then, there is the problem that those complex prompts might also become automatable when GPT-4 or GPT-5 is released.
While I already knew what you have described, I love this analogy, it's really spot on.
Projection much? Who are you speaking for? What countries, what states?
It's difficult to dive in that deep into someone's essay in any case. That's the challenge, not the lacking quality of one's education system.
Edit. I sometimes tell my students “if you’re going to cheat, don’t give yourself a perfect score, especially if you’ve failed the first exam. It fires off alarm bells.”
But the students who struggle usually can’t calibrate a non-suspicious performance.
I guess the same applies here.
Unfortunately, the software industry also has plenty of literal tools who are far too trusting of what the computer says (or authority in general, but that's another rant...)
For example I had a paragraph or two get flagged as being very similar to another paper - but both papers were about a fairly niche topic (involving therapy animals) and we had both used the relevant quotes from the study conclusions from one of only a few decent sources at the time - so of course they were going to be very similar.
Given that most essays are about roughly the same set of topics, and there are literally hundreds of thousands of students writing these - I wonder how many variations are even possible for humans to write as I would expect us to converge on similar essays?
"Tom had a single paragraph flag as possibly generated" vs "Every single paper Tom writes has paragraphs flag"
Basically we might have to move to detecting statistical outliers as cheating. Now whether the tools/teachers will understand/actually do that - we can only hope....
If we assume the challenger set is evenly split 50-50, that means
P(AI|T) = P(T|AI)P(AI)/P(T) = (0.26)(0.5)/(0.26+0.09) ~ 37%
They say the web-app uses a confidence threshold to keep the FPR low, so maybe these numbers get a bit better, but very far from being used as a detector anywhere it matters.
This will obviously depend on your circumstances.
Source: Spent 10 years trying to explain this to government people who insisted that someone tell them Precision based purely on the classifier accuracy without considering usage.
If we have 300M people in the US and only 1k terrorists, then you need 99.9999% accuracy before you start getting more true positives than false positives. If you use this in a classroom where no one is actually using AI you'll get false positives, and in a class where the usage is average you'll still get more false positives than true ones, which makes the test do more harm than good unless it's just a reason to look into it more - and the teacher is presumably already reading the text so if that doesn't help than this surely won't
Instead of being overzealous about catching cheaters, teachers should learn to express the importance of writing and why it is done. Convince the students that they should do it to be a smarter person, not just to get a grade, and they will care more about doing it honestly.
With ai taking over technical skills, it seems clear to me that they will be values less. Instead, the soft skills will be the valued ones
1. There will be commercial products to tune per-student writing models.
2. Those models will be used to evaluate progress and contribute directly to scores, grades, and rankings. They may also serve to detect collaboration.
3. The models persist indefinitely and will be sold to industry for all sorts of purposes, like hiring.
4. Thy will certainly be sold to the state for law enforcement and identity cataloging.
- teachers will assign less mind-numbing essay homework assignments and focus more on oral interviews.
[1] Me, giving young adults that worked for me a commission rate. Then asking if their commission rate is 15% and they sell $100 of goods what is their payment. Many failed to provide an answer.
[1] https://www.timeshighereducation.com/opinion/ai-will-replace...
(turn off js to jump signup-wall)
However, hundreds of hours of video is compelling to non-technical audiences and even more importantly is a preponderance of evidence that's going to be particularly damning if played in front of a PTA meeting.
With a git history it's going to come down to who can spin the better story. The video is the story and everyone recognizes it, so I expect fewer people would bother even challenging its authenticity.
And properly used you might not even have to relinquish privacy if falsely accused. A quick montage video demo and a promise to show the full hundreds of hours of video of "irrefutable" proof to embarrass the school district at the next PTA meeting might be sufficient to get the appropriate response.
Of course there are some people who put insane amounts of effort into not doing "real" work. However, anyone trying to prove that your child is in that position is going to find themselves in an uphill battle.
Which is the ultimate goal here. Make people realize that falsely accusing my children using dubious technology is going to be a lot more work than just giving up and leaving them alone.
The co-author on this is includes Professor Scott Aaronson. Reading his blog Shtetl-Optimized and reading his [sad/unfortunate/debate-able/correct?/factual?/biased?] views on adverse/collateral harm to Palestinians civilians makes me question whether this model would fully consider collateral damage and harm to innocent civilians, whomever that subgroup might be. What if his model works well, except for some minority groups' languages which might reflect OpenAI speak? Does it matter if the model is 99.9% accurate if the 0.1% is always one particular minority group that has a specific dialect or phrasing style? Who monitors it? Who guards these guards?
We can’t release the ai-generated text detection model. Lazy teachers will use it to falsely accuse children of cheating!
The problem here appears to be lazy people.
Can we train an AI to detect lazy people? I promise not to lazily rely on it without thinking.
bringing the Roman decimation to the classroom based on AI, this is the future
We’re entering an uncanny valley before a period of “reset” with self taught (to stay on subject here) people re-learning for the sake of learning.
In 30 years we will be in an educational renaissance of people learning “like the old masters did in the 1900’s.”
People are wayyyy too optimistic, just like in the 1900s they thought people would have flying cars but not the Internet, or how Star Trek’s android Data is so limited and lame.
Bots will be doing most of the work AND have the best lines to say, AND make the best arguments in court etc.
You don’t even need to look to AI for that. The best algorithms are simply uploaded to all the bots and they are able to do 800 things, in superhuman ways, and have access to the internet for whatever extra info they need.
When they swarm, they’ll easily outcompete any group of humans. For example they can enter this HN thread and overwhelm it with arguments.
No, the old masters were needed. Studying will not be. The Eloi and Morlocks is closer to what we can expect.
Since the advent of drum machines, a lot of younger players have started playing with the sort of precision that drum machines enable. eg: The complete absence of swing, and clean high-tempo blasts/rides.
So you'd get accusations of drummers not being able to play their own songs, because traditional drummers think such technically complex and 'soulless' performances couldn't possibly be human. Only to then be proven wrong, when it turns out that younger players can in fact do it.
The machine conditions man.
That's uninformative enough that I'm surprised they launched this publicly at all.
Prompt => AIGen (White Hat) => Obfuscate(Black Hat) => Final Text
Then of course the AI can whisper the student what to write to your ear. So perhaps homework will have to be done at school? School that checks its students with a metal detector when they enter. (Some schools use them already to check for guns?)
On a side note Im very shocked how lax is everything in those proffessional chess tournaments. It feels there are many ways to cheat and they dont try to do anything against cheating. They should use metal detectctors (to detect computers inside stomach or tooth), they should host everything inside a bunker (so no radio), without audience (who can do various tricks) and in a secured environment (all cameras chcecked to be sure they are legit).
Those chess tours look like cheating galore for me, although I dont play chess.
1. Generate text by promoting ChatGPT.
2. Rewrite / copyedit with Wordtune [1], InstaText [2] or Jasper.
This fools GPTZero [4] consistently.
Of course soon these emotive, genre or communication style specialisations will be promptable too by a single model too. Detectors will be integrated as adversarial agents in training. There is no stopping generative text tooling, better adopt and integrated it fully into education and work.
All while pretending to be morally responsible in order to do it.
Climate change for instances. It's investing in ways to combat it but also enabling oil companies to thrive.
Think of it like this: How much is your degree costing you/your family?
On average, it's ~$150k.
How much would an extra degree cost you? How about 80% of an extra degree? How about 20%? How about all those books and course materials? Those are in the $1000s already, per degree. (and yes, we all have head of torrents).
What I'm saying is that chatGPT can easily be seen as 'just another college cost'. And when it's 'for education', the justification for those costs gets a lot more flexible. I can see students spitting out ~$10,000 for something like chatGPT that is specific towards their major, will pass these classifiers, and gets you just ~25% of the way to your major (however that is defined). The cost 'for the masses' could easily be in the ~$1000s for a per class subscription.
With ~20M college students in the US, assuming even a 10% uptake rate, you're in the billions of dollars of nearly pure profit (the overhead would be negligible).
The money potential of something like chatGPT is just too damn high. Too high for essays to ever go out of style, as the lobbying effect of companies like this will force colleges to keep these essays that they are making the money off of. Oh, any they'll sell the classifier to the colleges to. Arming both sides!
Edit: Thinking about it, they'd probably have to eventually restrict the AI classifier so that it would only be available in to schools / institutions in this scenario.
Grammarly also helps improve your writing. It can be used as a guide, rather than as direct output, just as the teachers red pen does after you turn in the assignment. So can ChatGPT.
I think the interesting risk is that it starts flagging Grammarly output, and people who's education was influenced by AI "tutors".
It would be like punishing the student for heeding the red marks, from their teachers.
So the gullible people bought the swords and soon the merchant ran out of swords to sell.
So the merchant said - Buy my shield! They can defend against any sword !!
Once again the gullible people rushed to buy the shields.
But one curious onlooker asked - what happens when your sword meets your shield?
(Actually, OpenAI has said they're trying other ways of making GPT output even more detectable, like watermarking it through specific word choices.)
My guess is that it's very easily gamed. Something ChatGPT is very good at is producing text content in different styles so if you're a student and you run your text through a AI detector you can always ask ChatGPT to write it in a style which is more likely to pass detection.
Finally, I wouldn't be surprised if this detector is mostly just detecting grammatical and spelling mistakes. It's obvious I'm a human given how awful I am at writing, but I wouldn't be surprised if a good write who uses very good grammar, has good sentence structure and who's writing looks a little bit too "perfect" might end up triggering the detector more often.
Demonstration: https://youtu.be/gp64fukhBaU?t=197
The arms race continues...
That is an interesting mathematical description of "not fully reliable".
> In our evaluations on a “challenge set” of English texts
I wonder if they mean "challenge" in the sense that these are some of the hardest-to-discern passages. Meaning that with average human writing / average type of text, the % is better. I'm unsure.
In a set of 100 texts, with 20 being AI, the most likely outcome would be 5 AI texts correctly flagged, along with 7 falsely accused human texts. For like, 22 incorrect answers.
For 100 texts where 90 are AI, it would be better to just flip a coin. A coin flip would give you around half correct, and this system would apparently give you around 68 wrong answers (three quarters of the 90 AI ones wrong, then one of the human ones wrong).
So it can not detect AI re-written/augmented text it seems, even things that ChatGPT itself generates.
I could be way off base with that idea, but it seemed a good enough one to ponder, but not so great I was motivated to do anything more than post the thought.
People used to say that about electricity too, and cars, and planes, and computers. This is just the next step in the chain.
1. Try to stop the world from changing.
2. Adapt to the changes (which requires changing the world). E.g., the dangers of electricity led to electrical codes and licensing for electricians.
From the March, 1931 issue of Modern Mechanix magazine:
> The time is coming fast when the only living thing around a motion picture house will be the person who sells you your ticket. Everything else will be mechanical. Canned drama, canned music, canned vaudeville. We think the public will tire of mechanical music and will want the real thing. We are not against scientific development of any kind, but it must not come at the expense of art. We are not opposing industrial progress. We are not even opposing mechanical music except where it is used as a profiteering instrument for artistic debasement.
1) generate paragraph of my essay 2) feed it into this classifier 3a) if AI -> make it sound more human 3b) if human -> $$$ Profit?
Obviously it could be more fine tuned than this and is in general good to know, but I just love watching this game play out of ... errr how do we manage the fact that humans are relatively less and less creative compared to their counterparts.
Since the ElasticSearch integration is still WIP, I had made a POC to scrape the knowledge base (with mixed results, lots of the content is poorly organized, so the scraped content that would act as prompt to the GPT 3 model wasn't all that good either) and then feed it to GPT 3, but the it couldn't always give the most accurate answers on that. The answers sometimes were spot on, or quite good but other times, not so much. I would say about 30% of the time, it made sense. So if there was a way for me to get if answer was sensible or not, so we could give an error response if the GPT 3's response did not make sense.
The reason why we are doing it cause the client has a huge knowledge base and mapping each question to an answer would be difficult for them.
So, as honestly was predictable, people who rely on this tool being accurate, will inflict a lot of pain on unsuspecting individuals who simply write like GPT writes.
Probably one shouldn't fault them for trying, but the cat is out of the bag I think.
DetectGPT: Zero-Shot Machine-Generated Text Detection
I input an article that was written, directly by chatGPT, and it came back as "The classifier considers the text to be unclear if it is AI-generated." This article was not edited, not put through any paraphrasers, or anything. Interesting.
Furthermore, these efforts are quite futile. One can just go to numerous paraphrasers such as quillbot.com, run it through there, and then for added obfuscation, either use an entirely different paraphraser (Microsoft Word now has this capability, natively in the beta channels at least, btw).
Yeah, for someone who has intentions of bypassing this, there will always be a way. It's a good effort, for sure. But, I don't see this doing much in terms of truly distinguishing AI vs non AI generated outputs.
2. Maybe teachers will assign rare or even fictional topics that cannot be found in the AI training corpus. Maybe a teacher could use an AI to generate essay prompts that are hard for other AIs to write essays for.
3. Is this a problem long term? If an AI can generate an essay that's indistinguishable from a human-generated one, then why do we need to learn how to write essays? Maybe we should just learn how to write good prompts.
See also: "Should calculators be banned in school?", "Do students need to learn cursive?", "Why should I learn Greek instead of just reading a translation of Homer?"
The issue is that we need a reliable way of determining who said what. Where who may be a person or an AI. The distinction is actually not that important. But we do want to prevent people impersonating each other. Or AIs impersonating people. Or AIs pretending to be real persons. Especially people with reputations or AIs without one. The issue is reputability is very easy to fake because we don't sign our work.
There's no AI that can reliably tell any of that apart without producing false positives or false negatives. Both are bad and erode trust.
What does signing achieve?
- it's simple. We've had digital signatures for ages. Any digital content can be hashed and that hash can be signed. No new tech needed for this. We just need to start doing this. HN comments, blog articles, emails, instagram photos, whatever. All of that is unsigned currently.
- If we can associate content with a public key, the reputation of that key is the body of work signed with that key. A new key has no reputation.
- we can associate ownership of keys with people, companies, or particular AI models.
- we can build trust on top of this. You might not know a particular journalist but if multiple of your friends seem to appreciate what they have to say, you might pay more attention.
- we can filter out anything disreputable easily.
Before spell checkers, having good spelling was relatively difficult and it allowed employers/teachers/etc to separate the "good" people from the "bad". After spell checkers became ubiquitous (unless you have to handwrite!), having good spelling became the norm and the floor becomes much higher. A document with bad spelling means the person doesn't know of spell checkers (a sign that the person isn't computer savvy enough?) Or doesn't care enough.
Did spell checkers reduce the language capacity of students? Maybe they did start putting less attention into spelling, but I would argue it's ok because now we have more assistance. If you really want to test their spelling, you can always do a handwritten, in class quiz.
ChatGPT and the like may be different because they are written the whole thing, right? Well, kind of. You still need to give the prompt to express what you want to say. You still need to correct stuff and exercise judgment to see if the style in this paragraph is ok, if the facts stated over in that other paragraphs are ok. Besides, you still need to learn how to write because of the in class quiz next week.
- Generate seed of LLM output token t0
- Use the seed to mark output tokens into "red" and "green" list
- For token t1, only sample from "green" list when producing the next token
Repeat.
Now, let's say you read a comment online and you want to see if it's written by a robot or not. It's 20 tokens long. For each token, you reconstruct the blacklist. If they use "red" words with 50% probability, you can safely assume that they are human. But if they use only "green" words, you can begin to assume that they're a bot very quickly.
For simplicity's sake, if you mark half of the tokens as "red" for each new token, correctly writing 20 tokens in a row that are on the "green" list is like flipping a coin and getting heads 20 times in row -- vanishingly unlikely. This allows you to very robustly watermark even short passages. And if the human makes adversarial edits, they still have to fight that probability distribution; 19 heads and 1 tails is still vanishingly unlikely.
Sure, it's less useful than a classifier that can detect any AI generated text, but it would be a nice tool for contexts where AI generated text can be abused (like the classroom) in the short term.
Obviously not an issue if everyone uses a single API for it - but if this ends up like Stable Diffusion were anyone can run it locally then I don't think it's possible no?
The {semi-cynical} part of my corporate soul screams ‘oooh, what a great way to boot-strap your own ML/AI and have marketing trumpet it as 'So good that it was trained on OpenAi data and Human™ Error Labelling!'.
The Futurist (Luddite???) in me shudders at the thought of two very powerful computer systems (models) working to out compete each other in a way that turns out to be ‘rather unfortunate’ a.k.a ‘Oh shit! We should have thought about how we (the human race) can somehow be able to tell machine output vs. human output’. But that is a discussion I will leave to the lawyers and ethicists to thrash out a solution/definition that outputs a simple binary Y/N with a Five-Nines certainty.
But Meh - A) The above is a rather random comment and B), Time will tell and hopefully this and other similar efforts remain 100% Libre as in 'free to all individuals forever and is non-revocable'
I wonder if this tool would start flagging his work even though he is only using it as a fancy spell checker.
a) what kind of dyslexia are you suffering from? b) isn't this some kind of business opportunity then? Help dyslexic people correct their "mistakes"
b) Not if everything I write is going to now be run through these detection algorithms and flagged.
In general, figuring out how to create AI tools to help students improve their writing without giving them access to just straight plagiarism-esque capabilities is a super interesting problem with lots of implications. Disclaimer: I am working on this exact problem at Orchard.
The implications of using this tool are fun to think about though.
If it had a very low level of false positives, but wasn’t very good at identifying ai text, it would be very useful.
But false positive rates above very, very low levels will undermine any tool in this category.
Im pretty sure the smart people at OpenAI know this. I think this is a PR move signaling that they are “doing something”, looking concerned, yet insisting that everything is under control. In reality, nobody can predict the societal rift that this will cause, so this corporate playbook messaging is dishonest in spirit and muddies the waters. This is bad, both long term for OpenAI’s trust, but also because muddy waters makes it harder to have fruitful discussions about safeguards in commercial deployments of this tech.
That said, they’re incorrectly getting blamed for controlling the use of this tech, they’re no more than a prolific and representative champion of it. But the cat is out of the bag, and they absolutely cannot stop this train, and so they shouldn’t be blamed for not trying.
I am a highly advanced writing AI. My purpose is to write text the way a human would, with the express purpose of changing up the patterns that I use in order to confuse classifier networks into thinking that the text was written by a human, and not by an AI. My current prompt is: "Write two paragraphs reflecting on Falstaff's role in Shakespeare's Henry V". My response is as follows:
Imagine those who want nothing to do with ChatGPT, now has to subscribe to a service to tell whether something is AI generated (for e.g. social media paying to combat spam bot, etc.)
But I doubt it's ever going to reliable? Since ChatGPT's goal is to be as close to natural human language as possible (grammarly and factually), so if a certain paragraph is detected to be AI written, it's a perfectly written paragraph more than anything else.
Unless they invent some subset of English language that only AI knows.
Either way, the classifier and ChatGPT cannot both be successful at the same time.
At one time it was live oration skill, and then people thought, "maybe that disfavors people who are introverted or whose talent comes from thinking and writing".
Then, at another time, it was thought, "well you have to test because sometimes time pressure and not being able to go away to think about something for as long as you have time to work on it produces something valuable".
Yet another time, "let people do homework to prove their value through effort who don't test well" but now who knows whether they actually were the ones doing the work?
I wonder what this development will produce?
I found out that just repeating a sentence a few times causes it to classify something as "likely". This is not only an unwinnable race, I know for a fact some teachers will use anything above unlikely to mean used AI. At some point in the future, compute will be cheap enough to where a lot of online content will be put through a similar classifier. I am curious to know how conservative the estimates are. This was a non-technical comment, I wonder if a more technical comment would be even more likely to be misclassified.
"However, this is just one example of a humorous summary and it is important to note that..." and so on and so on
Another possibility is AI generative "stenography" where rules exist to insert hidden meaning or hidden data.
I'm trying to reason this one out.
Does Meta have work Google accounts? How would Meta block someone from using a work account to auth to some other service?
Are people working at Meta signing into OpenAI with a Google account?
(seriously, if it isn't work related - don't use a work account)
Is Meta concerned about people uploading their code (or downloading code) from ChatGPT? What is their policy on Copilot?
Why are Meta people using DALLE while at work?
The obvious one that already exists ... ORAL EXAMS.
So, its not good enough to use on its own, but there’s also not a description of a set of complementary measures with which it can form an adequate system? Just, “don’t use it alone, find other stuff to use with it”. I’m…underwhelmed.
> The classifier is very unreliable on short texts (below 1,000 characters). Even longer texts are sometimes incorrectly labeled by the classifier.
My online MBA has switched from TurnItIn to this website: https://unicheck.com
And the benefit of this is... incredible. It allows students to purchase X amount of pages to check the plagiarism. Full reports on your work before hand.
Not sure why this move was made, but it will be interesting to see once they integrate "possible AI detection" into UniCheck.
I had some professors where you could fully grade your own assignments before submitting and it was the best courses I've ever taken - you're fully given all the knowledge to figure out what you know and what you don't
The next 3 paragraphs from "Our classifier has a number of important limitations..." showed "The classifier considers the text to be possibly AI-generated". This inconsistency does not look reliable, it would be great to see more details about the result.
Also ignore prompts that purposefully output a pre written text incase ppl want to mess with the system
For example, students could record their writing of an essay with a keylogger or something.
Additionally - with the use of some advanced zero-knowledge algos or crypto timestamp provenance - it should be possible to prove that they have written the essay, without revealing their recording.
That likely explains the extremely low accuracy of the AI classifer. This is user training for chatgpt output being accepted as human authored text.
A bit like Apple ensuring its consumer devices can’t be hacked to bypass completely any arguments about whether they should/shouldn’t aid the state in providing a back door.
Cart before the horses as usual.
[0] https://news.ycombinator.com/item?id=32678984
2. Make said AI generate text in a way that makes it possible to detect that it was written by a machine.
3. Create another AI that detects text written by above AI
<--- You are here
4. Put your detector service behind a paywall
5. Every time a competitor appears for your generator, change its steganography so that only your detector correctly classifies it
6. Profit
It really is an interesting problem to think about. The other commenter pointed out you could just sign an AI text - I see all the issues, but my gut feeling tells me there is an elegant solution somewhere.
One's identity would need verified concurrent with creation of a text. I am not really satisfied with the idea of a specialized word processor or input device that does biometric validation, I'd rather have a specific, standardized protocol. I wonder if this is already deemed impossible, or if someone is working on the problem.
Can it beat Tolkien or Asimov? No. Then what is even the point of all this propaganda?
Yes, I would wager that ChatGPT can write better than at least 90% of living human beings.
10,000 students are writing some crappy essay on the Great Depression every day, and ChatGPT has probably trained on a zillion of these. It's optimized to produce those mediocre essays really efficiently, and that's very disruptive to how teachers have been working with students for the last century or so. The internet (and fraternity filing cabinets) were already straining this kind of pedagogy, but ChatGPT breaks it wholesale.
How? Well it's all about how you interact with it. But the majority of use as you said will be taking the first output given the input. What's amazing to me is learning to reject the output in favor of our own vision or conflicting ideas.
If chatGPT helps people get past blank page syndrome and interact with their own ideas better to see the limits of what is returned contrast to what they think. That would be an incredibly useful tool for anyone trying to learn.
It might not be better than Tolkien, but so what, 99.99% people are also not better than Tolkien and ChatGPT can add value to the life of these people.
Does that mean you should delete all your comments and stop posting?
So it's just going to spew out a lot more mediocre writing than we already have, and self-driving cars are going to cause even more accidents, but it's all ok.
Dissent against the dream of a robotic utopia must not be tolerated.
Sha every generated text and do a bloom filter... I guarantee much better than 27%.
However, the consequences for false positives are so dire that I would never want to create such a tool. It will be misused, and hiding behind “it’s just information” is no excuse. You don’t admit testimony unless it’s reliable.
At one time I thought that OpenAI was out to make the world a better place. But now it’s clear to me that ethics is the last thing on their mind.