GPT2, Counting Consciousness and the Curious Hacker
medium.com
medium.com
OpenAI decided not to release the full scale version of their model (1.5B), because they were afraid of its potential security implications, in particular in generating fake news.
One thing I believe gets indequate recognition, including in this article, is OpenAI’s stated motivation for their release strategy. From the GPT2 release announcement[1]:
This decision, as well as our discussion of it, is an experiment: while we are not sure that it is the right decision today, we believe that the AI community will eventually need to tackle the issue of publication norms in a thoughtful way in certain research areas. Other disciplines such as biotechnology and cybersecurity have long had active debates about responsible publication in cases with clear misuse potential, and we hope that our experiment will serve as a case study for more nuanced discussions of model and code release decisions in the AI community.
It was, in part, due to direct safety concerns, but it was also to force the conversation about how to handle the release of legitimately dangerous capabilities in the future.
1. GPT2 is the next inevitable step in the trend of AI-generated content, and powerful organizations are already capable of these things
2. Delegating your trust to algorithms or other groups is not a good solution to fake news
3. Humans are still smart and will be able to adapt
Actual GPT2-1.5B replication post: https://medium.com/@NPCollapse/replicating-gpt2-1-5b-86454a7...
On a side note, I'm still amazed that the NYT is able to uphold its curated public reputation so well that it is still consistently used as an example of an organization that would never dare publish something low-quality.
Here's a few apropos outputs from a GPT-2 model I'm working on to conditionally generate programming article titles based on keywords "Java", "AI", and "death":
In the age of machine learning, is a human becoming too powerful? If so, what impact does AI have on how we view death?
An AI death machine created in Python
The death of an AI: What the world is looking to AI for now
Why Is AI So Bad At Killing And Exploring For People In Java?
Java is a death sentence – and AI is coming for all of us
How to avoid death by AI in Java
The AI is using Java to make it's own deathmatch
Google's "AI for death" is now a death sentence for Java
Why is Java a death sentence for AI?
Java is not a death sentence, but a step towards AI
The AI is dying... And it's not even on Java.
Go to Java to see the death of AI
Google's AI will stop every death before it's too late
Alpha male: @marthamaccallum is a huge supporter of @realDonaldTrump and an amazing person. He is incredible on women he is great for business.
Join me on Wednesday May 17 in Albany New York at 6pm for a #TaxReform rally! Tickets: https://t.co/TN1QMA7bio. https://t.co/yIqBTUKrRC
A must watch: Attacking the "Dossier" is like attacking a sore thumb - bad for the country!
Why isn’t Hillary Clinton being investigated for her dealings with Russia-Russia? There is no reason why her corrupt ledger should still be closed. (and of COURSE!)
Totally illegal Donna Brazile calls Barack Obama "The Drilling America" on her Sunday morning television show.
Unbelievable evening in Albany New York last night! Disgraceful given the way Crooked Hillary is treated. There is no way this should be happening. She is a disaster!
So funny Bernie Sanders saying that the Democrats should go to a constitutional amendment, like our present system. They will never win if we continue toovertake!
Very sad that Crooked Hillary Clinton can't close the deal with Bernie Sanders. I can't imagine she has $10 million on hand! A vote for her is a vote for defeat.
Thank you Ed Klein! It is so great how you have done so much for the people of Vermont. So proud of you as their Governor. Now need your support to #MakeAmericaGreatAgain!
Wow Crooked Hillary Clinton has just indicated that she will not run in 2020 & that Bernie Sanders is a fraud! What does she have to hide!
Will be doing @TheIssue with @Greta at 9pm tonight
Heading to Albany New York for an interview with @CNN. Enjoy!
Join me in Albany New York- tomorrow evening at 6pm! #NYPrimary #MAGATickets: https://t.co/NVTaAQ53Zo https://t.co/Z0cKQhC8Up
"@greta: Will @realDonaldTrump run? If he doesn't run and if we don't get a real developer and if we don't get a businessman that can get us the answers we seek
So many false and phony T.V. commercials being broadcast in New York State. Federal Election Commission rules prohibit "buying or selling of political ads."
I very much appreciate the many nice statements Mayor Stephanie Rawlings-Blake has made regarding the need to rebuild and restore trust. AARP currently representing VA
Unbelievable howarty Shore needed a rebuild just like the people of Virginia. So much corruption - big money and all. RAISE YOUR HANDS!
.@AlexSalmond Can you believe-you spent $1000000 to take this issue to the Scottish Parliament. You couldn't even get United Kingdom!
for Scotland! https://t.co/HVruc6Up48
I agree! https://t.co/R0rRYTJ7Xh
I agree! https://t.co/d2tsQS0YTb
We are getting down to the wire on our Great American Infrastructure. Presidential Memorandum on #Infrastructure was well received and easy to understand.
Unbelievable howarty Shore needs a rebuild just like the people of Virginia. So much corruption - big money and all. RAISE YOUR HANDS!
We must stop the crime and human trafficking taking place on our streets. It is very sad that many more people are being robbed and killed by certain groups. We must get smart!
"@XKidd92: @realDonaldTrump help me Donald Trump I am so tired of reading about your so called "victories" !"
"@BeaumontAnthony: @realDonaldTrump you are the man Mr Trump!" Thanks.
"@BitcoinMoneyMan: Bitters night at the Trump Hotel in Chicago. Tin Roof is better than the day I became a citizen. #nice!"
"@TiniQBoy: @realDonaldTrump don't let it be said you were a great guest of Putin. You were a good friend. Let the truth be known!"
"@BitcoinMoneyMan: @realDonaldTrump I would call my daughter Ivanka that. She is a great person. She has a lot of class."
"@TWIAMundo: @realDonaldTrump you were a great role model in life. Make us all proud. We are all very proud of you!
"@JenaFeuers: @realDonaldTrump @MillionWit Yes! We are so happy you are coming back to Vegas! It is such an amazing place."
"@ChrisFoley_: Just set my DVR to record Celebrity Apprentice starting next Sunday night at 9 pm EST"
"@liamvanvorhis: @realDonaldTrump attending Canizal 2015 in Verona. En Charles
Sometimes I question consciousness.
And then I realize people demand so little that entertainment could be entirely automated in ten years.
I'm not particularly impressed with the results.
Here are some random samples from OpenAI's version of the model for comparison: https://raw.githubusercontent.com/openai/gpt-2/b5ef71a922efc...
Of course, take these with a few grains of salt, these aren't apples-to-apples, etc.
"Unlike in OpenAI's blogpost, I did absolutely nothing to manually cherry pick the quality here. Some examples are good, most are bad. Almost all outputs decay in quality as the post gets longer. I wanted to keep the raw outputs to give an accurate feel of the strengths and failure cases of the model. The truth is that using a GPT2 type model to create text you want is more an art than a science and can be very finicky."
Then of course there is the fact that the network can generate text a lot faster than a human can read it. Think for a moment how many distinct passages these systems can generate -possibly even infinite- and how few of them a single person can realistically hope to read. It's very hard to know if the few passages one ends up reading are representative of the output of the network, or not.
It'd be even harder to compare two systems just by eyballing their output side-by-side.
Given the large difference in quality, it only takes a few comparisons. Again, just look at the samples. Read a few. Do they really look the same in coherency and quality and realism?
Looking at two texts side-by-side while knowing which is which may well create an illusion of striking differences, which however are not really there. This is why concrete metrics are necessary- because human judgement is biased and inconsistent.
But, like you say- there are no good metrics. In fact, the only thing we have to go by when discussing Open AI's model in the first place is opinions and convictions. Not least the opinion of OpenAi that their model generates text of unprecedented coherence, which itself is not based on anything else than eyballing.
- "And I think no currently existing technique can even scratch the capabilities humans have in this area [detecting truth], even with all of their biases".
This is literally why OpenAI is releasing the model slowly.
- "I really do think that if people just knew that something like GPT2 exists and is out there in the wild, it would force them to improve their standards for what information they trust".
I disagree. Look at how the knowledge that 'shills' and 'russian trolls' exist has shaped online discussions. It becomes an easy way to dismiss things that challenge your worldview, but does not significantly improve their ability to determine veracity.
- "I think we have reached a point where it is no longer, in general, possible to determine whether a given text is human generated or not".
- "It means that even if we had a system that can perfectly detect AI generated babbling and deployed it, it would censor not just AI, but a good chunk of real, human communication as well".
Based on what? This is a very thoughtless dismissal of a very important (and very much unanswered) question. Deep learning often leaves identifiable artifacts, such as deep fakes not blinking. GPT-2 is impressive, but to assume that it is identical to human generated text is a lazy assumption.
- "I’m as anti-war, pro-human as you could imagine, but if I was alive in the 40s, and the US offered me to work on the Bomb…I don’t know if I could have said no. Not because I wanted to hurt people, this is the important thing to understand about the curious hacker, but because it was just so damn cool. Splitting the literal building blocks of matter itself to make a giant explosion? That’s fucking awesome!"
- "a digital 21st century Manhattan project (Which, again, sounds like heaven to me)".
There are no words to describe my disgust at this sentiment. I'm not anti-war, but I think prioritizing 'having fun' over human lives is evil.
(This is not meant to say that this guy is bad, in fact neither he nor the content of the article matter at all. The only necessary context is that a single person decided to do what a group of top-notch researchers considered as maybe a bad idea.)
It seems OpenAI wanted to release this project in phases, allowing people time to adjust to its nature. If in the future an even more disruptive project is created (by OpenAI or others), if the creator feels they cannot release it in their own perception of what a “safe” way is, they may simply avoid publishing and instead privately communicate with companies and powerful individuals. Which I don’t think is the outcome the author wants. So I hope he reconsiders here.
Do you really think anyone besides AI enthusiasts are paying attention? It's not like the general public is even aware of this, let alone following its progress.
Security disclosures are much simpler - we found a vulnerability and we will provide time for the company/team/organization to patch it before announcing it to the world so it won't be exploited by bad actors.
If OpenAI truly feels they have something akin to nuclear weaponry, and that fewer actors having it is better, than they have to openly admit that they consider themselves better gatekeepers of the technology than the public and back away even further from their non-profit/limited profit ideals. "We are creating this technology for the good of the world, but it's so good we are afraid to let you use it, so only we will benefit from it."
I find them wildly inconsistent in their messaging, trying to have the best of everything with none of the drawbacks.
As for the best one, I quote the ones that struck me during the training process, and some are highlighted in https://www.gwern.net/GPT-2#unconditional-samples
Some of the ones I like are 'We never say "Thank you"', 'Thy soul, thy very soul is burning!', '"It is morn!” said the clover-bush', 'And they have seen the last light fail', 'There comes a murmur low and sweet'.
Probably the best IMO is 'The sun is gone, and the night is late', but of course everyone will have a different favorite.
Everything is generated by the 117M model, correct? If so, do you expect the quality to improve for larger models, or is there not enough poetry to train them on? I wonder how much of total poetry is contained in Gutenberg poetry corpus...
By the way, here's some poetry which has been generated by a Markov model: http://www.kurzweilcyberart.com/poetry/rkcp_poetry_samples.p...
These models are often much more general than you seem to be thinking. There's a base model which is incredibly computationally expensive to create from scratch. It is trained on a very large, very general set of data. Then there are specialized versions which are much cheaper to create - you start from the base model that you already have, and you train (much more briefly) on a specific set of data in order to tailor the output.
https://www.tensorflow.org/hub/tutorials/image_retraining
> Modern image recognition models have millions of parameters. Training them from scratch requires a lot of labeled training data and a lot of computing power (hundreds of GPU-hours or more). Transfer learning is a technique that shortcuts much of this by taking a piece of a model that has already been trained on a related task and reusing it in a new model.
Is there some way to test this claim, other than waiting for the model to be released on the 1st July?
Great, admirable work still.
This reasoning sounds circular and doesn't go deep enough. Couldn't NYT or individual reporters at NYT abuse our trust on rare occasions if they felt the stakes were high? Couldn't they unintentionally mislead us because they were mislead by a source? No: we trust NYT because they have a financial interest in being a reliable source of news. People wouldn't read NYT anymore if they thought that they didn't take that responsibility seriously.
- upon verification the model appeared to be weaker, than the smallest one released by OpenAI
- the author ended up declining to release it
No one ever really does expect it.
This was not happening 10 years back.
In my experience the majority of 30 and 40 year olds in Tech today pushing AI code out have no clue who Kahneman, Hariri and Pinker are.
So expect subprime meltdowns/Trump/Brexit type unintended bullshit for a few more years till the psychologist+historian+sociologist coders take over.
People attacked Faraday for suggesting electromagnetism was a thing, hardly paid attention to Maxwell for doing the Math because it was too complicated, and attacked Oliver Heaviside his whole life, for simplifying the math that everyone uses today. Each of these characters made mistakes and had weaknesses too and its easy to find fault.
That's the way things work. There are always going to be more people who fixate/react to mistakes, than use them as stepping stones to new discoveries.
Without these writers I would have hardly any awareness about these subjects, because these subjects were hardly discussed when I was in school. And it's not even that long ago. They opened the door. History will show that was the role they played.