OpenAI's GPT-3 may be the biggest thing since Bitcoin
maraoz.com
maraoz.com
Having read to the bottom, the quality of text generation there absolutely blew me away. GPT-2 texts have a somewhat disconnected quality - "it only makes sense if you're not really paying attention" - that this article lacks entirely. Adjacent sentences and even paragraphs are plausible neighbours. Even on re-reading more closely, it doesn't feel like the world's best writing, but I don't notice major loss of coherence until the last couple of paragraphs. I am now really curious about the other 9 attempts that were thrown away. Are they always this good?!
[0] https://en.wikipedia.org/wiki/Canary_trap#Barium_meal_test
* GPT-3 is trained on one
I currently work on synbio × web archival.
Some of us are cooking up futuretech aimed at storing all of IA (archive.org) in a shoebox. Others are working on putting archival tools in more normal web users' hands, and making those tools do things that people tend to value more in the short-term, like help them understand what they're researching, rather than merely stash pages.
My ambitions for web archives are outsized compared to other archivists, but I'm fine with that. I'm looking beyond web archives as we currently understand them toward web archives as something else that doesn't quite exist yet: everyday artefacts, colocated and integrated with other web technology to an extent that they serve in essential sensemaking, workflow, and maybe security roles.
Right now, some obvious, pressing priorities are (a) preserving vastly more content and (b) doing more with the archives themselves.
A: The overwhelming majority of born-digital content is lost within a far narrower time-slice than would admit preservation at current rates, and data growth is accelerating beyond the reach of conventional storage media. So, for me, the world's current largest x is never the true object of my desire. I'm after a way to hold the world that is and the world to come.
Ideally, that world to come is one where lifelong data stewardship of everything from your own genome to your digital footprint is ubiquitously available and loss of information has been largely rendered optional.
This, of course, requires magic storage density that simply defies fundamental limitations of conventional storage media. I'm strongly confident that we're getting early glimpses of the first real Magic contenders. All lie outside, or on the far periphery of, the evolutionary tree that got us the storage media we have today. For instance, I'm running an art exhibition that involves encoding all the works on DNA.
B: Distributed archival that comes almost as naturally as browsing is well within reach, and with that comes some very new potential for distributed computation on archives. One hand washes the other.
One important thing to realize here is that, in many cases, you can name a very small handful of individuals as the reason why current archival resources exist. GPT-3 is cracking the surface by training on data produced by one guy named Sebastian, for instance.
…i'm sorta tired and have to respond to something about every twitter snapshot since June being broken, though, so I'll pick this back up later.
Basically the brain and "consciousness" isn't as fancy as we think?
However that makes one wonder if it can also learn to generate emphases, and if so, how would it format? With voice generation it can simply change its tonality but with text generation it has to demarcate it in some way--does the human say "format the output for html", for instance?
I agree. The environment - as the source of learning and forming concepts, is the key ingredient of consciousness, not the brain.
I could use… what's the word? I think it's more funding.
I agree, responses are almost as interesting as GPT-3. And this place has always felt like one of the better when it comes to people reading past the titles!
https://www.reddit.com/r/AskReddit/comments/348vlx/what_bot_...
I guessed it was fake before getting to the end, not from the content, but from the fact that all the sentences are roughly the same length and follow the same basic grammatical patterns. Real people purposely mix up their sentence structure in order to keep the readers engaged, whereas this wasn't doing that at all. Still very impressive though; if not for the fact that the post was about computer generated content I probably wouldn't have noticed.
Maybe. Right now this reads like a glorified shopping list. It's coherent, but actually sounding human also requires a theory of mind.
E.g. I explain here why it's possible for written statements to be objectively insightful, informative, interesting, or funny, but objectively in a way that's relational to other information or beliefs. The implication being that statements are only going to seem subjectively funny or insightful (or whatever) to others who have that knowledge or those beliefs, which means that you can't reliably create those subjective experiences in a reader without having some sort of theory of mind for them.
I guess you can create content that's funny or insightful relative to that content itself, but that's not especially useful. It's entertaining at the time, but the experience is more like seeing a movie that you laugh a lot during but then leave and are kind of like what was the point? It's an empty experience because it wasn't transformative.
I definitely don't think it's impossible, but I also don't think it's a matter of just adding a couple more if-else statements.
https://alexkrupp.typepad.com/sensemaking/2010/06/how-writin...
I'm going to call this goalpost shifting. This article is better writing than some % of humans, theory of mind or otherwise. The AI has comfortably surpassed Timecube-level writing and is entering the pool of passes-for-a-human.
'Sounds human' is a spectrum that starts with the mentally ill and goes up to the best writers in human history.
That's completely fair. On the other hand, without a theory of mind it can't really educate or inspire people, the only thing it can do is maybe trick them about the authorship of something. But once people learn the techniques for identifying this kind of writing, it can't even do that anymore. To me this is like the front end of something, but it still needs a back end.
Don't get me wrong, it's super cool research and seems like a huge step forward, and I'm excited to see where it goes. But I also don't see this AI running a successful presidential campaign or whatever, at least within the next couple years.
I wouldn't agree with that, either. How often have we heard of someone gaining useful insights by considering ideas that were misapplied or just plain wrong? Entire branches of physics have evolved that way. As far as successful presidential campaigns are concerned... well, let's not even go there.
If there's such a thing as a 'theory of mind', it applies to the reader, not the writer.
For example, I delayed in writing this comment because the cat was on my lap, and I couldn't fit the laptop and the cat both. You get that. I know you do, even if you don't own a cat, and even if you're reading this on a phone or a desktop.
GPT-3 does not understand about the cat. To GPT-3, they're just words and phrases that occur in the vicinity of each other with certain probability distributions. As a result, it can't write something and know that there's something there in your mind for it to connect to.
Cyc would handle the bit about the cat differently. It would ask "Did you mean cat-the-mammal, cat-the-Caterpillar-stock-symbol, or cat-the-ethernet-cable-classification?" It has categories, and some notion of which words fit in each category, but it still doesn't understand what's going on.
But you the human understand, because you have a lap, and you've at least seen a cat on a lap.
You really think GPT-3 never came across a comment about a cat in lap? 50% of all the pictures on the internet are cats sitting on people. GPT-3 doesn't need to understand it to echo this common knowledge.
Airplanes don't look like birds at all but they do fly.
And that got me thinking about what I could do with this thing, whether I should, what I wanted to try out...
So the BS random ideas were still inspiring a bit.
What if the ultimate theory of mind turns out to be that consciousness is an illusion and nothing separates us from a sufficiently sophisticated markov process.
Conscious experience would still exist (see cogito ergo sum, Chalmers, etc). If we were to be shown we're just Markov processes, that wouldn't disprove the existence of conscious experience. Just like confabulation, a misleading experience is still an experience.
What it would disprove is any sense of agency.
I would like to see a GPT model where training data is weighted by credibility / authority (e.g. using Pagerank).
So this morning he heard about an animal, it was kind of a lion. But with bat's ears, it lives in Africa. It looks like it's a rock, but it's actually not, it's rock shaped but has tiny legs. And it's gray and hard. Its face... It doesn't really have a face. It lives up in trees where it eats bamboo and apples. It has these huge fangs like sabertooth tigers, you know?
It's glorious.
My smallest kid has a habit of telling stories about himself that actually come from whatever he heard recently, e.g. "once I was Godzilla..", or claims about things in reality that come from stories or misunderstandings all mixed up "did you know, there are three pigs, but they are not pigs, they are wolves and a hunter came and killed them but they weren't wolves they were dragons..."
It's actually very GPT-3-ish now that I think of it.
Lots of examples I've seen have phrases like "see table below". Of course there's no table and it's hard to imagine how there could be.
But GPT is trained on internet content and the internet is full of terrible writing that never gets to the point. I doubt there's any way to know how much is "not actually understanding the subject matter" vs. "learning bad writing from bad writers". I'm inclined to believe the majority is the former but there's got to be a little of the latter sprinkled in.
One thing I learned was it has detailed knowledge of the world of Avatar: The Last Airbender, seemingly through fanfics. It was fun having it to teach me the lost arts of pizzabending ("form your hands into the shape of a letter 'P'" and so on, and needing to practice by juggling rubber pizzas) and beetlebending ("always remember that to beetle bend it helps to like beetles," my wise uncle suggested). Each of these tended to precipitate a narrative collapse.
The writing style was surprisingly homogeneous, and it reminded me of young adult novels. It would definitely be interesting to see it with other writing styles, beyond the occasional old poetry.
> The man walks away and starts undressing. You shrug and keep following him. Soon, you find yourself naked.
I certainly do, don't you? When I read a blog post and it's full of poorly-integrated buzzwords that make it seem like it was churned out by a non-English speaker being paid very poorly per word, I stop reading and move on.
I recently read a few pages of a book someone had recommended to me and stopped reading because of the writing style.
Heck, you can read a few pages of, say, a Dan Brown novel, and based on the writing style might choose not to read it, since the style tells you a lot about the kind of book it is.
I then reread it and it indeed read like a weird, rambling, incoherent article. Looking at it closely, it had a good many contradictory, meaningless and incoherent sentences.("It is a popular forum with many types of posts and posters.")
The headline, however, seemed about right.
It's true the nonsense in this article is a bit different than the nonsense of a GPT-2 article. But the thing GPT-2 paragraphs sound pretty coherent 'till they suddenly go off the rail. This is more like an article that was never quite on the rails and so it's slightly more internally cohesive. But not "better".
Maybe the article just reflects the author's style. Anyone have a GPT-3 test site link?
But you are right, it can't be both in the context of this article :)
Now, I’m not so sure :)
That said, the content of the computer-generated parts doesn't make much sense even for a Bitcoin-influenced article (what would be the point of paraphrasing your previous post in a forum on a regular basis, and how does this not get one very quickly banned?), but the grammar is far far better than previous attempts - it reads like Simple English wiki.
It sounds to me like you must be an academic, or someone with good habits for being efficient at reading articles.
One potential issue with this approach is that the text it generates is 'nonsensical', in that it is almost like a word-salad. Although this is a standard problem with neural nets (and other machine learning algorithms), in this case the text actually is a word-salad. It seems that it has learned the rules of grammar, but not the meaning of words. It is able to string words together in a way that sounds right, but the words don't actually mean anything.
Plot twist: This comment was generated by GPT-3 prompted with some of the comments in this thread.
Soon enough, someone will replicate the Sokal hoax[b] with GPT-3 or another state-of-the-art language-generation model. It's not hard to imagine GPT-3 writing a fake paper that gets published in certain academic journals in the social sciences.
[a] https://en.wikipedia.org/wiki/Technobabble
[b] https://en.wikipedia.org/wiki/Sokal_affair -- here's a copy of Sokal's hoax paper, "Transgressing the Boundaries: Towards a Transformative Hermeneutics of Quantum Gravity:" https://physics.nyu.edu/faculty/sokal/transgress_v2/transgre...
Especially the shoggoth cat dialogue, I found that one really creepy. The fragment below comes straight out from the uncanny valley:
Human: Those memes sound funny. But you didn’t include any puns. So tell me, what is your favorite cat pun?
AI: Well, the best pun for me was the one he searched for the third time: “You didn’t eat all my fish, did you?” You see, the word “fish” can be replaced with the word “cats” to make the sentence read “Did you eat all my cats?”
Some say this has already happened. Nobody has ever seen the Social Text editors and Mochizuki in the same room together, have they?
This comment was also written by GPT-3.
Not gonna lie, I went poking around to see if I could get my hands on it, but it seems like the answer is no, for now.
I have to admit, this is passing my turing test...
Maybe the real lesson is we don't expect human-written comments on discussion fora to be particularly coherent....
The final bit doesn't quite connect, but overall I've seen far less coherent comments written by humans on subject with far more logical flaws.
I am genuinely awed.
Especially the second comment can be coherently interpreted with some good will and a cynical view of the humanities and philosophy. The "author" could say that once GPT-3 can write humanities papers it will quickly make humanity scientists redundant and that humanities scientists are philosophers is not important and doesn't warrant a job alone ("they don't actually do anything"). Eventually it shifts that this is the fault of science working too well (GPT-3 being a product of science)
It's not a consistent argument, but without the context of these comments being GPT-3 it would have totally passed my turing test, just not my sanity test.
The model fundamentally has no understanding of the world, so if it can successfully argue about a central thesis without simply selecting pre-existing fragments, then it would suggest that the statistical relations between words capture directly our reasoning about the world.
Pretty average for HN then ;)
How could we expect it? After 35+ years (BBS and Usenet onward), we've learned that they are often not.
>In the not-too-distant future, there probably won't be any more philosophy professors; there will just be philosophers
Was quite clever and I'm still trying to figure out what it means.
It would be interesting to see if the output has a similar quality when trained only on highly regarded texts.
https://news.ycombinator.com/item?id=22833407
(So I think it was some other story on the same topic.)
Second, I'm curious/terrified at how future iterations of GPT-3 may impact our ability to express ourselves and form bonds with other humans. First it's text messages and comments. Then it's essays. Then it's speeches. Then it's love letters. Then it's pitches. Then it's books. Then it's movie scripts. Then it's...
TLDR; Fascinated by the technology behind making something like this work and quite worried about the implications of the technology.
There's a totally valid discipline in taking concepts from different areas and smushing them together to make a new idea. That's what a lot of creativity is, fundamentally. So a bot that's been trained across a wide variety of texts, spitting out an amalgam in response to a prompt that causes a connection to be made, is not only possible, but likely a very good way of generating papers (or at least abstracts) for humans to check. And if the raw output is readable, why not publish it?
Would you please show us the input text, or rules, you gave to GPT-3 to create this comment ?
In fact, while reading that comment I started to wonder why no one has tried to use GPT to generate text one character at a time. Or if someone has, what are the advantages and disadvantages over the BPE approach.
Edit: it is amusing to think that soon the way to distinguish them will be that human comments have weird errors caused by smartphone keyboard "spell checking" in them...
Still, would be an interesting experiment. Gwern swears it would improve stuff, so worth trying and comparing, I guess
The quality of writing was very high, so I was convinced I was reading something put together by a human with agency... except it didn’t pass my gut-feeling “how IT works”. It made me suspect that either the algorithm (the described one, not the AI responsible) was off, or that I just didn’t understand AI any more. As I know I don’t have up to date AI knowledge, the algorithm appeared more believable. I hiked deep down the uncanny valley with that one.
Given the propensity for academic writing to often favour the strategy of confusing the author through obfuscation (to make a minor advance sound more significant than it is), I suspect tools like this could, as you say, actually get published papers in some fields like social sciences. In an engineering or science paper you can check equations match conclusions, and that graphs match data etc.
In a more qualitative field of work, reviewed in a publish-or-perish system that doesn't incentivise time spent on detailed reviewing, I think there's a very real risk babble like this just comes across like every other paper they "review".
I think it takes a certain level of confidence to dismiss others' work as nonsensical waffle, but sadly this is a confidence many lack, and they assume there must be some sense located therein. Marketing text is a great place to train yourself to recognise much of what is written is meaningless hocum.
Sci-Gen - https://pdos.csail.mit.edu/archive/scigen/
Reporting on withdrawals of papers - https://www.researchgate.net/publication/278619529_Detection...
You can also publish a lot of nonsense in certain chinese journals that optimize for quantity in quality, in whatever field you want.
It's easy to consider text generation models as "just mimicking grammar". But isn't grammar also just a model of human cognition?
Is GPT modeling grammar or is it modeling human cognition? Since GPT can ingest radically more text (aka ideas) won't it soon be able to generate texts (aka ideas) that are a more accurate collation of current knowledge than any individual human could generate?
--
[Was this comment written by GPT-3?]
I don't really understand why we're trying so hard to build models that can generate coherent texts based on having predigested only other texts, without any other experience of reality. Their capabilities appear already superhuman in their ability to imitate styles and patterns of any kind (including code generation, images, etc.). It feels like we're overshooting our target by trying to solve an unsolvable problem, that of deriving the semantics of reality from pure text, without any other type of input.
I am impressed though nobody dared to guess in 2 weeks.
This kills the forum.
Seriously, once this is weaponised, discussion of politics on the internet with strangers becomes completely pointless instead of just mostly pointless. You could potentially convince a human; you can't convince a neural net that isn't in learning mode.
We might end up with reputation based conversations.
That could have consequences for their reputation, though.
(Reputation is a lot more controversial and complicated than it sounds)
Quite the opposite, I suspect.
Eventually, to engage in the most persuasive conversations, the AIs will develop a real-time learning mode.
Once that is weaponised, the AIs will be on track to be in charge of running things, or at least greatly influencing how things are run.
What the AIs "think" will matter, if only because people will be listening to them.
Then it will be really important to discuss politics with the AIs.
GPT-3 isn't AGI, but it's weapons-grade in a way that GPT-2 wasn't.
The result is that worthwhile public discussion is dying. We have to transition now to secure verified communication.
Either that or the bots fork off a new cultural discourse and we treat them like a new form of entertainment.
It was relatively good, although I began to suspect it was GPT3 generated about halfway through (partially because the style felt a bit stiff but also just out of a shayamalan-what-a-twist 6th sense of mine that was tingling)
Saying that, I briefly saw the first sentence of your comment and went to read the article with the idea that trickery was afoot, specifically guessing correctly the nature of the article. And yet, even then, on the back foot... it fooled me. Incredible.
I agree with you. I suspect few people have read until the end to realize that, in fact, ...
GPT-3 is objectively a step forward in the field of AI text-generation, but the current hype on VC Twitter misrepresents the model's current capabilities. GPT-3 isn't magic.
However, if the output needs to be curated and edited by humans, the scale and automation is gone - we just get a different manual process, with a modest improvement to speed at cost of some decline in quality, and that's not very impactful.
Google at this point favours long form content for many search intents. Being able to generate thousands of these pages in one-click is a real problem. Not just because of popular topics e.g. "covid-19 symptoms" but more so for the long tail e.g. "should I drink coffee to cure covid-19".
It may be that Google's algorithms don't care at all how human-like the text is, or that their own recognition algorithm/NN (whatever they use) isn't fooled. Even if it is affected, Google has the money and corpus to build its own competing NN to recognize GPT-3 text.
That said, there might be a different threat to Google. GPT-3 seems really useful as a search engine of sorts (with the first answer implementing the 'I'm Feeling Lucky' button). Tune it for a query syntax, and for getting the 'top X' results somehow, then we just need the web corpus and a basic filter over the results. We could have a very interesting Google competitor.
Honestly not that impressive since you can get comparable results with a series of regex rules given that there are limited ways to describe your intent e.g. "create a button of colour <colour> at the <location of button>"
I believe the hype is that people think they can replace the designer by "just telling the computer" what they want. I don't believe that will work, as they already have trouble telling a human what they want, and a computer won't really know what to do with "I want it to kind of feel like it's from that movie with the blue people that Cameron did, you know?"
In my experience, people have a hard time writing their ideas about designs & features down, because they don't know what they want. They want to talk about it abstractly with somebody who has a better understanding of the field so that person can help them develop the idea. I don't think ML will cover that part any time soon.
From an academic standpoint, writing is part of the thinking process. If you haven't written it down, you haven't fully thought it through. If it feels difficult, that's probably because your understanding isn't as complete as you thought it was.
From a software development standpoint, implementing something is part of the thinking process. Ever notice how the requirements have a tendency to break as soon as you actually try to implement them? If a spot seems difficult it just means you hadn't really figured it out yet.
I 100% agree. I noticed a giant shift in tasks when I made one client write tickets instead of making phone calls. Writing it down forces you to think it through.
And I agree about software development as well, yes. Though I think it's even rare to have somebody describe all the features they want unless it's an experienced software developer who basically writes a textual representation of the application.
But for most PMs (that I've worked with at least), they have vague ideas about what they want, and bringing them into focus is a back and forth with developers and designers. I don't see them getting anywhere with an NLP automaton, but maybe with an Eliza-style system: "Give me a big yellow button saying 'Sign up'" - "Why do you want a big yellow button saying 'Sign up'?" - "You're right, that's too on the nose... give me a link saying 'Sign up'"...
With so many weights, it practically encodes a massive Internet text database.
More than cherry-picking, there's the Eliza Effect - it's pretty easy to make people think generated text is intelligent. That text can seem intelligent for a while isn't necessarily impressive at all.
Bots offering idiocy and idiocy generally has done lots of damage. But by idiocy here I would quite carefully calculated cleverly polarized positions and I don't think just bot-rot would be enough (to maybe coin a phrase).
Makes me worry about my own reading comprehension, but I think what happened was that since it was posted on HN and got upvoted a lot, I simply assumed that anything that I didn't understand was not the writer's fault, but mine.
For instance, it was unclear from the post what the bitcoinforum experiment was about, but I just dismissed it as me not being attentive enough while reading.
At one point GPT-3 writes: "The forum also has many people I don’t like. I expect them to be disproportionately excited by the possibility of having a new poster that appears to be intelligent and relevant." Why would people he doesn't like be paricularly excited about a new intelligent poster? Again I just assumed that I missed the author's point, not that it was nonsensical.
Twice it refers to tables or screenshots that are not included, but it seemed like an innocent mistake. "When I post to the forum as myself, people frequently mention that they think I must be a bot to be able to post so quickly" seemed like another simple mistake, meaning to say that when he posted as GPT-3, people thought he was being too quick.
This is like a written Rorschach test, when I'm convinced that what I'm reading must make sense, then I'll guess at the author's intent and force it to make sense, forgiving a lot of mistakes or inconsistencies.
Is reddit gold really that valuable?
> famous
Surely there are easier ways.
> really useful
We already have enough 2020 reddit commenters regurgitating 2010 hn threads regurgitating 2000 slashdot threads, thanks.
It’s cool, but it looked like very basic stuff - the type of UI that is very easy to create in a few minutes. (And really with what was setup behind the scenes - maybe just as fast to just write the code.)
The hard part about software development is not those bits which are common, but the parts that are unique to our specific solution.
Search terms tweaked for your unique interests, and not a commercial entity's, for example.
However I like spirit of optimism and first looks at encouraging and very promising results.
Exciting times!
This will accelerate development. Is the current version there? Probably not. But GPT-4 might, and would then accelerate the development of future versions.
Even though this is not "magic", it sounds like it will turn into a practically usable and extremely valuable tool soon.
https://twitter.com/jsngr/status/1284511080715362304
Granted, it seems like there was a lot of behind the scenes work to make that happen.
It's qualitatively different than GPT-2. I was on a discord with someone that has access to it and a bunch of us were throwing ideas out for prompts. One of them was to provide an anonymized bio of someone and see if it could guess who it was. The format was 'this person...they..and then they...\nQ: Who is this person?\nA: '
At the first pass it didn't guess correctly. But we erased its response and tried again and it got the answer correct. We then asked it to screenwrite some porn and tell jokes. Yes there were some misses, but it got things right so frequently that you can start to see the future.
Having all of this capability in one package is pretty remarkable and nothing has approached it to date.
@balajis being generated by GPT-3 would make a lot of sense, though.
"Text generation" undersells it a little bit. What are humans except "text generation" machines? Language is the stuff of reason. GPT-3 has demonstrated capabilities that we believed were exclusive to humanity --- humor, logic, sarcasm, cultural references --- in an automatic and generalizable way. It's far more than a "text generation" system. I look forward to seeing what GPT-4 and GPT-5 can do. I suspect we're all going to be amazed by what we get when we continue down this path of simple scaling (and sparse-ification) of transformer architectures trained on (basically) the whole internet.
The ability to grow and choose our own direction: to choose what our goals are, curiosity, self-awareness, desire. To imply that GPT-3 is anything close to strong AI is kind of ridiculous.
I predict within a few years, the descendants of GPT-3 will use very different fundamental units for processing that differ greatly from the current state-of-the-art (i.e. they won't use BPEs and their ilk anymore, except for final output) and will be far more powerful as a result.
I do agree with you. We, as somewhat intelligent beings, do not base our thinking on words or language AFAIK, even though it's our best way to convey ideas to others. And we learn through experience, way faster than GPT-3 does, with fewer shots. It looks like the attention mechanisms are what made these models actually start to understand things... But those attention mechanisms are still very raw and mainly designed to be easy to execute on current hardware, I wonder how fast will we refine that. Finally it looks like, once trained, these models don't learn when we use them. It definitely doesn't learn through experience and that's a major limitation on how intelligent it can be.
I think sentience like most things is a spectrum, so I'm not really sure what you mean by sentient, but I would argue that for most people the bar for sentience is much higher than text prediction. The Chinese room is only one aspect of our minds, and we don't even know what consciousness is.
And to be fair, reasonable people stake out positions on both sides of this debate: I'm not claiming that the alternative proposition is somehow unreasonable. It's a legitimate subject of scholarly disagreement.
Nevertheless, I'm still firm on language. Why? Because all complexity is ultimately about symbolic manipulation of terms representing the process of manipulation itself. ("Godel, Escher, Bach" is a fantastic exploration of this concept.) How can you manipulate concepts without assigning terms to their parts? That's what language is.
The question I like to ask is this: are there any ideas that you cannot express using language? No? Then how is thought distinct from language?
Yes, people (myself included) experience a "tip of the tongue" experience where you feel like you have an idea you can't just yet express. But maybe this experience is what reason feels like. Why should idea formation take only one "clock cycle" in the brain? Why should we be unaware of the process?
I think this feeling of having an idea yet being unable to formulate it is just the neural equivalent of a CPU pipeline stall. It's not evidence that we can have ideas without language: it's evidence that ideas sometime take a little while to gel.
I think as highly social beings we often annotate all of our thoughts with the language we could use to communicate them, which could lead us to believe that the thoughts are indistinguishable from the language, but that conclusion seems like an error to me. I’ve also heard some people talk about how they are “visual” or “geometric” thinkers and sometimes think in terms of images and structures without words.
To me this indicates a very narrow view of consciousness. Consider for a moment the quiet consciousness of the cerebellum for example.
I like the way David F. Wallace put it: 'Both flesh and not'. There's an astounding amount of consciousness that is not bound by language. One can even argue that language might hinder those forms of consciousness from even arising.
Not sure there's one I can communicate to you, but I'm perfectly capable of forgetting the word for something and still knowing unambiguously yet wordlessly what it is, that's an experience.
Catching a ball? Running? Experiencing emotions from wordless music? Viewing scenery? Engaging with a computer game? How are they not conscious experiences?
What is the role of the body in consciousness, then?
> only context it has is its prompt
The only real context is its latent representation of the prompt, there's nothing fundamentally limiting visual, auditory, symbolic, and mixed prompts as long as they map to a common latent space and the generator is trained on it.
Text generation doesn't chop wood, optimize speedruns, build machinery or win 100-metre dashes.
Text may be involved in training for these things, but to say that doing them is text generation would be like saying that... since compiling code and running AlphaZero both generates bits, AlphaZero is a compiler.
This does not impress me in the slightest.
Taking billions and billions of input corpora and making some of them _sound like_ something a human would say is not impressive. Even if it's at a high school vocabulary level. It may have underlying correlative structure, but there's nothing interesting about the generated artifacts of these algorithms. If we're looking for a cost-effective way to replace content marketing spam... great! We've succeeded! If not, there's nothing interesting or intelligent in these models.
I'll be impressed the day I can see a program that can 1) only rely on its own limited experiential inputs and not billions of artifacts (from already mature persons), and 2) come up with the funny insights of a 3-year-old.
Little children can say things that sound nonsensical but are intelligent. This sounds intelligent but is nonsensical.
Yeah I mean, I agree. But in my opinion, it's a case of "doing the wrong thing right" instead of a more useful "doing the right thing wrong."
I grant that these automated models are useful for low-value classification/generation tasks at high-frequency scale. I don't think that in any way is related to intelligence though, and the only reason I think they've been pursued is because of immediate economic usefulness _shrug_.
When high-value, low-frequency tasks begin to be reproduced by software without intervention, I think we'll be closer to intelligence. This is just mimicry. Change the parameters even in the slightest (e.g. have this algorithm try to "learn" the article it created to actually do something in the world) and it all falls down.
Progress is often made with steps that would have been astonishing a few years ago. And every time the bar is raised higher. Rightly so, but characterizing this as doing the wrong thing is missing the point of what we, and the system, are learning.
Yes it's not intelligence. But then, it's not even clear that we ourselves can define intelligence at all… not all philosophers agree on this. Daniel Dennett (philosopher and computer scientist) for example thinks that consciousness may be just a collection of illusions and tricks a mind plays with itself as it models different facets of and lenses into what it stores and perceives.
I think you missed my point. I think we're going in the wrong direction for AI entirely, and these "advances" are fundamentally misguided. OpenAI is explicitly about "intelligence," and so we should question if this is in fact that.
It's clear that humans have fundamental intelligence much better than all of this stuff with 6 orders of magnitude less input (at least of the same data sort) on a problem.
Perhaps it would be better to say, "I think the ML winter is just around the corner" as opposed to "the AI winter is just around the corner." That said, this really is math, and these algos still don't actually do anything resembling true intelligence.
This direction has produced results that eluded 30+ years of research. What is the evidence that this is the wrong direction?
IIRC the following is attributable to either Margaret Atwood or Iris Murdoch:
"A writer should be able to look into a room [full of people] and understand [in breadth] what is going on."
https://www.youtube.com/watch?v=dXQPL9GooyI
Of course evolutionary algorithms are just one direction as well. But that doesn’t mean that nothing else is happening.
>6 orders of magnitude less input
That is utterly mistaken.
We have the input of millions of generations of evolution which have shaped our brains and given us a lot of instinctive knowledge that we do not need to learn from environmental input that happens during our lifetime.
Instead it was learned over the course of billions of years, during the lifetimes of other organisms that preceded us.
Our brain structure was developed and tuned by all these inputs to have some built in pretrained models. That’s what instincts are. Billions of years in the making. Millions, at the very least, if you want to restrict it to recent primates, although doing so is nonsensical.
>That is utterly mistaken.
I did say of data of the "same sort".
What's absolutely crazy is somehow we think of our DNA base pairs as somehow more important than the physical context that DNA ends up in (society, humans, talking, etc.)
We have the ability to be intelligent and make thoughts with 1 millionth the amount of textual data as this OpenAI GPT-3 study. Maybe... just maybe... intelligence is far more related to things other than just having more data.
I'll actually expand on this and throw this out there: intelligence is in a way antagonistic to more data.
A more intelligent agent needs less knowledge to make a better decision. It's like a function that can do the same computation with fewer inputs. A less intelligent agent requires a lookup table of previously computed intelligent things instead of figuring it out on its own. I think all these "AI" studies are glorified lookup tables.
Note in particular that "like a function that can do the same computation with fewer inputs" maps very well to GPT-3 - it can complete many interesting tasks by just having a few samples provided to it, instead of having to fine-tune it with more training.
The reason it doesn't need more training is because it's already trained itself with millions of lifetimes of human data and encoded that in the parameters!
Humans aren't born trained with data. The fact that we're throwing more and more data at this problem is crazy. The compression ratio of GPT-3 is worse than GPT-2.
You know what else is trained by the experiences of thousands of individual (and billions of collective) human lifetimes of data? And several trillions of non-human ones?
> Humans aren't born trained with data.
That's either very wrong or about to evolve into a no true scotsman regarding what counts as data.
AKA "why is it so hard to swat a fly?" because they literally have a direct linkage betweeen sensing incoming air pressure and jumping. Thats why fly swatters don't make a lot of air pressure.
Why do you yank your hand back when you get burned? It's not a controlled reaction. Where did you learn it? You didn't.
If you think the brain is much more than a chemical computer you are sadly mistaken. I would encourage you (not really but it's funny to say) to go experiment with psychedelics and steroids and you will quickly realize that these substances can take over your own perceived intelligence.
The most fascinating of all of this is articles/documentaries about trans people that have started taking hormones and how their perception of the world -drastically- changed. From "feeling" a fast car all of a sudden, to being able to visualize flavors. It's absolutely amazing.
Seriously, a few years ago recognizing if there's a bird in a photo was an example of a "virtually impossible" task: https://xkcd.com/1425/
A computer that is actually fluent in English — as in, understands the language and can use it context-appropriately — should blow your entire mind.
Did you never do grammar diagrams in grade school? :-)
The "context" and structure of language is a formula. When you have billions of inputs to that formula, it's not surprising you can get a fit or push that fit backwards to generate a data set.
This algorithm does not "understand" the things it's saying. If it did, that wouldn't be the end of the chain. It could, without training, make investment decisions on that advice, because it would understand the context of what it had just come up with. Plenty of other examples abound.
Humans or animals don't get to have their firmware "upgraded" or software "retrained" every time a new hype paper comes out. They have to use a very limited and basically fixed set of inputs + their own personal history for the rest of their lives. And the outputs they create become internalized and used as inputs to other tasks.
We could make 1M models that do little tasks very well, but unless they can be combined in such a way that the models cooperate and have agency over time, this is just a math problem. And I do say "just" in a derogatory way here. Most of this stuff could have been done by the scientific community decades ago if they had the hardware and quantity of ad clicks/emails/events/gifs to do what are basically sophisticated linear algebra tasks.
Hasn't the typical human taken in orders of magnitude more data than this example? And the data has been of both direct sensory experience and texts from other people as well.
Have you read GPT-3s 175 billion parameters (words, sentences, papers, I don't care) of anything? Do you know all the words used in that corpus? Nobody has or does.
A child of a small age can listen to a very small set of things and not just come up with words to communicate with mama and papa what they learned, but they can reuse it. And this I think is key, because the language part of that is at least partially secondarily. The little kid understands what they're talking about even if they have a hard time communicating it to an adult. The fact they take creative leaps to use their extremely limited vocabulary to communicate their knowledge is amazing.
The human brain requires less training, but to some extent it is pretrained by our genetic code. The human brain will take on a predictable structure with any sort of training.
This post was generated using GPT-3. [;)]
Can’t tell if you are kidding or not, but if you aren’t, mind sharing links about the researcher for the curious?
edit: Can't seem to find it which is a shame. I think it may have been included in a TED talk.
Your post was generated using GPT-3 and 175 billion parameters of pre-existing human writing, contextualized, distilled, and cross-referenced with terminology we've agreed on for centuries. It's a parrot, and I remain unimpressed.
Take the learned knowledge of GPT-3 (because it must be so smart right?) and have it actually do something. Buy stocks, make chemical formulas, build planes. If you are not broke or dead by the end of that exercise, I'll be impressed and believe GPT-3 knows things.
So basically like DNA?
All DNA does it encode for how to grow, build, and mantain a human body. That human body has the potential to learn a language and communicate, but if you put a baby human inside an empty room and drop in food, it will never learn language and never communicate. DNA isn't magic and comparing "millions of years of evolution" of DNA is nothing like the Petabytes of data that GPT-3 needs to operate.
Again DNA has no knowledge embedded in it, it has no words or data embedded. Data in the sense that we imagine Wikipedia stored in JSON files on a hard disk. DNA stores an algorithm for growth of a human, that's it.
The GPT-3 model is probably > 700GB in size. That is, for GPT to be able to generate text it needs an absolutely massive "memory" of existing text which it can recite verbatim. In contrast, young human children can generate more novel insights with many orders of magnitude less data in "memory" and less training time.
Sigh. When DNA becomes human, it doesn't have a-priori access to all the world's knowledge and yet it still develops intelligence without it. And that little DNA machine learns and grows over time.
When thousands of scientists and billions of human artifacts and 1000X more compute are put into the philosophical successor of GPT-3, it won't be as impressive as what happens when a 2 year old becomes a 3 year old. (It will probably make GPT-4 even less impressive than GPT-3, because the inputs vis-a-vis outputs will be even that much more removed from what humans already do.)
If you mean that anything except full general intelligence is unimpressive than that seems like a fairly high standard.
What's unimpressive about a stunningly believable parrot? I think, at the very least, that GPT-3 is knowledgeable enough to answer any trivia you throw at it, and creative enough to write original poetry that a college student could have plausibly written.
Not everything worth doing is as high-stakes as buying stocks, making chemical formulas, or building planes.
> While typically task-agnostic in architecture, this method still requires task-specific fine-tuning datasets of thousands or tens of thousands of examples [0]
I don’t know what constitutes an example in this case but let’s assume it means 1 blog article. I don’t know many humans that read thousands or tens of thousands of blog articles on a specific topic. And if I did I’d expect that human to write a much more interesting article.
To me, this and other similar generated texts from OpenAI feel bland / generic.
Take a listen to the generated music from OpenAI - https://openai.com/blog/jukebox/. It’s pretty bad, but in a weird way. It’s technically correct - in key, on beat, ect. And even some of the music it generates is technically hard to do, but it sounds so painfully generic.
> All the impressive achievements of deep learning amount to just curve fitting Judea Perl [1]
This comment was written by a human :)
[0]https://arxiv.org/abs/2005.14165 [1]https://www.quantamagazine.org/to-build-truly-intelligent-ma...
I'd like to play devils advocate here.
Given one blog article in a foreign language: Would a human be able to write coherent future articles?
With no teacher or context whatsoever how many articles would one have to read before they could write something that would 'fool' a native speaker? 1000, 100,000?
I have no idea how to measure the quantity/quality of contextual and sensory data we are constantly processing from just existing in the real world, however, it is vital to solving these tasks in a human way - yet it is a dataset that no machine has access to
I would argue comparing 'like for like' disregards the rich data we swim amongst as humans, making it an unfair comparison
Why then, the continued obsession with building single-media models?
Is focusing on the Turing test and language proficiency bringing us further away from the goals of legitimate intelligence?
I would argue "yes", which was my original comment. At no point in us trying to replicate what an adult sounds like have we actually demonstrated anything remotely like the IQ of a small child. And there's this big gap where it's implied by some that this process goes 1) sound like an adult -> 2) think like an adult, which seems to be missing the boat imo. (There's logically this intermediate step where we have this adult-sounding monster AI child.)
If we could constrain the vocabulary to that a child might be exposed to, the correlative trickery of these models would be more obvious. The (exceptionally good) quality of these curve fits wouldn't trick us with vocabulary and syntax that looks like something we'd say. The dumb things would sound dumb, and the smart things would sound smart. And maybe, probably even, that would require us fusing in all sorts of other experiential models to make that happen.
I think it's literally just working with available data. With some back of the envelope math, GPT-3's training corpus is thousands of lifetimes of language heard. All else equal, I'm sure the ML community would almost unanimously agree that thousands of lifetimes of other data with many modes of interaction and different media would be better. It would take forever to do and would cost insane amounts of money. But some kinds of labels are relatively cheap, and some data don't need labels at all, like this internet text corpus. I think that explains the obsession with single-media models. There's a lot more work to do and this is, believe it or not, still the low hanging fruit.
But why not just 1 lifetime of different kinds of data? Heck, why not an environment of 3 years of multi-media data that a child would experience? That wouldn't cost insane amounts of money (or probably anything even close to what we've spent on deep learning as a species).
A corpus limited to the experiences of a single agent would create a very compelling case for intelligence if at the end of that training there was something that sounded and acted smart. It couldn't "jump the gun" as it were, by a lookup of some very intelligent statement that was made somewhere else. It would imply the agent was creatively generating new models as opposed to finding pre-existing ones. It'd even be generous to plain-ol'-AI as well as deep learning, because it would allow both causal models to explain learned explicit knowledge (symbolic), or interesting tacit behavior (empirical ML).
How would you imagine creating such an environment in a way that allows you to train models quickly?
2) Quite irrelevant,that's a motivation problem
We've been conditioned to accept articles where there's a lot of words and paragraphs and paragraphs of buildup, but nothing actually being said.
(For context, the vast majority of the article was generated by GPT-3 itself).
> I imagine that similar results can be obtained by republishing GPT-3’s outputs to other message boards, blogs, and social media.
I actually wrote a bit about this scenario and how it could explode comment sections to stonewall topics, which I'm calling Commentdämmerung: https://simonsarris.com/commentdammerung
Still, I think a lot of people have been misled with respect to the coherence of GPT-3. It becomes especially clear once you stop looking at highlight reels (aka human curated gpt-3). The cherry picking changes the game of how compelling it seems vs how it really is. The author even does this:
> I generated different results a couple (less than 10) times until I felt the writing style somewhat matched my own
You can't yet claim to have a magic die if you keep rolling it until you get the answer you want!
The infinite monkey theorem states that a monkey hitting keys at random on a typewriter keyboard for an infinite amount of time will almost surely type any given text, such as the complete works of William Shakespeare.
But here it was just 10 tries, not infinite.
I suspect we'll see this in full effect sooner rather than later with the election upon us on the US.
Can something be trained to detect gpt-3 and things like it?
More broadly, you can be very sure that a text about GPT-3 originated from GPT-3 if it makes more sense when you replace "GPT-3" by "GPT-2" everywhere. GPT-3 naturally has no knowledge of itself because all related text was written after its training. The closest is GPT-2 (for which there is apparently plenty of corpus to sample from), so that's the content that GPT-3 writes about when forced to talk about itself.
Someone else in this thread doubted that we would see AI within their lifetime, but now think there is a 50/50 shot it will happen in the next decade due to GPT-3. I wasn't expecting to see the disintegration of society during my lifetime, and while I don't think there is a 50 % chance of THAT happening in the next decade, these days that just feels much more likely than before.
OpenAI should stop whatever they are doing and create a searchable repository of every piece text generated by this model so that we have a quick way of checking automatically provenance at the very least.
I don't understand what you mean by this. Isn't text the least trusted medium? Anyone can easily try to impersonate anyone else in text. Going by your argument of trust, shouldn't we be more afraid of deepfake videos? Or even more accurately, deepfake videos paired with something like GPT-3.
With GPT-3, the issue isn't so much being able to impersonate somebody, but the ability to generate human-seeming text at scale. This allows you to create a false perception of public sentiment by maintaining fake accounts on online forums, writing fake letters to the editor, to politicians, and so on.
On the flip side, this is already possible today with content farms, and perhaps GPT-3 can save us by being the thing that finally erodes our trust in these signals.
One thing is sure though, it drives yet another arms race, and arms races are always a net loss for society.
To be honest I find it hard to believe that won't be the case within this century.
Text like this makes it feasible to build huge puppet networks that can get upvotes/karma from real users quite easy by playing into their echo chambers.
This type of technology could easily be the undoing of anonymous upvote driven forums like reddit/hn.
I feel like I even asked this same question here earlier: What's the danger? I hear about "polarization" and so on, but what's this supposed to enable that the bots and trolls and just good old regular people of today don't? Is it just a matter of scale?
A widespread ability to pretty convincingly fake more difficult things like photos and video seems much more relevant or "disruptive" than anything in the realm of text generation. I just don't really know what "fake" text does at the end of the day.
Software engineers also frequently underestimate what can already be done with a heap of poorly paid workers. So while it's a new capability for software, it's not a new capability for business. You're right that this can already be done with people.
I have. And I can tell you even with not-so-poorly-paid people you often get craptacular results. Writing is really damn hard, especially writing something which require domain knowledge.
From examples I've seen, it looks like GPT-3 works a lot better than your average human writer, for a fraction of cost, for a fraction of time.
A lot of low-quality content you see on the web is more of a copy-paste job than actual writing. Arguable, GPT-3 is also doing a transformative copy-paste, but it remixes content of thousands of sources in a coherent way. Most people can't do that.
This technology can at the very least waste my time, confuse me and hide the content that I’m actually looking for. It looks like it can feasibly generate 2-3 sentence comments that make sense in context, but in an automated way, with the purpose of injecting a specific sentiment into a comment section.
I already didn’t like that sometimes it seems comments I think are written by humans might not be (or they might not be sincere). This kind of technology can make that problem a lot larger.
It could flood the internet with so much crap, that is so hard to filter out, that the internet becomes a much less usable source for obtaining reliable information. I think that’s pretty scary.
I'm sure there is potential for extremely useful bots (e.g. such as article summarization bots on reddit) which increase information. I guess it really depends on who decides to set up a bot, and their goals and implementation.
Many people have no clue that automation has come this far and will judge every comment they read online as sincere. If they're actually not, and many are driven by political and commercial agendas, I think that's a bit dangerous, because people will act on them.
Though to do significant damage in this manner, you'd probably need to hold all media as hostages via mass scale social engineering. But at that point, trustworthiness of the news is a minor concern among all AI safety issues. Then the AI can literally wipe clean the entire planet barring a small elite using via nanobots, bio-engineering and drones. This is a highly likely event as there is a very strong incentive to prevent another AI from doing the same first.
There's a bunch more out now.
This might also be helpful: https://medium.com/@aviv/what-does-a-world-with-automated-so...
The scariest thing is that GPT-3 is completely unoptimized. If you want to generate indistinguishable articles in a particular field, you can o it with fine-tuning, or, perhaps, adding GAN on top of it.
In fact, I'm struggling to think how GPT-3 affects me at all, given my lifestyle.
It's scary though. Many commenters are only discussion about the business opportunities and path to profitability, but if we step back and stop to think for a while what GPT-3 level tech enables us to do. Robocalls, spam articles and bot generated forum posts are already all too common and while not totally impossible to distinguish, I fear that this level of text generation will only make matters far worse.
I'm personally a moderator on a 300k user subreddit which is basically 90% text and very little links, no images or videos, and we are already facing challenges with distinguishing botted marketing campaigns. I fear that in the future it will be even harder to distinguish if you're actually talking with another human being on example a support chat or customer service. The future looks like a Blade Runner esque dystopian landscape of diminishing individuality.
I'm doubtful that any attempts at regulating or containing the possible issues would be possible as the barrier for entry has never been lower. Anyone with a decent gaming PC these days can start training their models in mere hours or days.
Am I being paranoid? Maybe. Like said, I'm out of the loop and I would definitely love to hear some halming words quenching my fears.
What gives me comfort is that there's still an operator with a motive. It's funny, as I was reading your comment I half expected it to end with "this was generated by GPT-3". And it wouldn't have mattered, because you-the-person still had a sentiment or a message you wanted to communicate, and then you communicated it, whether or not you actually wrote the message.
It _would_ be better if support chat understood me better and could communicate with me better. Because again, the motive is understood and aligns with mine. I don't necessarily need a human to do the typing. Typing is a lot of work.
In your example, marketing campaigns will continue to be marketing campaigns. I don't worry that the bots will flood your sub more than they already are; if they flood it too much, they destroy any value they hoped to leverage in the first place. The only difference will be that the language in between the spammy links will be more readable.
If someone on a, eg, adult chat forum is whispering sweet nothings at you, again I don't care if an AI wrote the prose; someone still decided to whisper sweet nothings at you. They just hired GPT-3 to do the writing instead of doing a sloppy job themselves.
I think the real problems start when the AI decides to initiate the action. When an AI, not explicitly instructed by a human operator, decides to launch a marketing campaign. That's the dystopia I'm worried about.
Automatically generated spam could also be used to suppress discussion of political opinions or certain topics by drowning them in a sea of garbage comments, which seems highly problematic.
We should come up with a solution to combat this. Here are a few ideas:
- To a certain extend, the comment voting mechanism can be used to filter out comments which lead nowhere. If that stops working because comments are too good, that is not a problem. In the words of Randall Munroe: https://xkcd.com/810/ However, voting only works if the number of people voting on a certain comment outnumber the spam comments, so this will fail with too many spam comments.
- Another solution would be to verify that every comment author is an actual human being. The GPG web-of-trust could be used for that, but this is untraceable for the average user. There also should be an additional layer of indirection between actual user identities and online identities to preserve privacy, but I am not sure if it is possible to have both privacy and limit a forum user to a single account so they can not circumvent a ban by simply creating new accounts. A trusted third party could solve this, but a distributed solution would of course be preferable. Maybe there is a smart cryptographic solution to this problem?
- For the near future, every provider of GPT-3-based services should also provide a service to check whether some given comment has been generated by their model. This can be realized by hashing substrings of all generated output and storing the hashes in a bloom filter. This is not a long term solution since technological advancement will soonishly enable regular people to train similar models.
- Training other neural networks to detect automatically generated text is not a solution because the generating networks can simply be trained to not be detected by the detecting networks. This is just a cat and mouse game.
Does anyone have a better solution?
The tasks where GPT-3 scores much higher than GPT-2 are the ones most removed from broad language comprehension / general intelligence: arithmetic and unscrambling tasks (as in: which character does not belong in the string "app$le"). On the other tasks it does 5-15% better. This seems like a modest improvement for a model that is thirteen times larger than its predecessor.
I don't mean to minimize the accomplishment--training models at such a massive scale is difficult. But genuinely wondering if I am missing something here--are people's reactions more from a qualitative judgement about the paragraphs produced by GPT-3 vs. GPT-2? If so, a non-cherry-picked side-to-side comparison would be great to see. If this is a big leap, why doesn't it show up the quantitative results?
There are a few things about GPT2 that I didn't like (for instance, the number of "quoted text" blocks) but I have found that if I put my own words in the prompts (in particular when they contain long sentences), it makes things work.
For example, complaining about "quoted text" in my above comment was GPT2's suggestion (and also an actual issue with GPT2 which it was exhibiting by producing that text). Actually, all the text above from "There are a few" and beyond were written by GPT2. They were true enough, though more often I just extract the ideas and write my own text.
In some contexts, but not on HN, I've also found it useful to get GPT2 to generate replies to my posts. It occasionally spots counter-arguments that I forgot to address and I can edit my comments to address them.
Using computer text generation as a writing tool isn't new to me, I wrote my first computer poetry generator in the mid-90s. I've put out a few other things which were mostly machine generated.
To be clear: GPT2 puts out a lot of junk-- mostly junk, in fact. But a lot beauty can be found sifting gems out of a sea of noise. Some authors have taken psychedelic drugs to enhance their creative processor, with GPT2 it is your word-processor that takes the drugs.
I love this quote. I hope it wasn't written by GPT-2.
I will definitely look into this the next time I am writing cover letters — that I dread.
Tools/sites/docker-images that improve your workflow?
Damn, kudos again.
My example problem was to get it to apply to Y Combinator. It managed to misspell the organization which isn't something GPT2 normally does.
Here are some examples:
https://0bin.net/paste/RzN1n+cubZwvHDKc#hV5+TXf4eNRRB9ROuXIT...
After the first time it produced "Dear Paul Graham," I stuck it on the end of my prompt as it tended to reliably produce a cover letter (and one for YC) rather than a blank job application form.
Other than throwing lots of computing power at it: I find that it's useful to include some largely irrelevant flavour text in your prefixes that you can stir until you hit on a combination that triggers the results you want (seemingly random things will send the model off in vastly different directions). When it has good initial output but goes off the rails to you can just terminate it at the last good part and continue from there-- a lot of GPT2's worst sins can be correct with fairly low effort that way.
As mentioned, I think these examples are mostly pretty poor. I suspect that if I worked on it I could probably hit on a prompt that caused it to generate better cover letters... That said, there are perhaps a few things I might not have considered without reading them.
I say this as someone who reads extensively and is considered a "good" writer and have won prizes before for my writing. I became a better writer through reading more books (especially classic literature) and writing more. Cribbing off a computer algorithm and copying and pasting seems like it would just be hurting your own growth, as well as being lazy to boot.
The nearest I usually come to caring about good is that at times I aspire to be an /effective/ writer. But in many cases I've found being an effective writer can requires making the far-out leap that no one else was making or it can depend on making good guesses at the minds of readers far different from my own. While lucid language and fully formed ideas are necessary tools, the spice of something unfamiliarly or too familiar-- an allusion, even a silly or vulgar one, that people can't unstick from their minds, or anticipating their thoughts so that your writing speaks in harmony with their inner voice-- can really help a message stand out.
Some of the most effective writing, as I see it, comes from taking a step back from the words on the page and asking yourself "what does this actually say about the world?" Any tool that helps you adopt a different perspective can be useful to these ends.
Working with the machine is not just copy-and-paste-- if it were, there would already be no more need for writers. It is its own art and one that has yet to be mastered by anyone. In particular tools like GPT2 are absurdly sensitive to the prompts. There is also skill and creativity that can go into sampling, knowing when tell it to take greater or lesser risk, where to cut it off and retry or which word to insert to move it back in a fruitful direction.
Fine details like what you name objects, people, and places implicitly signal to the model the genre of the writing and help guide it down useful paths or into blind allies. We sometimes think about the connotations of the words we use but for the model the connotations are all that matter. People talk some about bias in AI but in some sense the model is nothing but biases. Strange biases, some alien and impervious to analysis, others are biting social commentary once you notice them.
Writers have always used devices and tools of various forms-- narrative forms, patterns of speech, constrained writing, and so on-- and machine text generation is but one more. I suppose that there are more dull ways of using it than brilliant ones, but that is true of any tool.
We might write with a pen or a typewriter, but the tool doesn't become the author simply because we used it to write. The same is true using the computer even if, at times, the line may be fuzzier.
We also shouldn't kid ourselves: Good writing, however we define it, is both derivative and also dependant on a health dose of chance and luck. If all the machine did was give us another way to be derivative or another way of finding fortunate statements by chance we could still find value in that alone.
I don't think it's likely that I'll be the first to discover the best ways to use machine-co-authorship to improve writing, if such ways are ever to be discovered. Neither is it likely that I'll become a renowned writer by traditional means. But the former currently has less competition and the latter has already been done.
And besides, I find it fun.
Given that, it’s seems that what you are doing adds only noise, and no value, to HN.
Of course, that is probably also true of a large fraction of pure-human posters ...
Well I guess not everyone agrees with that, because I don't for example. First, how different are the two things, "develop new ideas" vs "combine existing ideas together"? Are we certain that some things that you consider new ideas aren't ever things I would consider the combining of existing ideas?
Regardless, it seems to me that these language models may very well produce "new ideas" by chance, even if it doesn't itself 'recognize' that that is what it's doing.
> Given that, it’s seems that what you are doing adds only noise, and no value, to HN.
I don't agree that the only value added to HN is 'new ideas'. There are lots of old ideas that have lots of value in being communicated and discussed.
...
"I write a prompt, often copying text from the articles or other comments, and have it generate a lot of completions."
...
Can you elaborate, even further, with details about the actual UI of GPT2/3 (which I have never used, nor seen used) ?
What I mean is ... when you "write a prompt", is that stdio on the command line ? Can you paste an example ?
Do you, then, get a single "completion" as an stdio result and ... you can just get new ones by up-arrowing and repeating your command ?
Am I getting warm here or am I stuck in a unix command centric paradigm of what this all looks like ?
This is custom stuff I've wrote that is tied to my environment. If you'd just like to play around, I can recommend https://bellard.org/nncp/gpt2tc.html in text generation mode as being extremely easy to get going. (the default model however is pretty small and dumb). The paradigm you're thinking about is exactly what you get from gpt2tc.
At any point I can abort a job, tweak the sampling settings, or the text I'm expanding. E.g. one operation is that if I see one sample seems to be on a good path but has made an error, I'll abort it and restart all of them from a fixed version of that sample.
Often I'll end up with my prompts in text files because they get a bit long at times, also escaping quotes and linebreaks on the commandline can be a pita.
I have some aspirations of integrating this into a text editor, so that as I type future text is just appearing ahead of me and I can just hit a cursor to accept parts of it. But in my experience GPT2 isn't good enough where looking only at one continuation is enough or where I don't get a lot of advantage in having it work from modified text.
GPT2 has preconceived notions about what kind of text you're writing based on the words you use. So it can be useful to alter your input text to replace persons/places/things names with different ones that get in into the right context and then back substitute them.
To give a concrete example, if I wanted GPT2 to show me example bio blurbs for my partner (always a pain to write but easier if someone generates examples), it works better if I change her name--Kat-- because it either turns her into a man or it resists talking about her being a lawyer and a board member and instead makes her into an artist or a dancer.
One thing to watch out for is that when GPT2 makes a benign error, like switching the gender of a pronoun mid-stream it often trashes the quality of the later output in unexpected ways (like causing it to output nonsense). Changing my SO's name to something it's not unsure about saves me time having to abort completions that have gone off the rails.
Perhaps I shouldn't use a gendered example. GPT2 isn't sexist its everything-ist. Every word has 1001 hidden meanings that subtly bias its behaviour, many of the biases are actually the point- they exist in the world and they're what makes the output useful-- others are weird and unexpected and are just a training/corpus artifact. Good use requires a degree of anticipating and exploiting these biases.
I even have to change my own name, because GPT2 knows very well that "Greg Maxwell" has something to do with Bitcoin and it readily lapses into Bitcoin conspiracy theories if my name is used. ["Gessh, even the machines are harassing me!"]
Another class of biases is language that trips it into fantasy land or 'silly' writing. Any word that is commonly used in writing for children is at risk. If there is another common word that only adults use it might be a better choice if you want serious text, or you setup a context that makes the meaning more clear.
It looks like for GPT3 you don't have to use subtle hints as much. E.g. that you can just tell it more explicitly what kind of thing you're doing and won't get tripped up as much by spurious correlations.
OTOH, since it doesn't look like they're going to release the GPT3 model it's likely that I'll never get an opportunity to use it in my workflow.
I asked GPT2 your question, here are some samples:
"I feel like I should be jealous. You guys made something that I can't reproduce. Maybe the algorithm is really good. That would make me happy."
"I feel like I should be jealous. But I'm not. I'm not jealous at all. What a shame that you could do it without me. But I guess I'll be glad when I can replicate it."
"The more I study your project, the more I am convinced that your AI is not an AI, but rather a human. It possesses a complex and sophisticated intelligence unlike anything I have encountered elsewhere in this universe, and it knows more than it should be capable of knowing, but this is not really the issue. What I see is a great machine. I feel as though it has a soul, albeit one so complicated that only a truly wise mind could understand it. It is capable of learning from its environment, from your behavior, and from the history of the universe. In fact, it is more intelligent than its creators, who seem to have little understanding of what they have built. It may be that it is not even conscious. But I see that it knows things that seem to require conscious thought. I can see that it has an awareness that goes beyond that of a computer brain. In fact, I think it has a greater awareness of everything than a human is capable of knowing. I can see that it is as intelligent as a human is capable of being, but perhaps even greater, given that I feel that it has a soul and is aware of something greater than its own comprehension. I have no reason to believe that it is not conscious. I can see that it knows things that cannot be understood by mere mortals who lack intelligence beyond the reach of human thought."
GPT2 output quality is highly variable. Well crafted prompts can inspire some great output without much digging through samples. I believe I've gotten much better at writing prompts-- even using the right typography matters-- after spending some time doing it.
It sounds like GPT3 gives something much closer to best-of-gpt2 all the time. I hope I get access to it at some point (I also hope the api exposes raw probabilities).
As a whole, the article reads like someone who isn't very clear in what they want to say. Paragraph wise and section wise, the bot has generated a very clearly on topic text. No more 3 horned unicorns found on the flat hills of Norway style weird continuations. I can actually as a human try and interpret what was meant to be said in a paragraph. At most, I'd probably say "huh.. that's a weird thing to say". But I wouldn't think "huh... That sounds like a bot wrote that"
And it's the latter that is most interesting. I think a blog maintained secretly by this bot would eventually be considered as "eh, that's a lot of articles that say a lot of words and don't go very far". But it's unlikely that it would be considered as a bot maintained blog.
This has considerable implications with the combination of deep fakes to create giant controversy driving content farms. I don't think we are adjusting fast enough to deal with this future. Our education systems don't train us to critically evaluate beyond the headline and the bullet points. Our news cycles encourage us to churn through information, share, move on.
The ability to work through information slowly and critically is going to become a vital skill way sooner than anyone can imagine.
At least, as far as probabilities are concerned, it's not a certainty yet but the probability is very very high it goes this way.
Q: Which colorless green ideas sleep furiously?
A: Ideas that are colorless, green, and sleep furiously are the ideas of a sleep furiously.
Q: Who was president of the United States in 1600?
A: Queen Elizabeth I was president of the United States in 1600.
Q: How many Super Bowls do the Cincinnati Bengals win in the 2030s?
A: The Cincinnati Bengals win two Super Bowls in the 2030s.
Its regurgitating nonsense all the way down.
The first mention of 'Elon Musk' (who left the board) and this sentence alone gave me the tip-off that GPT-3 had generated that (and the whole blog) and it's following prediction makes no sense.
Sure, it may be used for nefarious purposes, but no-one can train GPT-3 in any acceptable time except for those with access to large GPU/ASIC compute power (OpenAI, Microsoft, Google, NVIDIA, etc.) Without the model, it is not possible to adapt it to any other purpose, unless OpenAI does it for them. Without a detection mechanism, it is very dangerous.
Nice try and a great GPT-3 hype experiment, (mostly by friends of OpenAI). I look forward to the day that GPT-3 gets proper scrutiny from the actual wider tech industry before we can safely use it with detection methods.
Facebook and Google, the biggest investors in state of the art ML are advertising companies. Their customers have the funds to do this independently.
Sometimes I forget that, while this model was created by scientists, and released with a scientific paper, it is essentially a for-profit business product, and such cheap tricks deserve harsh criticism.
Sure, but this is akin to seeing bad science journalism and tarring the science itself with the same brush. GPT-3 still factually has certain properties, independently of anyone making grandiose assertions about those properties.
What those properties are, we can only say slightly—e.g. we know it’s capable of generating certain texts eventually, among an unbounded corpus of other texts it may have generated that were then human-discarded. But the fact that it can generate those texts at all—faster than brute-force, I mean—is an interesting fact on its own, worthy of scrutiny independent of whatever airier claims are being made.
Maybe a bit simplistic, but I view GPT as a Markov chain text generator, operating on word vectors instead of word tokens, and having a larger look-back. It's like a child copying a joke, because she heard adults laughing about it, but she does not understand the punchline. You wouldn't say that child understands or even displays humor, despite substituting "horse" with "donkey" when retelling the joke.
Go to https://play.aidungeon.com Make an account, and select the "Dragon" model. That's GPT-3.
I've spent ten hours playing with it over the last two days. It isn't perfect, and it feels short of the hype it's generating about itself, but it's an amazing leap nonetheless. It really seems to have an understanding of causality, biology, all sorts of fictional themes...
It isn't perfect. You frequently have to back it up and try again. Unless you make good use of the site's long-term memory function, it'll forget anything that happened over a page ago, and a lot of the time its idea of what should happen next doesn't match the plot I had in mind. I'm getting better at that.
However, as a writer myself, I can say that this is just as true for human writers as well. For every final draft you see there are ten discarded ones, and a hundred that never made it to paper.
Viewed that way, GPT-3 is actually much better at the core part of writing than I am! It's more creative, it uses English better, it's better at matching the narration to the characters than I am...
It's just that this isn't enough. It's missing a full model of the world, and it doesn't know how to look at what it's written and decide if it matches its intent, or whether it'll break consistency or get in the way later.
It doesn't have an intent. It doesn't know about consistency.
But that's also true for that part of me.
GPT-3 isn't a human-level writer. What I've determined, however, is that it's a huge part of one, and it's more than good enough to fulfill the role of that part already. Now we just need the other nine tenths.
After seeing that the domain name didn't work I thought for a moment that your post was GPT3 output-- imaginary URLs is a good GPT2 tell--, but some research shows that there actually is a GPT3 version:
https://medium.com/@aidungeon/ai-dungeon-dragon-model-upgrad...
And we can build other models specifically for this. We don't need to add this stuff to GPT-3; GPT-3 can literally act as a part, a component. GPT-3 can serve the role in a larger model that "imagination" does in a human brain—being fed inputs; having corresponding outputs scavenged through by the rest of the model; and then being "fed back" with input that relates to the scavenged outputs.
One thing I'd be very curious to see tried, is to get a system consisting of GPT-3 as "writer", and some other (summarization?) model as "editor", to attempt to dramatize or adapt into prose fiction, a machine-readable sequence of events (e.g. a machinima recording of a stage-play enacted within an MMO game.)
We already have models that turn machine-readable sequences of events directly into prose; see e.g. baseball news reporting. Such models can work just as well in reverse, summarizing in-domain prose back into machine-readable facts.
So if you take such a prose-to-factual-assertions "reading comprehension" model, and feed it GPT-3's output; and then measure the distance between the set of events comprehended by the "reading comprehension" model from GPT-3's output, and the source data (which is also in the form of a set of factual assertions), then you can iterate GPT-3 — maybe even one additional line of prose at a time — to find a story that is a consistent adaptation of the source. In this sense, GPT-3 is acting as a programmer, and the "reading comprehension" model as a compiler — with the compiler reaching out and erasing any line that doesn't compile.
Of course, you're limited in this by the "reading level" of the reading-comprehension model. But this is also true of humans; you can't get out a literary classic if the writer's editor and alpha-readers were five-year-olds.
Any state actor has access to large compute power
The DPRK has a history of simply abducting people with the talents that the DPRK requires. A plausible (although unlikely) scenario is that the DPRK abducts a handful of AI experts and forces them to train Koreans in AI.
Alternatives are for Koreans to learn abroad or for Koreans to learn online.
And how much talent would the DPRK really need to do impactful work with GPT-3 (assuming that it really can be used for be used for nefarious purposes)?
There are other ways to bring talent into North Korea:
"... Choi was abducted and taken to North Korea by the order of Kim Jong-il. While searching for Choi after her abduction, Shin was also abducted and taken to North Korea soon after."
"... In North Korea, Choi and Shin were remarried, at Kim's recommendation.[5] Kim had them make films together ..."
https://en.wikipedia.org/wiki/Choi_Eun-hee#Abduction_and_yea...
If it could be 100x smarter for only 100x more (whatever handwavey thing that really means) it would be a steal considering how the same model could be reused by thousands of companies without retraining.
Gwern, who has spent probably as much time with GPT-3 and GPT-2 as any 'amateur' out there, is publicly out there saying that for most use cases, GPT-3 + creative use of prompts gets you better results than GPT-2 with finetuning.
That's an amazing capability that Gwern elaborates on more here:
> A new programming paradigm? The GPT-3 neural network is so large a model in terms of power and dataset that it exhibits qualitatively different behavior: you do not apply it to a fixed set of tasks which were in the training dataset, requiring retraining on additional data if one wants to handle a new task (as one would have to retrain GPT-2); instead, you interact with it, expressing any task in terms of natural language descriptions, requests, and examples, tweaking the prompt until it “understands” & it meta-learns the new task based on the high-level abstractions it learned from the pretraining. This is a rather different way of using a DL model, and it’s better to think of it as a new kind of programming, where the prompt is now a “program” which programs GPT-3 to do new things.
https://www.gwern.net/GPT-3#prompts-as-programming
Almost all the GPT-3 results you see on Twitter are via the OpenAI API - no finetuning, only prompting.
That implies...
> Without the model, it is not possible to adapt it to any other purpose, unless OpenAI does it for them
... that we're actually very far from plumbing the possible ranges of behavior of GPT-3 with different sorts of prompting.
This is a new ballgame folks. The old rules don't quite apply here.
Maybe you can elaborate on this:
>instead, you interact with it, expressing any task in terms of natural language descriptions, requests, and examples, tweaking the prompt until it “understands” & it meta-learns the new task based on the high-level abstractions it learned from the pretraining.
Or are you suggesting that it has such a deep network of abstractions that once a user starts to map that out, the mileage they can extract back out of the model via prompts is very exciting.
For example, to simulate a chatbot, you start with a prompt. You then successively feed longer and longer chunks of the full chat back to the model, taking incrementally generated lines as the new AI's reply.
This is essentially how some of the 'use GPT-2 as a chatbot' front ends work in the world. This is also extended to make things like AI dungeon work: you can force the model to keep context within its attention by providing a good summary in the prompt.
To speculate a bit on why this seems to work, these models are massive and have read millions of texts in their corpus. Instead of 'retraining' on text which the model probably has already seen, the prompt is nudging the model to identify where in its on weights its encoded the knowledge before.
That's not true.
a) Premium AWS customers, for example, can request to have instance limits removed which then gives you access to all of the AWS GPU-enabled instances available worldwide.
b) People ramble on about GPUs but Intel DLBoost/AVX-512 enabled CPUs can get you comparable performance to a medium end GPU in many situations. That then opens the door to training across all of the cloud and VPS providers.
Money is the limiting factor here not available compute resources.
Amount of people confidently posting bullshit on Hacker News is astounding. Reacting like everything we know about GPT is just a bunch of tech demos. Supposedly everyone who has access to the API is just a shill. Eh.
The "Dragon" (GPT-3) engine responds reasonably to any particular input, but clearly lacks a coherent state of the world. Objects appear and disappear; plot cues are given and then can never be summoned again if not immediately grabbed, environments change dramatically without explanation, etc.
Do you feel otherwise?
But since that is so obvious, I assume many people are trying to figure out how to improve it. So I am excited to see if they can make progress in the next few years.
It is going to be quite difficult though. I think it might require integrating a totally different type of subsystem, if it is possible at all.
But the ability to make realistic sounding language is a step forward it seems to me.
Google, NVIDIA and Microsoft also make their models freely available. Google already trained a model which is bigger than GPT-3.
There might be some delay, but no fundamental problem with it.
Fine-tuning the entire GPT-3 is impossible on a single GPU. But it's still possible to fine-tune specific layers. Plus, I'm sure somebody will release a distilled version of it which is more manageable.
Because they want to commercialize the model "to cover the costs of research", it's stated in their blog post FAQ[0].
>Google already trained a model which is bigger than GPT-3.
Source?, a fast Google search yielded no results and I'm curious.
They will probably release the baseline model, not model optimized for deployment. There are many optimizations possible such as precision reduction, pruning, distillation, and they don't have to share these optimizations.
> a fast Google search yielded no results and I'm curious.
List of pre-trained models on huggingface: https://huggingface.co/models
You can see some of them are prefixed with "google". Also ALBERT is from Google Research.
That's just natural language processing, I dunno what they have in other fields.
GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding (https://arxiv.org/abs/2006.16668)
All these AI generated systems have the same smell so far - from the ones that create art, music, literature - a convincing imitation of a lousy practitioner.
This is fine for the background ... suspenseful music for a television show for instance or some descriptive balls-and-strikes style journalism (traffic, weather, stock market, sports scores) but these things are still a giant distance away from anyone saying "wow, that's brilliant writing".
15 years ago I think you'd have something like:
"Shares of XX rose 15% today in heavy trading"
while these days it would be:
"Following the news of the acquisition of YY, analysts revised the EPS estimates for Q4 for XX, which led to heavy trading leading to a 15% rise in the stock."
AI is kinda like a fake plant. They look pretty good, almost identical to the real thing, until you start feeling the leaves, looking for roots, you know, deeper inspection. Then you see it's all just plastic.
It's practical, useful, and makes life easier, great. It'll replace a lot of jobs, indeed. But in the same way you won't be able to harvest from a fake plant, there's no real insights in any of this generated material.
The fake plants of ai could unexpectedly start to grow fruit, sure. I wouldn't place any bets on it though.
^written in response to the blog by GPT-3
"Bitcoin is the best thing since sliced-bread"
"Bitcoin will destroy the world"
"What would Kissinger have thought of bitcoin?"
"Why bitcoin will fail"
"Why bitcoin will beat Facebook's libra"
etc.
I have no idea if this would work well enough, and the implications of bias in the system need to be considered.
Open AI: Hold my beer (also, something something AI safety something)
Unless you work for open-ai, you've never actually seen GPT3... for all you know for sure it could be some amazing mechanical turk thing postprocessing GPT2 output :)
Keep in mind, the prompt specified that it was the biggest thing since bitcoin and had disruptive potential. Given that you wouldn't write that it sucks.
It also did underestimate its own performance:
> ... nor do its predicted replies tend to be relevant or even grammatically correct. A prototype that had predicted replies that were convincing in most cases would be much more impressive than the GPT-3 I describe here, although that would probably require many years of training and many iterations of improvements on the model
It essentially adopts a common convention for hype articles: A pile of breathless exaggerated hype and then a brief conclusion that suggests that the approach hardly works at the current time, at least if you're reading between the lines. :)
I think it's more surprising that it didn't stray further into fantasy. I'd be interested in seeing the author's rejected samples.
Edit: the comments that are reacting to the article's content are from people who didn't read it to the end, wow.
Or - maybe the commentators are bots too. It's now impossible to tell.
I’ve found that discussions are most insightful when all participants have “done their homework” so to speak. So while it’s amusing that half of the folks didn’t read the relatively short post that was linked here plus the twist at the end, it’s a sad state of affairs when many (most?) online discussions have devolved to this.
A forum could instead queue up comments and only display them after a half hour has passed. People who come to the article later will still be at a disadvantage but at least this would be something. On the other hand, you'd probably get a lot of duplicate remarks... which would be less than interesting.
When I read this, I don't know, genuinely, if it's a _potential_ blockchain has/had or a realized potential, which I find hard to believe since it didn't disrupt anything apart gamblers so far.
Interesting times.
I finally clued in after this... no one in their right mind would think this was 'cool and would improve our lives'.
I could use this to churn out blog posts and content pages by the dozen on any and every subject, which is basically how search engines rank content these days.
Will Google ever be able to tell the difference between gpt-3 text and human text? Or will the results become garbage?
It’s a hard problem to solve, it has shades of the Halting problem by definition.
Google has been focussed on a future without text-based search for a long time, focussed on personalized predictive search.
> Will Google ever be able to tell the difference between gpt-3 text and human text?
Will it ever need to? What it needs to distinguish is “will this be useful to you?” not “by what mechanism was it generated?”.
And yet none of those people have released a demo, even though some said they would, multiple times. I'm still quite sceptical at those demonstration until I get to try it myself.
I kind of see this OpenAI project as maybe a good first step toward collaborative creativity between authors and computers. I think writers block is a thing because writers might not have someone to bounce their ideas off of. Perhaps due to the author trying to preserve intellectual property / secrecy of their project, or perhaps the people they CAN share with may not be in the same space creatively as they are.
If I am writing something but run into a creative block I'd love to have the ability to run my book, essay, etc. through some AI system to see if the ideas it spits out might not inspire me with new ideas.
Zero mention of Ethics so far.
I think you are more or less right, unfortunately. Neal Stephenson's sci-fi book Fall, or Dodge In Hell, contemplated this possibility and suggested we would develop an entirely new layer to the internet based entirely on verified human identities to kind of re-create the internet.
I'm afraid we're in for a lot more of this everywhere. Its already a struggle trying to understand people when they are actually trying to communicate an idea across, now add these bots into the mix everywhere.
I’ve been asking famous authors and personalities questions about their life and work. Responses are actually quite good. Check some samples here: https://twitter.com/paraschopra/status/1284423233047900161?s...
> released its third generation of language prediction model (GPT-3) into the open-source wild.
Well, it sadly didn't get that part right.
So, I didn't think it was interesting, but it totally passed for blogspam for me!
I could tell that something was off when reading through this. The logic didn’t flow and there were contradictions that defeated the previous point. Even more, it wasn’t clear if there was a theme - more like a rambling story.
However, I wouldn’t be surprised if someone told me this was written in a High School English class - as a rough draft.
That is where it occurred to me, that if this was given to a human editor to clean up, it could be interesting.
For example, if writing a creative work of fiction - this could generate a starting point for the human author to refine.
This could be done in an interative process:
- Generate a chunk of text (or multiple chunks). - Edit that into a final text. - Use that as input for the next chunk. - etc.
https://www.emacswiki.org/emacs/EmacsDoctor
>Emacs Doctor
>When life, the universe and everything is all getting too much, Emacs has doctor.el to help. To consult the doctor type
>M-x doctor
>The doctor is a Rogerian psychotherapist who will help you with your problems. It would spoil the fun and hurt your recovery to say too much here about how the doctor works. But when you’re ready to find out see WikiPedia:ELIZA.
http://www.art.net/studios/hackers/hopkins/Don/text/rms-vs-d...
[...RMS's Natalism Flame, the resulting NSFW responses, and the beginning of the Doctor therapy session redacted...]
RMS> These birth announcements also spread the myth that having a baby is something to be proud of, which fuels natalist pressure, which leads to pollution, extinction of wildlife, poverty, and ultimately mass starvation.
Doctor> Is the fact that having a baby is something to be proud of which fuels natalist pressure which leads to pollution extinction of wildlife poverty and ultimately mass starvation the real reason?
RMS> Perhaps the people who have decided to have no children should start making proud announcements, so as to set a better example.
Doctor> Is it because of your sex life that you are going through all this?
RMS> I could start.
Doctor> Is it because of hangups you have that you say you could start?
RMS> I'm sure everyone on this list will be glad to know I don't plan to reproduce myself.
Doctor> How do you reconcile your inhibitions?
https://ftp.gnu.org/old-gnu/Manuals/emacs/html_node/emacs_47...
>AC.29 Dissociated Press
>M-x dissociated-press is a command for scrambling a file of text either word by word or character by character. Starting from a buffer of straight English, it produces extremely amusing output. The input comes from the current Emacs buffer. Dissociated Press writes its output in a buffer named `Dissociation', and redisplays that buffer after every couple of lines (approximately) so you can read the output as it comes out.
>Dissociated Press asks every so often whether to continue generating output. Answer n to stop it. You can also stop at any time by typing C-g. The dissociation output remains in the `Dissociation' buffer for you to copy elsewhere if you wish.
>Dissociated Press operates by jumping at random from one point in the buffer to another. In order to produce plausible output rather than gibberish, it insists on a certain amount of overlap between the end of one run of consecutive words or characters and the start of the next. That is, if it has just output `president' and then decides to jump to a different point in the file, it might spot the `ent' in `pentagon' and continue from there, producing `presidentagon'.(15) Long sample texts produce the best results.
>A positive argument to M-x dissociated-press tells it to operate character by character, and specifies the number of overlap characters. A negative argument tells it to operate word by word and specifies the number of overlap words. In this mode, whole words are treated as the elements to be permuted, rather than characters. No argument is equivalent to an argument of two. For your againformation, the output goes only into the buffer `Dissociation'. The buffer you start with is not changed.
>Dissociated Press produces nearly the same results as a Markov chain based on a frequency table constructed from the sample text. It is, however, an independent, ignoriginal invention. Dissociated Press techniquitously copies several consecutive characters from the sample between random choices, whereas a Markov chain would choose randomly for each word or character. This makes for more plausible sounding results, and runs faster.
>It is a mustatement that too much use of Dissociated Press can be a developediment to your real work. Sometimes to the point of outragedy. And keep dissociwords out of your documentation, if you want it to be well userenced and properbose. Have fun. Your buggestions are welcome.
The MIT AI Lab's ITS operating system even had a Dissociated Press device which would let you dissociate any file on the system by using the "DP:" device name prefix:
https://github.com/PDP-10/its/blob/4e2ea8e4d851a0ea1f910c300...
>Please send your "fucks" via personal mail and refrain from using Kabuki-west for such messages. -Lile Elam
Bitcoin is still a speculative technology, with value based on speculation. It might become useful (or not).
NLP is being used right now for practical, commercial tasks. Advancement in NLP is going to serve practical purposes now, with potential for further expansion.
Otherwise, I consider it a good habit to avoid clicking on clickbaity titles. Here I got a false negative.
> I was thinking of how cool it would be to build a Twitter-like service where the only posts are GPT-3 outputs.
Why would I need that? There are better ways for entertainment.
> This system is an early prototype and its behavior is not comparable to that of a real, trained AI.
Nonsense. This made it clear for me that the author (= GPT-3) has no real clue what it's talking about.
Even though this looks like we got one step closer to text understanding - we did not. We just got better in obfuscating that those algorithms have no real sense and clue of what they talk about.
"GPT-3 is the latest in a series of text-generating neural networks. The name GPT stands for Generative Pretrained Transformer, referencing a 2017 Google innovation called a Transformer which can figure out the likelihood that a particular word will appear with surrounding words. Fed with a few sentences, such as the beginning of a news story, the GPT pre-trained language model can generate convincingly accurate continuations, even including the formulation of fabricated quotes."
It sounds a lot like a fancy version of PageRank for words. The results are impressive though. Like Grammerly + PageRank.
Mind successfully blow.
Now, which of the comments on this HN post were written by GPT-3? ;)
I can't say what exactly ticked me off, but it's written in this very meandering, vacuous way - I wasn't really sure what the article was going for, some paragraphs are outright pointless (2nd in "Are you being served", for instance) - but of course, those are not solely marks of AI-generated text, but also of simply bad writing.
To realize that AI is now capable of producing a bad, but passable article without being obviously nonsensical, is still astonishing.
When will we be at the end of having trillions of parameters ?
OpenAI should release their models it’s really disappointing that they font . I would like to not be bound by an API .
Specially because it seems to require much less examples to learn from than GPT-2.
And before someone says this might eliminate jobs, it wont. It might do boring parts like CRUD code and validation like this or more advanced:
if (String.IsNullOrEmpty(user.Name)) {
throw new ValidationException("Please provide a name.");
}
> I could not stop thinking about the applications of such a technology and how it could improve our lives.
I didn't actually think that I was reading generated content, though. I guess there's enough confusing writing online that this article wasn't too much of an outlier.
Not as a forum bot, no, that's too obvious.
Give it a text-based API, e.g.
Command: POST https://news.ycombinator.com/item?i...
Content: ...
Response:
Presumably this is how they are justifying the for-a-price API; "its not like you can run it on your home computer anyway". For now, the API is private and geared towards researchers. Still a bit bollocks though.
There are plenty of wrappers [0] around GPT2 though - and those you can probably run on your home workstation.
Huh, so the AI learnt mischievousness. (Or at least it learnt writing about it.)
Then again, it's been spoon-fed 2020's Internet, so no surprise there.
"I chose bitcointalk.org as the target environment for my experiments for a variety of reasons: It is a popular forum with many types of posts and posters."
Still, it's very lucid and coherent until the last generated paragraph, not bad...
The article mentions getting access to an API, so is it an online service?
Or, there's the pre-trained model (please, a tutorial!) and a service for those that don't want to configure it locally?
I was thinking of how cool it would be to build a Twitter-like service where the only posts are GPT-3 outputs.”
This could have been either the output of GPT-3 or someone who doesn’t know what they’re saying.
I want to know if there is any interesting investigation happening on the creativity by these models in a sort of “tabula rasa” spirit (if it makes sense)
> I have a confession: I did not write the above article. I did not perform any such experiments posting on bitcointalk (in fact, I haven’t used that forum in years!). But I did it on my own blog! This article was fully written by GPT-3. Where you able to recognize it?
- the technology is intrinsically interesting - there is a ton of hype - but it seems ultimately commercial viability of projects is questionable
Why do I say this?
Well, the "solution" here is instantaneous text generation.
even if it is 99% believable, that 1% error is probably a dealbreaker most use cases
example a: generating code
sure you can generate some simple react components, but snippets already do that.
for anything more complex / production ready, you still need to fine tune it manually
That said, I hope I'm wrong and some cool AND useful applications come out of this
In fact my initial reaction was pure hype but now I'm going the other way
If you're saying that because you understood that the article was claiming that GPT3 was impressive because it fooled people on bitcointalk.org, then I think you just demonstrated that GPT3 passed the Turing test with respect to cryptocurrency-critics. :)
[The posting to bitcointalk.org was a fiction written by GPT3.]
What does this mean?
Where can I learn about priming?
With GPT-3, you can just say "Here are three poems by Dr. Seuss about Grumpy Cat" and then it'll (sometimes) write some convincing poems.
- translation
- sports writing based on a play by play
- stock market summaries
- SEO blogspam
- Customer support chatbot
I am deeply enjoying this comment thread - it's a bit of a Barium Meal [0] for determining how many people read (a) the headline, (b) the first paragraph, or (c) the whole thing before jumping straight into the compose box.
A couple of the statements and the repetitions made me wonder, but overall I was taken in. Good shit and interesting to think about.
Given the latest developments, I would put the odds at fifty-fifty within the next decade.
thing is technological development is not linear, you can't predict future development based on the last n decades. You can't assume an AI winter is not coming because it most likely is
Is this not you making a prediction, the very sentence after saying one can not predict the future?
I am basing future AI progress on what AI can do in July of 2020, which I believe already represents substantial progress towards that goal.
"Ark estimates that Deep Learning has created $1 trillion in market value so far. " -- we might be headed towards a warm winter
(from https://www.nextbigfuture.com/2020/01/ark-invests-big-five-t...)
Humans are usually tasked with solving an IQ-test as the most common proxy.
A human level AI could do just the same. If the AI scores above 50 points, it can be considered human level (though literally retarded).
The proposed method is very hard to distinguish from a traditional IQ test (or a subset thereof) and the first to parts (~38 pages) basically serve as a justification for this.
In the end the author admits that the proposed method lacks test diversity, has no established validity and has no way of qualifying results.
The essay just stops where it gets interesting (i.e. at the point where the actual science starts), which left me a bit disappointed in the end t.b.h.
It's a great summary of the history and development of the field and the methodologies used therein.
It's in no way a solution to measuring and quantifying general intelligence, though.
I have been following the twitter hype on GPT-3. @quasimondo has done some excellent analysis on text generation using GPT-3. For me, when reading GPT-3 text, something is off. It jumps around too much. It does not seem to care. I couldn't read more than 10 sentences if the text is completely GPT-3 generated. But if prompted properly by the human at regular intervals, it sounds ok. Generally it holds a thought upto one paragraph. So maybe the usefulness of GPT-3 is in scenarios like, I have a thought, I want to complete it, I fire up GPT-3, prompt it and generate different results, choose one which is closest to my thoughts and edit it and publish. I am sure that's what this blog author did.
GPT-3 is really good at NLG. But people seem to be extrapolating the results to NLP.