Show HN: Talk to Transformer – Generate Text with OpenAI's Latest GPT-2 Model | Hacker News Reader

Show HN: Talk to Transformer – Generate Text with OpenAI's Latest GPT-2 Model

talktotransformer.com

93 pointsAdamDKing55 comments

Wow, this is good. I gave it "Configure your pipeline state object with a compute shader that consumes uniforms and vertex buffers." and it came up with this:

Note that the compiler will add a compute shader at compile time and require you to configure your pipeline state every time. This is the equivalent on a more traditional GPU where shader compilation occurs between the GPU and CPU. For example if you want to set the shader with a single pass on a CPU it is reasonable to compile it each time after GPU rendering, and even compile it once on the GPU. This is an extension of the parallel compile-to-script: run-to-script approach.

It's a very compelling illusion of actually understanding the topic. I've seen less coherent responses on Stack Overflow and, dare I say it, Hacker News.

> It's a very compelling illusion of actually understanding the topic. I've seen less coherent responses on Stack Overflow and, dare I say it, Hacker News.

I think you've highlighted a very interesting angle. A lot of writings by users on the internet have common grammar mistakes or more like, the apathy towards adhering to correct English grammar.

The true test for this sort of text transformer will be to generate an equally incorrect syntax and idiosyncrasy through writing style and skew towards the use of specific group of vocabulary (ab)used by the author, meaning an entire Reddit drama thread generated purely by AIs, complete with trolling, argument traps, and generalization, the complete toolbox of an Internet keyboard warrior.

edit: I finally got a response back from the generator: I typed 'lick my butt' and the result is shockingly good-its everything I'd expect from someone who would start off with that phrase: lick my butt. I don't know what kind of ass you got. I'll just crawl next to you. I'll crawl next to you like this, so that if you get a little pissed, you have time to take it easy and go back to enjoying the show. Don't make me go back to enjoying the show. You can go back to enjoying the show. You can walk with me. You can take me. You can crawl next to me. Don't go. It's okay. I'm just making you comfortable again. And, I don't want you to leave. And, it's fine. I'll let you keep my body. You can take me but don't keep me. I don't need you to keep me.

That's just the opening I mentioned from when she says goodbye and leaves him for it. It has been a while since I have been "comforted;" and not comfortable in a nice way. He's gone over her body in a way that just takes her from me, into my body again; but, doesn't take her any further. She continues to leave behind me that way, but we both know that this isn't just about me anymore. It's about her.

I don't know what to say to her at this point. As much as I want to go over what I experienced with her last time, I can't just let this go. After all, this wasn't a last-minute fuck because I couldn't come. It wasn't because he wasn't enjoying himself. It was because he has a girlfriend; and, he feels that she deserves to be comforted. (The way he says that doesn't work either; it's one thing if he needs me to be comforted, but it's another to tell her he wanted her too and he just needed time to enjoy himself. She deserves to be comforted too; after all, he is getting older and older.

Especially that line 'last-minute fuck because I couldn't come' is completely in line with the overall tone of the generated text. You would think this post was written by a 20 something red pill poster.

>After arguing and correcting people on the Internets for the past 20 years, the semantics became more urgent than the syntax.

[Continuation]

By the summer of 1995 a man calling himself "Wakefield" had launched a Facebook page calling for "an event on the Day of the Dead which was to be held in Washington DC, on June 30" in which "all manner of Satanic rituals were to be performed in a mass-exhibition". The event was to be attended by more than 2000 people and was apparently intended to be an "expose" of the occult. The message of the day stated, "Satanic rituals are a symbol of the death and resurrection of human beings and the cult of Horus [as well as related concepts] is in direct opposition to Christianity and Judaism" The date was listed in the same manner so that most people would have seen it as July 1, 1995. Some claimed to see it through July 31, just as July 1 marks the...

Well. I think Transformer certainly understands the cult-like nature of semantics over syntax. :)

After arguing and correcting people on the Internets for the past 20 years, the semantics became more urgent than the syntax, and as a result, I often read my own writing in aghast-the realization that when people write tl;dr they really meant it-because as soon as you hit submit the original meaning is lost and with our limited cognitive resources we choose to filter out only what we want to read and ignore the rest. One moment is worth 2, when it's only one second worth 4, when time has a finite amount you should stop reading everything and enjoy yourself. In this way, people stop reading before they know what they're looking at, and they are in the process of filtering out their own words and expressions. "How does this change my reading?," was my initial reaction, and it turned out to very different than how I thought it should change me if people realized the literal meaning of their words. Once people stop reading, they become increasingly interested in the meaning, and so they are not only more engaged in the conversation, but they actually have become more open to their own words and expressions.

[prompt] 1. d4 Nf6 2. Nc3 e6 3. Bg5 Bb4 4. e4 d5 5. e5 h6 6. Bd2 Bxc3 7. Bxc3 Ne4 8. Bd2 c5 9. Be3 cxd4 10. Bxd4 Qe7 11. c3 Nc6

[continuation] --------------- This looks like a very easy opening, but it is really a hard one. I don't really know what is going on on either sides, but since I have read on net about White's opening (for example, http://www.chessgames.com/forum/viewtopic.php?f=45&t=318749 ) I think Black has more of a chance here since he can attack the first four moves. But at least White is more threatening to the Bishop. Let's focus on this opening... A good opening for White is the g3. The bishop is too far back, there isn't enough space for b3, the c2 is too shallow to get the queen on time and the knight has a good escape from c7, and c7 seems a bit strong. This is not a good position for Black, but it may be able to take some games from Black. I have given

I really enjoy that it hallucinated a perfectly cromulent-looking, but invalid, chessgames.com URL.

LOL, it generates truly believable text.

> Elon Musk announced..... that Tesla, SpaceX, and SolarCity have signed an agreement to create the first national solar company—the Solar Energy Industries Industries Association.

The Solar Energy Industries Association will be headquartered at its HQ in California where Musk and company will spend $8.7 billion and generate more than 10 million jobs over a 10-year timeframe. The organization is an independent, non-profit corporation with members who include utilities, utilities industry organizations, tech startups, research organizations, and other energy utilities and technology companies. Solar Energy Industries Association (SEIA) represents the interests of hundreds of energy and renewable energy utilities (NREUs) like Suniva and Silevo, to the solar industry and the public.

Input: "how do you say hungry in chinese?"

Output:"lol lol lol lol lol lol lol LOL LOL lol lol lol lol lol lol LOL LOL LOL LOL LOL LOL LOL LOL LOL LOL LOL LOL LOL LOL LOL LOL LOL LOL LOL LOL LOL LOL LOL LOL LOL LOL LOL LOL LOL LOL LOL LOL LOL LOL LOL LOL LOL LOL LOL LOL LOL LOL LOL LOL LOL LOL LOL LOL LOL LOL LOL LOL LOL LOL LOL LOL LOL LOL LOL LOL LOL LOL LOL LOL LOL LOL LOL LOL LOL LOL LOL LOL LOL LOL LOL LOL OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG OMG"

Completing this actionable guide will require significant effort on your end, but just remember: validation dollars are infinitely cheaper than development dollars. And you don’t have to complete this guide in one sitting, so don’t be intimidated. So that's it! This guide gives you a complete overview of building a reusable business logic framework. To get a very comprehensive overview of this topic, see this article . The code for this guide includes unit tests for all of the elements necessary to achieve this goal - it is fully documented, and provides for an API that is straightforward to extend and customize. To learn more about testing, check out this post on TDD-Unit Testing . Feel comfortable using the code when reading the code. This is not a tutorial on how to create a framework, but rather a tutorial on how to build a business logic framework. I'll walk you step by step through building this framework, and demonstrate how it should function in production. You can read the entire code sample here . If you'd like to make additions to or remove code from this framework, then you'll need to write unit tests for the framework. A common problem I face when writing systems is, "what's the best way to add a new element to a single page item?" To answer that question, I will write out a functional imperative programming paradigm called the Agda Pattern. The Agda Pattern is named after the author of the original Agda paper, and as I've noted before , Agda has been used not only by Agda authors, but by other functional languages

Guess where the original text ends and where the model generated text begins :)

After "Completing". lame

I fed it a small snippet of Python code, and I got this back (edited to fix syntax errors):

  import matplotlib.pyplot as plt

  ax = plt.Axes()
  ax.set_xlabel(name='Person ')
  ax.set_ylabel(name='Name ')

  plt.scatter([df.name, ax.x(1/2)).sum(), ax.y(1/2))

  plt.col, ax.rect(name='Name')
  ax.show()

It doesn't make total sense, but it's so cool to see that AI has seemingly learned to use the matplotlib API, save for some unbalanced parentheses :-)

I wonder if you could make an RL version of a GPT-2 model specifically optimized for code, where you try to compile the output and penalize whenever there is an exception/error that's thrown.

It seems pretty good at matching parens and braces for C code, although it decided it was finished before outputting the closing brace for the function (the prompt was just the signature):

  void send_file(int* fd) { 
    int file_type; 
    char *file_name; 
    size_t name_len; 
    FILE *fp; 
    char *name = NULL; 
    if(fmt::error_code != 0) 
      return error("Could not set file type"); 
    fp = &file->fp; 
    size_t file_size = fprintf(fp, "%02x", fp->fname); 
    if(fp->fname == fp->fname || _FTP_FILE_TYPE == 0) { 
      fp->fp->fname = 0; // Do nothing for errors 
    } 
    /* Write some info to fp */ 
    fp->fp->fd = calloc(1, fd) + (1 + file_size); 
    fp->fp->sig = calloc(1, &file->sig); 
    fp->fp->buf = (int *)((size_t)sizeof(*fp)); 
    /* Make sure fp isn't open */ 
    if(!fp->closed) { 
      // Close fd with error() 
      fp = new(FTP_FDCALLTYPE) void(1); 
      fp->fdclose((void *)&fp); 
      return; 
    } 
    fp->fp->buf_len = file_size; 
    fp->fp->name = name; 
    file_name = name; 
    fp->fp->cwd = nullptr; 
    /* Don't close fd after this point */ 
    fp->fp->fdclose(fprintf(f, "fd open", TRUE));

Not too bad, but I'm not worried about losing my job to a language model anytime soon.

Not saying you should be worried, but this model is not the one that would be in the running for your job, in case that ever happened

Hey I'm _____, what do I do ____________?

When I was 11, my mom told me every day, "Don't say anything bad about my Dad."

I knew that was a lie, but I needed to say it anyways, so I wrote down the reasons why I wasn't nice to him. I would go to the bathroom and look back over my shoulder for a few seconds before walking around without noticing that my Mom had written down every bad thing about him on a piece of paper. I would go to school to catch up a few more papers.

I fed it "The true destiny for humanity is"

Out:

> The true destiny for humanity is to be the species it wants to become," says William James, director of the Center on the Ethics and Future of Work at Harvard University.

> While not explicitly suggesting humans would be replaced by machines, James predicts that as more and more people, businesses and governments work more closely with them — and machines can learn and adapt — they will be less constrained in their creative output and more willing to work within certain limitations, as long as it's done with a basic understanding and trust in that person.

> While humans seem uniquely suited to take part in the future, we are likely to take a more hands-on approach, including helping others create work with computers.

> "We are already seeing a lot of companies taking that step and saying, 'No we're not going to have human resources, they've got enough human resources already. It may work out differently with robots or with artificial intelligence, but there will be plenty of work done by humans still,'" says James. "But there are a lot of questions that we can't really

GPT-2 models' robustness and worst case behaviors are not well-understood. Our findings support the view that this may be a very conservative estimate of the true effect of low-risk paternal circumcision because our model has a relatively narrow definition of risk. The robustness of the model also highlights the possibility that the risk of adverse effects reported is indeed due to the high risk of unintended pregnancy. The observed risk of unintended pregnancies to women in a GPT-2 setting does not approach that observed in the US population. However, an even more likely explanation for our findings would be that unintended consequences that would not be present in the US population would have been lost due to the high prevalence of safe sex, especially condom use, during early childhood.

...Man, wtf.

Thanks for the upvotes! The site is running quite a bit slower than planned, but I think I know why. I should be able to get it going at full speed around tomorrow.

[Prompt]

Measles is a very contagious disease caused by a virus. It spreads through the air when an infected person coughs or sneezes. Measles starts with fever. Soon after, it causes a cough, runny nose, and red eyes. Then a rash of tiny, red spots breaks out. It starts at the head and spreads to the rest of the body.

[Continuation]

When someone gets measles, the virus can be carried into a host for months. But the virus may spread back to another infected person or even the infected person's partner.

If you have been exposed to measles, you may have mild or mild to severe symptoms of symptoms such as:

Vomiting (bleeding from your eyes and nose) but you are not sick

Headache

Vomiting that is a little bit heavier than usual

Vomiting that is small

Headaches but no soreness

A rash

A fever of 100 or higher

How to keep measles under control:

You shouldn't have it!

Don't let

Built by Adam King (@AdamDanielKing) as an easier way to play with OpenAI's new machine learning model. In February, OpenAI unveiled a language model called GPT-2 that generates coherent paragraphs of text one word at a time. The model is extremely useful to quickly generate meaningful sentences, but King made an incredibly elegant alternative with GPT-1 - the same technique he uses in his original game.

(The original, non-Pillowwordly version uses a combination of the words, "stupid," and "cunning." The most powerful version, which uses only the original words, produces sentences with "I'm pretty smart." As long as you remember to replace stupid with cunning - and remember to replace cunning with stupid - this will be a very easy puzzle for you, and it'll be just as satisfying, if not more so, if you're more experienced by learning GPT-1 instead. I suggest you give this tutorial a try and give us a share in the profits if you find it useful, then share the tutorial link with your friends and fellow Googlers!)

This is the same system we use to generate this piece of music, and it works even better. Our goal is to predict a song's exact words and phrases from scratch. We used a similar system in the original game. We used GPT-1 to generate words and phrases for the song, and then our words and phrases were randomly generated based on their meaning in OpenAI's language model. To put this in more understandable terms, to give you an example, we've used a phrase we've previously called "dude. I can see him doing that..." -

My socks are blown off. At my most unimaginative, this is a glimpse of what high-schoolers and tabloid journalists will be using to get filler for their assignments.

This was the result of a prompt consisting of a single chicken leg emoji:

Offline

Activity: 1234

Merit: 1000

LegendaryActivity: 1234Merit: 1000 Re: BCH vs XCM - How to decide September 28, 2010, 02:17:45 PM #18 You don't know anything about BCH or XBC. Do you have something that you know that isn't what the majority of miners are thinking? I'm not aware of anything that you haven't heard about before in the Bitcoin world. I don't think some people think that it is more secure than any other alternative. Is there someone who knows more than you do about these currencies?

Quote The question of how we decide should also be debated. If we don't give a single vote on the block size limits, or the BCH and XBC proposals at least, there is no room for compromise. If I am a developer of an alt chain and someone gives me permission, I will work for it. If I am an alt chain developer and this user or group says "no" then I will support them. This allows developers to have different options if all the other devs, and even some developers who think it isn't worth working with the alt chain company, have some different opinions because there's a different way a transaction or code should be built that supports the idea of the alt chain.

Quote Can we stop saying this: "Bitcoin is secure"?

Well that's because in our vision, the future blockchain uses decentralized software, but the current software currently works for all purposes. So it's an experiment. At this point, it would be wise to get a better understanding of what you are trying to achieve. For example, if you want a more secure block chain, then how to achieve

Not so impressive on my example.

Input: Dear families, As you know, we document your child's attendance each day. If your child arrives late to school

Completion:

Dear families, As you know, we document your child's attendance each day. If your child arrives late to school, for example, your parent may request you to keep an eye on their child in the hallways. They may send your child an e-mail letting you know that your child is late to school, but that the child's parents have the right to determine when school will resume.

If your child is late to school for some reason (e.g., because of an illness, accident, or disability), or if he or she doesn't seem too happy or energetic in school, your parent may send you an e-mail asking you to stay or help with their child at least that day. Parents who make this request may not make one about their children.

If your child has been at school while your parent(s) have not been; if your child is acting up or in trouble while you were out of the house; if your child is crying in the hallways of school; or if your child is missing from the hallways of school, contact your Child Protective Services if they haven't been there during your child

Wow. I give it a prompt from a story on current events...

Prompt: _The Department of Justice on Monday protested a move by House Democrats to begin contempt proceedings against Attorney General William Barr, arguing the House Judiciary Committee was rushing the issue despite Barr's willingness to quickly disclose a redacted version of special counsel Robert Mueller's report._

Output: "Barrasso sent a letter to Justice officials Monday protesting that the panel and the White House "have not determined all of the details of this investigation, nor, specifically, have they released what documents may be relevant to this inquiry. In sum, the Senate has been advised of your view of the scope and scope of that investigation, however vague it may be, and it has agreed that any further inquiry into it must be expedited. The Department of Justice has made clear that it considers all information submitted" (eol)

Oh wow, the newer model is so much better!

If anyone is interested, I have a repo for using it as a discord bot [0].

I didn't know that openAI released a newer larger version though, good to know!

[0]: https://github.com/itsmehemant123/gpt2-discord-bot

I am glad for this because it shows what the whole fuss about "dangerous" was about. This is like an illusion that looks deceptive from afar, but is however transparent on any closer inspection. As long as it is impossible for deep learning to make sense, these things are not a danger.

This is just the 345M model, not the 1.5B one...

the samples from the full model did not make sense either, although they made less obvious grammar/syntax errors.

It is pretty good at generating "meaningless song lyrics". Sing along!

Prompt:

  la la la da da la la da
  ooooh weeee oooohh
  shoop woop shoop woop de woopity

Response:

  ooooh shoop woop shoop woop de woop
  oooh oohh woof woof woof woof de woof
  oooh ooooh woof woof woof woof
  oooh ooooh woof woof woof woof
  oooooohooOOOooooo oh yeah
  oooh yeah yeah yeah yeah yeah llo hhhhhhhhhhhhhhhhhhhh
  ooohhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh
  oooooohhhh... woomp woomp woomp woomph woomph...
  oooooooohhhh... woomph woomph woomph ... woomp woomp... woomp woomp... woomp woomp... woompwoomp... woomp woomp... woomp woomp... yo woop... woop wooop wooop... woop Woop Woop woop woopwoop Woop Woop woomp woop, woopppp, woomph woomp, woopppp... WOP poop poop... nop nop nop nop nop nop nop, n...

It can rap :) silly though

(Stan, fan, pan, man!) I'm an idiot, man!

So, now this is my own place.

And I didn't come here for all this.

This is one of my own…

I'm no different.

For now, the next generation's.

I'll be here for as long as there's anyone here.

Don't worry about it, I know.

[Hook, Solo]

Solo: So!

Bran: Why the…

So the rest is up to you guys. It's going to be a fun ride.

[Outro]

The entire world is watching me now (bark)

This is gonna be a fun ride.

[MUSIC]

Hey you guys

It's me, the one and only Jon Hamm!!!

You've got no idea who I'm, right?

But you know me, Jon!

I'm good! Now do you wanna bet? You don't wanna bet $50 that I'm really Jon Hamm!!!

Jon Hamm: What a great name!

If he were to say "Barry" (laugh)

I'd get a laugh! (laugh) That's Jon Hamm to ya!

Oh and that's not even the joke

It's not the funniest thing ever!

I know how to play, I know everything.

Here's a little secret

There

Just got a "Mission to Mars" prompt that ended with

"* I use PayPal for donations.* Thanks so much, if you decide to donate, please give generously or I will donate every penny you make in the end. You can find me at my PayPal account"

My first test showed that it's surely not perfect. I admit, after all the hype, I expected my SOCKS to be blown off.

My prompt: "If Timmy is"

Response: "If timmy is iced. then iced = iced + ( " "M " + m ) where iced ="

Hmmm.

As far as I know, this is the hamstrung model they released due to concerns surrounding ethical usage of the full model.

The default model isn't that robust. Even the outputs on the official GPT-2 page took a few tries.

Though I got an interesting response with the first part of your post as prompt (prompt in italics—to be clear, it stops after the closing quote on "If Timmy is"; the rest of that line is the beginning of the generated text):

My first test showed that it's surely not perfect. I admit, after all the hype, I expected my SOCKS to be blown off.

My prompt: "If Timmy is" — an all-male chat bot

The result:

What a terrible name for such an important task.

I started digging around the web looking for examples and found this hilarious Reddit thread. A guy named Timmy was asking a question that really made no sense:

"Is Timmy gay."

He's got so much potential, but he said nothing, which was very odd for a chatbot. After I had explained the concept of a conversation, Timmy seemed a bit confused. But eventually, without a reason (or an answer) he got down to the meat of the matter.

It turns out Timmy is actually a female voiceover (VO) who is supposed to speak to you while you write your email. All of our previous SOCKS tests showed that Timmy was a male chatbot, right? This was surprising, because I was using an SOCKS version called

In the process of consuming millions of human expressions, these systems see us in a way we cannot see ourselves.

It makes me wonder if there is a style of prompt that will start to give us a glimpse of that view.

Interesting, even as an English language model, inputting prefixes of other languages shows that the model has picked up a surprising amount of foreign language structure.

Pretty interesting!

(The meaning of life can be easily determined simply by)

examining the various ways in which our lives are arranged and evaluated. What then is one's life compared to

"one plus one equals" --> "one plus one equals 12"

guess it still needs to work on its math skills :P

I went to cinema with two friends tonight. The completion was something like we saw The Phantom Menace on TV, and Sully the time before.

Definitely this model doesn't understand the world. I wonder if multiple layers of neural networks will do. Maybe that's how our brain works, not with a single model, no matter how large.

I think the humans

  I think the humans were so stupid and cowardly.

You could cache the responses for the suggested inputs to handle the load better.

Animal Crossing is one of the best Nintendo games because

This is the greatest anime in the world

What hardware are you using for this?

A quick guess - a standard server instance. It's hosted on what looks like a Digital Ocean IP, and Digital Ocean doesn't have GPU instances AFAIK; and generation is very slow, much slower than I'd expect from GPU. So this might be running on CPU, which would save a ton of money.

It was relatively speedy before it hit the front page.

(also as an aside, I do have my scalable/virtually free GPT-2 API on a CPU ready, should be released this week.)

I would be stoked to have some GPU resources from DO. I would like to play around with this kind of stuff on DO for a few days and then destroy the droplet. I have done it a few times with other OpenAI projects using CPU. It was pretty cool to see Ai play all the games in the gym. That is all, the end

The site was using one K80 GPU but it's slowing down significantly so I'm adding a second GPU. The servers run on Google Kubernetes Engine.

I'm also working on implementing that particular workflow with GPT-2/GKE/GPUs (I'm curious on your deployment strategy if you want to talk more).

You may want to use preemptible GPUs if you aren't already.

Why are the Jews hated so much?

who are you?

The name of the place is

It's like a dream come true, a dream dream come true

Input: "You shouldn't go to Australia."

Output: "There's nothing to see here. I'm going to America to eat some shit. I don't know what to believe. No one cares what I'm saying."

His comments attracted the ire of Trump's campaign manager, Kellyanne Conway.

Trump's latest comments echo past remarks by former GOP frontrunner Mitt Romney in which he said, "I think the biggest, most important thing is to not allow ISIS to take over their home country and they're going to take over other parts of the world, especially Syria."

In the post, he went on to call for a "complete and total shutdown of Muslims entering the United States" until the country can be "safe for the safety of our people".

He also said of Muslim refugees seeking refuge in the USA: "I want to be clear about this: This whole thing is about Islamic radicalization. We're going after people who would rather die than come"

Looks like it has a long way to go before it's as scary as GANs for fake images.

Yeet

Why are Jews hated so much?

Reply on news.ycombinator.com