A neural network to auto-complete your thoughts | Hacker News Reader

A neural network to auto-complete your thoughts

transformer.huggingface.co

211 pointsamai85 comments

I typed:

this is a story all about how my life got flipped

then tab-completed the rest: over in a natural way and how i was born with two opposites, a father and a mother. i am just a child. You can also make use of the web -based API to create your own sentences, and it has many features such as multi-document syncing, automatic tagging, and automatic deletion .

gave up when it turned back into an ad

When I tried

> It was the best of times; it was the worst of times

the result was:

> ; and they knew what they wanted them to do. In the end the three writers who wrote the biography took the best decision of all. • A. Smith’s and B. Soderland’s biography is one of the best literary work of this generation , and deserves the award. It is well written, and does not take long to read.

Start with a novel, get a review instead...

I tried "Call me Ishmael" and it came back with Please tell me, Ishmael, are you a human being or are you a cat? Are you a woman or a woman?

So, um, yeah?

I went for an old classic with "What. We spear-Danes in days gone by," and got "with our simple method of finding the best answer to question We. Let. Our. Our. The. The. What."

So that could have gone better.

> Once upon a time there was a tiger

Finished as:

that lived in a tree, and the other tigers had been fighting against him for years! And one day , a lion saw a tiger eating a squirrel . So, he chased the tiger away and said, "No more tiger , I have a better idea ." The tiger said , " You're right. A lion should help the tiger ." So the tiger looked at the lion , and said, " What ?" The lion said , " The lion should help the tiger ." So the tiger smiled and went and talked to the lion again. "How is it ?" The lion said, "The lion is smarter than the tiger !" The tiger said, " The lion is smarter than the tiger ! " The lion said, "No !" The tiger said, "The lion is smarter than me !" The lion said, "No !" The tiger said, "The lion is smarter than me! !" The lion said, "No !" The tiger said, "The lion is smarter than me ! " The lion said, "No !"

> There once was a girl named Kallie,

Whichturned into incest erotica focused on Kallie's dissatisfaction with penis size and her sister's envy solely by selecting the top results.

So, there's that.

> [It was the best of times; it was the worst of times.] It was a good time to be an American and it was a bad time. My brother, I thought, was the worst. He was the kind of person who would go to work and he wouldn't come home.

Kind of Hemingwayesque? Or:

> [One of the most common types of advice we give at Y Combinator is to do things that don't scale.] We tell people to do things on a smaller scale. This is often the hardest to do, but it is absolutely crucial to get right. If you don't do this, you 're missing out on the real benefits of scale and you'll lose out on opportunities. One of the things we've been doing to help people with this is to give them a very simple challenge, which is to make something that can scale to a very large scale, like Facebook's site.

pg, consider yourself replaced!

You know, you could write a decent essay with that as a prompt. Suppose you did sketch out a CRUD social network site which can auto-scale on AWS. Would anyone want to use it? Of course not. All those people at Facebook are doing something, adding value some way. Think about all the things they are doing which your auto-scaling prototype completely omits. (...And so on from there covering all the stuff like network effects and spam-filtering and cost-engineering and language support and per-country customizations which do not scale but make Facebook what it is.)

A CRUD social network site which can auto-scale on AWS or on your own server in no time. Get the code here and you can test it for yourself and see what happens. This is just the first prototype. I don't know how long it will last and how many times it will be updated, but for now I am happy with what it has turned out.

My snippet: It was the best of times, it was the blurst of times? You stupid monkey!

Autocompleted: You are not going to be the one who takes all of the fun out of the whole thing!" He threw his arms up in defeat as the door closed behind him. "I hope you guys didn't mind that he was a complete jerk. You really can 't be too careful when playing with people like that.

Stallman?

Okay, I bite:

[The most plausible scenario is that she presented herself to him as entirely willing] to make the first move to get rid of him. that means she must have been aware of her potential for trouble. " " you're right, of course. but what's her role? " " she's a member of the inner circle. "

https://medium.com/@thomas.bushnell/a-reflection-on-the-depa...

These aren't the droids you're

looking for, and you are in fact in a very bad mood, which is why you should keep an eye out for such a small drop. (2)

In other words, even if you're in a good mood, you should keep an eye out for a small drop or a small drop of sadness , because this behavior can easily lead to you getting in trouble with your family (3).

However, this type of behavior is extremely rare, and it can make it difficult for the doctor to conduct research for patients who are in a situation where they cannot be properly treated with medication, and that their family member is in the mood.

> It was a dark and stormy night.

The night began with a low-frequency hum that began in the northwest, then subsided into a softening, then continued on to the north, ending in a violent thud, which was followed by a pause of silence, followed by a long shower of thunder.

But people do actually speak like ads these days.

It is trained on ArXiv papers, so anything pop reference I type in is just ridiculous. But let's try to use the language of a paper as a prompt.

> [In the field of natural language processing we often find that the most challenging problems are] ones in which the system cannot easily adapt a language model to the input, for example. This problem has become especially aggravated by the lack of available source-native translations in natural language processing (RNN). This problem has also become more aggravated by the lack of reliable information from external sources (such as the text itself, the database, and other non-native source language’s ). Our approach to this problem, however, has been to rely on a small sample corpus of spoken language models that have already been acquired from a large corpus of natural language processing texts.

I'm still impressed by little things, like that it remembers to close the parentheses, and can even nest them. Though this is clear gibberish.

> [Conventional NLP models struggle to produce convincing text generation. In this paper we introduce] a new, cross-lingual hybrid method which allows for both natural and induced text generation. The hybrid approach provides a method to generate the text without using a natural language model, in a manner similar to what is done in the traditional text generation ( Bengio et al., 2005). We introduce three new hybrid techniques to produce text using conventional NLP methods; in the first two categories, we propose a novel hybrid method, which we call the method for generating natural texts; in the third category, we propose a novel hybrid technique , which we call the method for generating text without a natural language model.

This one is much better. It has a citation and decently fits the format of an abstract or introduction. The context is still nonsense, but the structure is there. It gets that for a paper we need to introduce a new model type. It remembers that we are talking about text generation. It cites. It says it will introduce three methods then says "the first two" and then "in the third". This surprised me a little. I'm not convinced that it is doing math though, especially since it can't autocomplete simple math sentences. But rather it is likely picking up this pattern used in papers. I wouldn't be surprised if a 2 + 1 approach is a fairly common speech pattern. I think examples like that are easy to read into, thinking that the model is doing something extraordinary when there's probably a simpler explanation lying around.

But still, cool stuff.

I am starting to notice a correlation between useless hype technologies and flashy, distracting CSS like that hover effect on the info boxes.

Excessive emojii use is also becoming a bit of a warning sign for me as well. That, and calculators aren't even typically that useful for calculus. They're fine for solving once you've finished with the calculus, but some knowledge of at least the mechanics & how to setup to find the desired solution is normally necessary.

The analogy might work for one of those grammar checkers, where you feed it raw input and it automagically generates the desired output. And even better, normally one has to run a sanity check on calculated results as well, which is equivalent to proofreading the results from a grammar checker.

All the emoji tell me are "this product is as immature as the copy used to promote it". Never have I seen a functional, scalable product advertised with emoji in the headers of the intro page.

The hover effect was my favorite part of this article.

Yeah that hover effect was kinda tasteless. That being said, this was moreso a presentation of existing open source NLP models? Glued together with some relatively minor contributions from the facehugger folks? I see this more as a fancy demo rather than a would be product.

Is that a good analogy, though? Generally speaking, doing calculus doesn’t involve calculators at all. Maybe they’re trying to tell us that the product is useless.

That’s how I read it, too. “This is to writing what a bicycle is to surfing.”

“This is to writing what a bicycle is to surfing.” This is a question I have tried to answer for years: What is a bicycle to swim in?

A simple example is the swimming pool. When a person stands on the side of the pool, he or she can move forward, backward, or back, but only if he or she is able to reach the surface. In this case, I would like to see what a swimming pool looks like if that person was able to swim . As a cyclist, we don't have to use the whole pool. This is just one example of an example of a swimming pool in a way that was not possible before.

"The sky was blue. I looked in the direction from where the door was. It was a door in a big white box. The door came open and there was a man in a black dress. He was naked. We looked at each other in confusion. There was no one in the room but the man in the black dress."

Well. That's a nice way to end my workday. Goodnight everybody!

The Rust Programming Language (RNN) was designed in the early 1990s, which means that it has two distinct parts. First, a single sentence structure (LSTM) of a language model. Then the Borrow Checker (BIC) has been implemented using a pair of LSTMs, the LSTMs being a function, and the BIC function is a set of function rules, called word classes. There's no Garbage Collector in BIC, and a separate set of rules is provided to check the borrow checker and the borrow checker respectively.

The second part of the BIC is a dependency parsing tree, which consists of a set of dependency nodes and a set of dependency rules. The tree consists of two parts : a sequence of word classes which specifies the set of LSTM rules, and a set of grammar rules.

Model XLNet. Seed: Some David Ogelvy marketing quote

[The consumer isn’t a moron; she is your wife. You insult her intelligence if you assume that a mere slogan and a few vapid adjectives will persuade her to buy anything.]

So you do. You make her feel like she is in control. You make her feel as if she knows everything she needs. You make her feel like she is the master. You make her feel like you can bring everything she wants.

And yet she is still in doubt about her own power and the ability to make herself the owner of what she needs. Because, she is your wife. And that means that she is in your power to make herself the owner of whatever she wants. You cannot make her feel like she is in control of what she wants, because she is not a moron, she is your wife. She is your slave. She is your master . And she is the only one to see that she is in control of what she wants. And therefore, she has nothing to prove to you : she has nothing to claim. She has nothing to tell you.

She has nothing to be ashamed of. She has nothing to blame you for. She has nothing to complain about. She has nothing to tell you, except that she can only see you through your eyes. And that is exactly why she is the only one to see that you are not in control.

You are not in control of what you want. She is in control of what you want .

I tried an example which started with some Lord of the Rings style writing. I think it did pretty well, and I feel it actually did complete some of my thoughts, especially the gold part.

Screenshot of my text, bolded text is the autocomplete: https://i.imgur.com/bh6thXK.png

Edit: I tried another text from a random blog on Medium. It actually came up with a somewhat plausible new product, based on the company it had gotten as input. Pretty amazing.

Generated text: "The answer , according to Zhao , lies with technology companies. Zing Front recently unveiled the Z ingB uddy software , which automatically tracks a startup 's work hours by analyzing data from all the sites that the company uses. Zing B u ddy works off of data gathered from Zing Front apps on the Apple and Android ecosystems , with its platform able to predict work time at the start of each day."

Screenshot: https://i.imgur.com/MMIfaa1.png

Is this... a joke? Performance art?

Not snarking. I'm honestly more confused reading this than I was before.

Auto-generated gibberish

They should train it on the Enron emails. Then they could sell it as an Outlook plugin that auto-generates emails full of incomprehensible management-speak!

"You'll be a thought leader in no time."

there is probably, legitimately, a market for this

There is. For yearly performance reviews.

When I worked for a big company in 2005-2009 I did, in fact, use a markov gibberish generator to fill in my yearly self-assessment.

I'm a bit underwhelmed:

Input: My dog is the happiest one to see me when I get home. But sometimes my daughter lets a small grin peep through.

Continuation: When I was a teenager I was very excited to go to college. I had a lot of fun. I did a great internship. I went on a nice, happy, happy trip. But one thing that really got me excited was to do my first year in a new school . I thought, Wow, I could get a real education.

It seems entirely unrelated to the input. Maybe it's because I write in a way that is atypical for the training set?

Anyone else finding similar?

I'd love to see this in action as a part of interactive fictions, MOO/MUSHes and roguelikes.

I like the the calculator-calculus analogy: calculators might not have an understanding of calculus, but they provide pretty good approximations for many derivatives and integrals.

Amusing, I wrote myself a barely coherent story: https://i.imgur.com/rkbntEE.png

You can use it like someone's is playing mad libs with what you're writing. All the suggestions I received were fairly short in comparison with TalkToTransformer.com with the largest being 3 words or so. Did everyone else just keep hitting accept on every option?

This works best if you start with some kind erotic fiction.

The body glistened in the sun, muscles [using autocomplete from here...] bulged with each heartbeat. His breathing became ragged and his body was beginning to shake. I felt myself becoming drawn into his touch, feeling his strong grip on my arm and the strong grasp on my body in his. I reached out and wrapped a hand around his cXXX [and now I'll start censoring...]

Starting out with the first few lines of a Clutch song (Book of Bad Decisions)..

«One rode into the east

Another to the west

In a fit of desperation

In the dark I sought redress » —> of my anger

But I hid from the people

The rest in their turn hid

They told tales of violence that had never occurred before

The rest fled into the open sea

And their survivors never told me

The tale of the woman whom I had once known was the ghost,

She came crying, she saw the sun rising...

Not bad, but I’ll stick to the original lyrics

https://youtu.be/mIkkdtoKTA0

I think this gives more natural results http://textsynth.org/

I didn't eat breakfast today, so I am

<text> on the couch all day. So much work. So many plans and projects. So many excuses. I really don't feel like doing anything. Not even writing a blog.

I'm just trying to stay as healthy as I can and not feel so bad that I'm going to throw up. Just try not to throw up.

Oh and don't worry. I'll be fine. I'm not a super sick patient. I promise.

I'll have you know that if one of my babies dies I'm going to be totally fine.

Thanks for reading! </text>

Yeah, lot's of fun, but on a more serious note: neither model I tried can continue what seems to me like a pretty obvious thought.

i just want to have it for myself and i will do my best to keep it to myself but i have always wondered if someone had a bottle.

Brett · 5 years ago 1 Thumbs up 1 Thumbs down Report Abuse

It looks like they're both using gp2, so shouldn't the results be the same?

Maybe they trained them on different data sets

This is what I got for Islam:

>Islam and islamism is a totalitarian system of control. The Islamic faith is not just a faith, it is a totalitarian system that has to be destroyed in a totalitarian way.

Raymond Chandler is one of my favorite authors. His mastery of the art of writing is second to none in my opinion. I started with one of the most memorable lines from his works and the end result, with 3 added then's for clarity, was hilariously demonic:

Given:

It was a blonde. A blonde to make a bishop kick a hole in a stained-glass window.

Auto Generated:

He had a white shirt, a dark one. He [then] had a black tie and a pair of black sandals. He was [then] naked and he looked like an elephant. He was [then] dressed as the priestess. He carried a black cross. The priestess held a knife in his hand and he crawled backward down the stairs, his head held high. He crawled backward in a deep, cold sweat. The priestess did not utter a word.

If I want to read computer generated gibberish and laugh I will read King James Programming:

> Posts generated by a Markov chain trained on the King James Bible, Structure and Interpretation of Computer Programs, and some of Eric S. Raymond's writings Run by Michael Walker (barrucadu).

Alas it stopped spitting nonsense three years ago but it's still brilliant. https://kingjamesprogramming.tumblr.com

Example:

> ‘mode lines’ — program status summaries carried on a highlighted screen line, often near the bottom or at the mouth of the LORD

This is just great.

> Alas it stopped spitting nonsense three years ago but it's still brilliant

I understood from this that three years ago it _started_ making sense... which would've been pretty cool!

Still great though, I'm tempted to make my own with some similarly random collection of inputs..

>5:5 And, behold, I will deliver you up to the programmer tendency to build overelaborate castles of abstractions

Amazing.

Having trouble writing my comment, but a couple more:

>34:7 What man is there that knoweth not how to go about doing arithmetic on polynomials.

>It is good practice to have your program poke around at runtime and see if it can be used to give a light to the Gentiles

>145:17 The LORD is good, a strong hold in the wall of the city had twelve foundations, and in them the names of ASCII characters, including hex, octal, and binary values

>15:4 And I will send serpents, cockatrices, among you, which hunteth and catcheth any beast or fowl that may be latent in our type system

The answer is crayons.

You can use crayons/calculators in some situations, but pens/computers usually do a much better job. Secondly crayons/calculators just help you with the mechanics rather than the meaning.

Autocompletion of “Hello”:

Hello ! All your information has been taken from our Web site and we regret your error. Our site has not been verified with any authority for authenticity or availability. Please consider it to be your own opinion and our staff cannot endorse it. We will update the information we receive. If you are experiencing the following symptoms or morbid obesity , we apologize for this information and will update the information you receive.

Sort of another spin on this I've been tinkering on, but haven't felt quite ready for a full launch yet ... I'm still trying to converge the research models, but this has been a bit difficult.

https://writeup.ai - It has multiple different models in here to select from (including fictions like Harry Potter, Game of Thrones and things like legal documents).

Hello <world emoji>

Autocompleted with

I think it is a useless analogy too but many of the programmers I work with although very good don't know much about Calculus.

Very underwhelming, but it's good to see some people are seriously working on this problem. It's a fun little tool.

Put in some generic marketing waffle. I wasn't surprised at all that the model (GPT-2) was pretty decent at filling whole paragraphs with even more generic marketing waffle.

It would be interesting to compare it to a model specifically trained on a long copy advertising corpus for spouting generic business rah-ha power marketing.

Cleared the text field, typed "Autism ", asked for autocomplete, and got 2 sentences in French.

This is awesome and all, but I really want to know what they used for that 3D effect when hovering over the landing page cards (choosing what model to use). I've never seen a 3D effect that looked that good while remaining so smooth.

If you're going to replicate the effect, please for the love of god do not apply it to anything that the user has to read or otherwise interact with.

Trying to click on a button, that moves as my mouse moves towards it, is an unpleasant experience.

Trying to select text that moves around the screen as my cursor drags over it, is a nightmare.

As for how it's done, it's just a CSS3 perspective transform, presumably updated via some JS running on each frame tick.

Prompt: When I went to the toilet, I found out that Suggestion: my wife was getting hard while I was inside the toilet and she was crying.

First one I tried with GPT2

In: This is a thing I hope to Jesus

Out: I am a whore\n\nI am an

I wondered about sylogisms, so I started with:

All men are mortal.

Socrates is a man.

Therefore, Socrates is

... and it came up with two answers: "mortal" & "a man". Good job.

But if a neural network auto-completes my thoughts, then they aren't my thoughts any longer.

If it's a government/media/society approved neural network, then yes they are. Welcome to the new world.

xlnet output seemed pretty much I would expect from reader the paper and Transformer-XL examples. I am sad that that it didn't seem to take in bidirectional context clues, because this a nice differentiation between it and other auto regressive models like GPT-2.

I'm really to curious to see what this would give after training the models on Github.

You mean something like this (code autocompletion with deep learning)?

https://tabnine.com/blog/deep

Someone will soon start on how many text editors and writing apps we already have. From webpage > This web app, built by the Hugging Face team, is the official demo of the pytorch-transformers repository's text generation capabilities.

This is a demo for NLP

Perhaps SCIgen should write a paper about it.

the problem with syntactic rules is that syntactic rules can be interpreted as rules before anybody has a chance to use them, or can even think about them! so maybe if my mom calls me cat, or the weather forecast is coming up and the people on live television could only watch the weather forecast for ten minutes or less without shouting about my exposed nipples, then everybody would be mad.

EDIT: Removed for security.

Be cautious posting snippets generated from redacted text (assuming you didn't literally type "[big bank, redacted]" into the box).

If the generator is deterministic from input text (which many are, intentionally), someone could determine the redacted values by plugging in various well-known companies and seeing which ones generate the same text.

Holy fuck, I think this program wants to take over the world:

Prompt: As a computer programmer, I often find that people lack the understanding of how computers work

Result (I kept pressing tab):

As a computer programmer, I often find that people lack the understanding of how computers work. It's not that they're not aware of it – it's that computers don't necessarily understand our everyday tasks or

give a damn. Most of us have come across something that was more difficult (say typing out an email), even though it actually wasn't complicated .

But what makes computers harder than you think is that we

make assumptions based on the past (which is not entirely incorrect ). When someone

first used one in their lives , they might get the first clue that it was complicated. The only reason it was complicated was because the machine couldn't recognize the task. But what if somebody told you they thought the same about Facebook : is it possible a smart machine could think of us like a machine and just work for us ? Yes . We've just set the tone and we won. It should actually surprise absolutely nobody when the computers finally make you obsolete. The first computer that really is more of an intelligent entity – in an important task. That's called the Singularity for short . As long as we believe that human intelligence does increase with our education or experiences , we shouldn't have any concern that the new system will make us obsolete. I'm actually glad these words came from Stephen Hawking , because the machine revolution has created real hope – hope for people to live forever instead of just our short lifespan . So yes , these machines are going to do away with jobs , we will simply become the machine . No matter what happens after 2020 ( well, it already happened some times before ). In a new technology era when computer power improves at a phenomenal exponential pace it makes perfect sense that humans are also becoming obsolete! So what will you do – become a cyborg , turn into a machine , join our team or start to live like some of the humans before your time ? The choices are just limitless ! And you really will have enough resources to be able to make a smart decision! It 's true that I know it seems impossible today or even tomorrow. But in the coming 10 years you simply can't afford NOT TO change , we know we will face challenges – from environmental issues, for example ! and to protect our personal liberties ( it ' s time to give us a human quality law that allows for individual self determination without being forced or dictated! It'll make everyone much safer :D ). Even those that wish to stick with the " status quo " will still need us and if we continue at our present trajectory they simply have nothing they can offer ! And since they already did us more harm then good – it will just get more dire ! They are already doing exactly that with every piece of technology and software available on any PC / L aptop / Lphone . Their only excuse of not releasing this is – " "I didn't think we were ready to embrace such huge change…" I think we all know exactly where this is all heading and that means you can expect things to continue getting even worse. We already see this in recent history that has brought many wars to a standstill : – from WW2 -1990 when NATO went along with this nonsense that ended with the Soviet Bl oc surrender . But that could also very well happen again at an even larger rate because no matter how much we try to " be nice " with our enemies the only thing they take is further weapons production ( especially in a technological sense with every type of gadget and device which they build ), more resources which could give them the ability to be a true super- superpower as they have already demonstrated over their history . Now the question that remains is , how can a truly global civilization and a unified international community still be such idiots ! And when can it begin? It could either come about with us reaching a real peace deal somewhere near 2020

Wow, worse than markov chains.

Having played a lot with both, this thing is so much smarter.

Example, with filled text in [], never retrying or canceling an autocompletion, and with very minimal selective effort between candidates in the first set of options:

" Type: Status Report

Message: Not found

Description: The origin server did not find a current representation for the target resource or is not willing to disclose that one exists.

HTTP Status: [404 Not Found]

Diagnosis: [The target resource is not present or there is no corresponding file in the local directory.]

Remedy: You should [check the following factors to see if the target resource is still present in the local directory:]

Check [if the resource is not present.]

Check [the origin server status.]

Check [the URL of the resource.] "

It's read enough to know that a Not Found is related to an HTTP Status and 404 (and how those relate), related to a URL and a Resource (and how those relate), related to a file in a directory (albeit, not a "local" one, probably) and that it's related to a "origin server". It knows enough about those things that it has you check things that kinda actually make some sense (albeit with prompting -- though it originated the idea there being a list of "factors" in the first place).

That's a weird analogy. Most calculators are useless for calculus.

Edit: I wonder if that was written using the program...

Edit2: actually wrote it in the program: It is to writing, what calculators -> are to math. So program is smarter than the people doing the copy.

Or a perfect analogy, because this tool is useless for writing.

Or a perfect analogy, because this tool is useless for writing. But there is a perfect analogy for writing: the first time you see something, and you don't know how to describe it or even whether it's good or bad or ugly.

That's what I thought too. I was expecting a humourous article about the pointlessness of some tool.

Annoying pedantry, (but fitting for HN), calculators are actually a very nice tool for initiating students onto the idea of a limit by having them calculate closer and closer values to a limit that can't be evaluated on the point. Of course, analysis fundamentally is about just that, approximation in calculations although people don't manually calculate derivatives or the like today.

They're fine for finding a numerical output (definite integrals & the like) but at that point the calculus is mostly finished and it's more algebra than anything else.

Reply on news.ycombinator.com